Commit Graph

197 Commits

Author SHA1 Message Date
William Casarin
4cfe28d802 nostrdb/bech32: fix big in bech32 size parsing
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
034f2cc02f nostrdb/blocks: add note block iterator
This adds an api that walks along and pulls compact note block data out of
nostrdb.

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
kernelkind
6f9bd6c4f4 nostrdb/parser: handle period at end of url
Fix parsing URL when encountering a period at the end of the url by
setting it as disallowed from being present at the end of a
URL.

Some characters are disallowed to be present at the end of URLs.
Presently, the period character is the only disallowed character.
A character is the last character in the URL if it is followed by
is_whitespace() or if it's the last character in the string.

Signed-off-by: kernelkind <kernelkind@gmail.com>
Tested-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb5.com>
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
d73422db38 nostrdb/content_parser: add initial db decoders
We need to pull the data out as well! Let's add some initial decoders.
We still need tests to make sure it's working.

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
c3b06d281e nostrdb/bech32: add some initial tests
since we modified this recently, let's add some tests to make sure
we didn't break anything

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
1b09e9458c nostrdb/nostr_bech32: parse in one pass
since we will be decoding these in realtime, let's make sure we can
decode them in O(1)

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
e0a2dcf3db nostrdb/Inital embedded content parser
This adds some initial code for nostrdb content parsing.

We still need to write tests for encoding and decoding, so this is
likely not working yet.

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
9ff1f69a82 nostrdb/search: switch to cursor_align function
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
623b8603c2 nostrdb/cursor: add align function
handy function for padding buffers to some byte alignment

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
d8b083010d nostrdb/cursor: fix some warnings
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
887eb4e1e2 nostrdb/cursor: fix empty string pushing in push_c_str
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
b5ad3ed1a5 nostrdb/cursor: add pull_varint_u32
This is a varint helper that doesn't pull larger than uint32

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
371e9fb406 nostrdb/cursor: add malloc_slice
This is the same as cursor_slice except we don't memset afterwards

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
aa5809d792 nostrdb/nostr_bech32: only parse up to raw bech32 buffers
We will be storing raw nostr bech32 buffers directly into nostrdb, so
adapt our bech32 code to reflect this.

When doing our content parsing pass, we will only look for strings and we
won't allocate any intermediate buffers. Only when we write this string
block to nostrdb will we actually allocate in our nostrdb output buffer
(no mallocs!)

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
30ba0d72cc nostrdb/bech32: retab
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
373cd71f69 nostrdb/block: add bolt11 invoice encoding/decoding
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
acaf327a07 nostrdb/make: cleanup a bit, separate bench running
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
9f0bf7dff5 nostrdb/fix github action
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
88d7eb8a86 nostrdb/fix build
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
76862776b8 nostrdb/varint: switch to 64 bit varints
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
4c55459c1f nostrdb/test: disable migrate for now
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
f7cdc7bc31 nostrdb/cursor: re-apply infinite loop bug fix
since I keep overwriting it by accident

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
1bc4971111 nostrdb/add libnostrdb.a
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
6ce6c79160 nostrdb/add initial content parser
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
1ffbd80c67 nostrdb: move everything to src
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
1fb88a912a nostrdb: port everything over to be in as sync as possible
for now
2025-08-11 16:39:43 -07:00
William Casarin
954f48b23d c: move c files into nostrdb in prep for switchover 2025-08-11 16:39:43 -07:00
William Casarin
cc75a8450a nostrdb: add supporting files for the bolt11 parser
A lot of this was pulled from core-lightning. Not sure what is actually
needed or not.

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:43 -07:00
William Casarin
389c2c9695 nostrdb: add supporting files before the move commit 2025-08-11 16:39:42 -07:00
William Casarin
4a6121ba13 c: move compiler to nostrdb dir
we will be applying a patch here as well
2025-08-11 16:39:42 -07:00
William Casarin
a469f2e127 nostrdb/re-apply ispunct crash fix
since it was overwritten when we synced with damus

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:42 -07:00
William Casarin
2f8f18b846 nostrdb/build: fix constness on config pointer in ingester thread
otherwise build fails

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:42 -07:00
William Casarin
3a7cf4d08d nostrdb/rust: initial api for Ndb and NdbConfig
This is the start of our rust library for nostrdb. Implement idiomatic
interfaces for Ndb and NdbConfig.

Changelog-Added: Add initial rust library
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:42 -07:00
William Casarin
e3001cc240 nostrdb/cursor: fix warning that build.rs is complaining about
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:42 -07:00
William Casarin
d1ef113a8b nostrdb/api: don't expose many internals, like note
rust doesn't like packed structures, so hide this from bindgen

This also buttons up the API so less things are exposed which is good.

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:42 -07:00
William Casarin
f187f4f8f2 c: move cursor.h to nostrdb subdir
everything will be in here soon
2025-08-11 16:39:42 -07:00
William Casarin
4e9583ef54 nostrdb/stream: actually use file pointer in stream api
Right now it's accidently hardcoded.

Fixes: 8376e5bca05c ("add "import -"")
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:42 -07:00
Yasuhiro Matsumoto
cc95d5df6e nostrdb/add "import -"
Closes: https://github.com/damus-io/nostrdb/pull/21
Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:42 -07:00
William Casarin
4ca156fd83 nostrdb/build: fix additional compiler errors
When trying to build from rust

Signed-off-by: William Casarin <jb55@jb55.com>
2025-08-11 16:39:42 -07:00
ericholguin
414c67a919 Follow Packs
This PR adds and enables follow packs in the universe view.

Closes: #3012

Changelog-Added: Added follow list kind 39089
Changelog-Added: Added follow pack preview
Changelog-Added: Added follow pack timeline to Universe View
Changelog-Removed: Removed hashtags in Universe View

Signed-off-by: ericholguin <ericholguin@apache.org>
2025-06-16 10:34:18 -07:00
Daniel D’Aquino
7fa2118480 Implement Codable for NdbNote
Makes it easier to work with other Swift types

Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
2025-05-15 15:12:52 -07:00
e7fe4ab9b4 Inverse hellthread_notifications_enabled to be hellthread_notifications_disabled and add hellthread_notifications_max_pubkeys setting
Signed-off-by: Terry Yiu <git@tyiu.xyz>
2025-04-18 16:14:13 -07:00
c146bab08a Add notification setting to hide hellthreads
Changelog-Added: Add notification setting to hide hellthreads
Closes: https://github.com/damus-io/damus/issues/2943
Signed-off-by: Terry Yiu <git@tyiu.xyz>
2025-04-18 16:14:13 -07:00
Daniel D’Aquino
0ec2b05070 Implement safe interface for unowned NdbNotes
This commit introduces a new interface that makes it easier and safer to
handle unowned NostrDB notes, by leveraging new non-copyable and borrow
features from modern Swift.

Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
2025-04-16 11:48:52 -07:00
Daniel D’Aquino
09ce3af11e Add some miscellaneous documentation
This commit adds some documentation to miscellaneous functions and
classes.

Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
2025-04-16 11:48:52 -07:00
William Casarin
7d82d8b76f search: expand search results to 128
This continues our hack due to the way the compiler bridges to static
sized arrays. yes its horrible. no i don't care.

Changelog-Changed: Expand nostrdb text search results to 128 items
Signed-off-by: William Casarin <jb55@jb55.com>
2024-09-27 12:37:53 -07:00
William Casarin
cb241741e3 notifications: add more logging
needed this to debug stuff

Signed-off-by: William Casarin <jb55@jb55.com>
2024-09-05 20:02:07 -07:00
William Casarin
5d2fc0ed54 lmdb: patch semaphore names to use group container prefix
This is an attempt to fix various issues when acquiring a IPC
semaphore on iOS

See: https://github.com/damus-io/damus/issues/2323#issuecomment-2323181204

Running this patch gives us these names:

mdb_env_setup_locks: using semnames
  'group.com.damus/MDBrwDDi_FHxD' (29),
  'group.com.damus/MDBwwDDi_FHxD' (29)

From old Apple docs:

> IPC and POSIX Semaphores and Shared Memory
>
> Normally, sandboxed apps cannot use Mach IPC, POSIX semaphores and
> shared memory, or UNIX domain sockets (usefully). However, by specifying
> an entitlement that requests membership in an application group, an app
> can use these technologies to communicate with other members of that
> application group.
>
> Note: System V semaphores are not supported in sandboxed apps.
>
> UNIX domain sockets are straightforward; they work just like any other
> file.
>
> Any semaphore or Mach port that you wish to access within a sandboxed
> app must be named according to a special convention:
>
> POSIX semaphores and shared memory names must begin with the application
> group identifier, followed by a slash (/), followed by a name of your
> choosing.
>
> Mach port names must begin with the application group identifier,
> followed by a period (.), followed by a name of your choosing.
>
> For example, if your application group’s name is
> Z123456789.com.example.app-group, you might create two semaphores named
> Z123456789.myappgroup/rdyllwflg and Z123456789.myappgroup/bluwhtflg. You
> might create a Mach port named
> Z123456789.com.example.app-group.Port_of_Kobe.
>
> Note: The maximum length of a POSIX semaphore name is only 31 bytes, so
> if you need to use POSIX semaphores, you should keep your app group
> names short.

Link: https://github.com/damus-io/damus/issues/2323#issuecomment-2323305949
Signed-off-by: William Casarin <jb55@jb55.com>
2024-09-01 06:42:35 -07:00
Daniel D’Aquino
f9271da11c Add support for rendering highlights with comments
This commit implements rendering comments from the `["comment",
<COMMENT_TEXT>]` tag in a highlight note.

Comment contents get rendered like a kind 1 note's "content" field

This commit also adds the `r` "reference" tag as a standard tag reference type

Changelog-Added: Add support for rendering highlights with comments
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
2024-08-17 15:58:43 -07:00
ericholguin
b2ba1e0e3b highlight: fixes and improvements
This patch allows highlights to be included in posts as well as removes context
when creating a highlight. Highlights now route as the root and selecting the
highlight in root routes to the highlighted event.

Signed-off-by: ericholguin <ericholguin@apache.org>
2024-07-19 10:53:26 -07:00