c2383060aa
nostrdb/blocks: add ndb_blocks_flags function
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
432cdb96d9
nostrdb/fix: don't write the owned flag to the DB
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
f580c7dd93
nostrdb/fix clang compile issue
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
c677233dcb
nostrdb/blocks: expose block iterator internals
...
so we don't need heap allocation. we will be calling this a lot in tight
render loops, we don't want to be allocating on each frame.
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
d063362bd7
nostrdb/blocks: write note blocks on ingest
...
When ingesting notes, parse text/longform contents and store them in nostrdb.
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
088683696a
nostrdb/blocks: actually set the note block version
...
Version 1 to start
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
f2795aa71c
nostrdb/blocks: add ndb_blocks_free
...
In some situations we will need to have owned note blocks. For
example, when we try to fetch note blocks from the database and it's not
there yet. We will need to parse the content on the spot and return an
owned copy, since it will not be immediately available in the database.
Add a new flag field to note blocks that lets us know if it's owned by
malloc or nostrdb.
We the add a free function that checks this flag and frees the object if
its set. If it is not set then it doesn nothing because it likely came
from the database.
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
c831976078
nostrdb/blocks: add total_size
...
Fix this mistake that we have with ndb_notes where we don't know the
total size of the object
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
c2c73c3af6
nostrdb/header: move bech32 around
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
971fa3e4ef
nostrdb/invoice: fix crash in any-amount invoice parsing
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
dfa145dd4a
nostrdb/parser: fix bech32 block decoding
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
4cfe28d802
nostrdb/bech32: fix big in bech32 size parsing
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
034f2cc02f
nostrdb/blocks: add note block iterator
...
This adds an api that walks along and pulls compact note block data out of
nostrdb.
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
6f9bd6c4f4
nostrdb/parser: handle period at end of url
...
Fix parsing URL when encountering a period at the end of the url by
setting it as disallowed from being present at the end of a
URL.
Some characters are disallowed to be present at the end of URLs.
Presently, the period character is the only disallowed character.
A character is the last character in the URL if it is followed by
is_whitespace() or if it's the last character in the string.
Signed-off-by: kernelkind <kernelkind@gmail.com >
Tested-by: William Casarin <jb55@jb55.com >
Signed-off-by: William Casarin <jb55@jb5.com >
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
d73422db38
nostrdb/content_parser: add initial db decoders
...
We need to pull the data out as well! Let's add some initial decoders.
We still need tests to make sure it's working.
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
c3b06d281e
nostrdb/bech32: add some initial tests
...
since we modified this recently, let's add some tests to make sure
we didn't break anything
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
1b09e9458c
nostrdb/nostr_bech32: parse in one pass
...
since we will be decoding these in realtime, let's make sure we can
decode them in O(1)
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
e0a2dcf3db
nostrdb/Inital embedded content parser
...
This adds some initial code for nostrdb content parsing.
We still need to write tests for encoding and decoding, so this is
likely not working yet.
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
9ff1f69a82
nostrdb/search: switch to cursor_align function
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
623b8603c2
nostrdb/cursor: add align function
...
handy function for padding buffers to some byte alignment
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
d8b083010d
nostrdb/cursor: fix some warnings
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
887eb4e1e2
nostrdb/cursor: fix empty string pushing in push_c_str
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
b5ad3ed1a5
nostrdb/cursor: add pull_varint_u32
...
This is a varint helper that doesn't pull larger than uint32
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
371e9fb406
nostrdb/cursor: add malloc_slice
...
This is the same as cursor_slice except we don't memset afterwards
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
aa5809d792
nostrdb/nostr_bech32: only parse up to raw bech32 buffers
...
We will be storing raw nostr bech32 buffers directly into nostrdb, so
adapt our bech32 code to reflect this.
When doing our content parsing pass, we will only look for strings and we
won't allocate any intermediate buffers. Only when we write this string
block to nostrdb will we actually allocate in our nostrdb output buffer
(no mallocs!)
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
30ba0d72cc
nostrdb/bech32: retab
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
373cd71f69
nostrdb/block: add bolt11 invoice encoding/decoding
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
acaf327a07
nostrdb/make: cleanup a bit, separate bench running
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
9f0bf7dff5
nostrdb/fix github action
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
88d7eb8a86
nostrdb/fix build
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
76862776b8
nostrdb/varint: switch to 64 bit varints
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
4c55459c1f
nostrdb/test: disable migrate for now
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
f7cdc7bc31
nostrdb/cursor: re-apply infinite loop bug fix
...
since I keep overwriting it by accident
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
1bc4971111
nostrdb/add libnostrdb.a
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
6ce6c79160
nostrdb/add initial content parser
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
1ffbd80c67
nostrdb: move everything to src
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
1fb88a912a
nostrdb: port everything over to be in as sync as possible
...
for now
2025-08-11 16:39:43 -07:00
954f48b23d
c: move c files into nostrdb in prep for switchover
2025-08-11 16:39:43 -07:00
cc75a8450a
nostrdb: add supporting files for the bolt11 parser
...
A lot of this was pulled from core-lightning. Not sure what is actually
needed or not.
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:43 -07:00
389c2c9695
nostrdb: add supporting files before the move commit
2025-08-11 16:39:42 -07:00
4a6121ba13
c: move compiler to nostrdb dir
...
we will be applying a patch here as well
2025-08-11 16:39:42 -07:00
a469f2e127
nostrdb/re-apply ispunct crash fix
...
since it was overwritten when we synced with damus
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:42 -07:00
2f8f18b846
nostrdb/build: fix constness on config pointer in ingester thread
...
otherwise build fails
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:42 -07:00
3a7cf4d08d
nostrdb/rust: initial api for Ndb and NdbConfig
...
This is the start of our rust library for nostrdb. Implement idiomatic
interfaces for Ndb and NdbConfig.
Changelog-Added: Add initial rust library
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:42 -07:00
e3001cc240
nostrdb/cursor: fix warning that build.rs is complaining about
...
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:42 -07:00
d1ef113a8b
nostrdb/api: don't expose many internals, like note
...
rust doesn't like packed structures, so hide this from bindgen
This also buttons up the API so less things are exposed which is good.
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:42 -07:00
f187f4f8f2
c: move cursor.h to nostrdb subdir
...
everything will be in here soon
2025-08-11 16:39:42 -07:00
4e9583ef54
nostrdb/stream: actually use file pointer in stream api
...
Right now it's accidently hardcoded.
Fixes: 8376e5bca05c ("add "import -"")
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:42 -07:00
cc95d5df6e
nostrdb/add "import -"
...
Closes: https://github.com/damus-io/nostrdb/pull/21
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:42 -07:00
4ca156fd83
nostrdb/build: fix additional compiler errors
...
When trying to build from rust
Signed-off-by: William Casarin <jb55@jb55.com >
2025-08-11 16:39:42 -07:00
William Casarin