tyiu/damus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,108 Commits 139 Branches 0 Tags
c2383060aaa2726f522c1be32afbf3de9f8af01d
Commit Graph

4108 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
William Casarin
c2383060aa nostrdb/blocks: add ndb_blocks_flags function 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
432cdb96d9 nostrdb/fix: don't write the owned flag to the DB 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
f580c7dd93 nostrdb/fix clang compile issue 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
c677233dcb nostrdb/blocks: expose block iterator internals 
so we don't need heap allocation. we will be calling this a lot in tight
render loops, we don't want to be allocating on each frame.

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
d063362bd7 nostrdb/blocks: write note blocks on ingest 
When ingesting notes, parse text/longform contents and store them in nostrdb.

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
088683696a nostrdb/blocks: actually set the note block version 
Version 1 to start

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
f2795aa71c nostrdb/blocks: add ndb_blocks_free 
In some situations we will need to have owned note blocks. For
example, when we try to fetch note blocks from the database and it's not
there yet. We will need to parse the content on the spot and return an
owned copy, since it will not be immediately available in the database.

Add a new flag field to note blocks that lets us know if it's owned by
malloc or nostrdb.

We the add a free function that checks this flag and frees the object if
its set. If it is not set then it doesn nothing because it likely came
from the database.

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
c831976078 nostrdb/blocks: add total_size 
Fix this mistake that we have with ndb_notes where we don't know the
total size of the object

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
c2c73c3af6 nostrdb/header: move bech32 around 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
971fa3e4ef nostrdb/invoice: fix crash in any-amount invoice parsing 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
dfa145dd4a nostrdb/parser: fix bech32 block decoding 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
4cfe28d802 nostrdb/bech32: fix big in bech32 size parsing 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
034f2cc02f nostrdb/blocks: add note block iterator 
This adds an api that walks along and pulls compact note block data out of
nostrdb.

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
kernelkind
6f9bd6c4f4 nostrdb/parser: handle period at end of url 
Fix parsing URL when encountering a period at the end of the url by
setting it as disallowed from being present at the end of a
URL.

Some characters are disallowed to be present at the end of URLs.
Presently, the period character is the only disallowed character.
A character is the last character in the URL if it is followed by
is_whitespace() or if it's the last character in the string.

Signed-off-by: kernelkind <kernelkind@gmail.com>
Tested-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb5.com>
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
d73422db38 nostrdb/content_parser: add initial db decoders 
We need to pull the data out as well! Let's add some initial decoders.
We still need tests to make sure it's working.

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
c3b06d281e nostrdb/bech32: add some initial tests 
since we modified this recently, let's add some tests to make sure
we didn't break anything

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
1b09e9458c nostrdb/nostr_bech32: parse in one pass 
since we will be decoding these in realtime, let's make sure we can
decode them in O(1)

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
e0a2dcf3db nostrdb/Inital embedded content parser 
This adds some initial code for nostrdb content parsing.

We still need to write tests for encoding and decoding, so this is
likely not working yet.

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
9ff1f69a82 nostrdb/search: switch to cursor_align function 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
623b8603c2 nostrdb/cursor: add align function 
handy function for padding buffers to some byte alignment

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
d8b083010d nostrdb/cursor: fix some warnings 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
887eb4e1e2 nostrdb/cursor: fix empty string pushing in push_c_str 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
b5ad3ed1a5 nostrdb/cursor: add pull_varint_u32 
This is a varint helper that doesn't pull larger than uint32

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
371e9fb406 nostrdb/cursor: add malloc_slice 
This is the same as cursor_slice except we don't memset afterwards

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
aa5809d792 nostrdb/nostr_bech32: only parse up to raw bech32 buffers 
We will be storing raw nostr bech32 buffers directly into nostrdb, so
adapt our bech32 code to reflect this.

When doing our content parsing pass, we will only look for strings and we
won't allocate any intermediate buffers. Only when we write this string
block to nostrdb will we actually allocate in our nostrdb output buffer
(no mallocs!)

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
30ba0d72cc nostrdb/bech32: retab 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
373cd71f69 nostrdb/block: add bolt11 invoice encoding/decoding 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
acaf327a07 nostrdb/make: cleanup a bit, separate bench running 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
9f0bf7dff5 nostrdb/fix github action 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
88d7eb8a86 nostrdb/fix build 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
76862776b8 nostrdb/varint: switch to 64 bit varints 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
4c55459c1f nostrdb/test: disable migrate for now 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
f7cdc7bc31 nostrdb/cursor: re-apply infinite loop bug fix 
since I keep overwriting it by accident

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
1bc4971111 nostrdb/add libnostrdb.a 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
6ce6c79160 nostrdb/add initial content parser 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
1ffbd80c67 nostrdb: move everything to src 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
1fb88a912a nostrdb: port everything over to be in as sync as possible 
for now
 2025-08-11 16:39:43 -07:00
William Casarin
954f48b23d c: move c files into nostrdb in prep for switchover 2025-08-11 16:39:43 -07:00
William Casarin
cc75a8450a nostrdb: add supporting files for the bolt11 parser 
A lot of this was pulled from core-lightning. Not sure what is actually
needed or not.

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
389c2c9695 nostrdb: add supporting files before the move commit 2025-08-11 16:39:42 -07:00
William Casarin
4a6121ba13 c: move compiler to nostrdb dir 
we will be applying a patch here as well
 2025-08-11 16:39:42 -07:00
William Casarin
a469f2e127 nostrdb/re-apply ispunct crash fix 
since it was overwritten when we synced with damus

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:42 -07:00
William Casarin
2f8f18b846 nostrdb/build: fix constness on config pointer in ingester thread 
otherwise build fails

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:42 -07:00
William Casarin
3a7cf4d08d nostrdb/rust: initial api for Ndb and NdbConfig 
This is the start of our rust library for nostrdb. Implement idiomatic
interfaces for Ndb and NdbConfig.

Changelog-Added: Add initial rust library
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:42 -07:00
William Casarin
e3001cc240 nostrdb/cursor: fix warning that build.rs is complaining about 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:42 -07:00
William Casarin
d1ef113a8b nostrdb/api: don't expose many internals, like note 
rust doesn't like packed structures, so hide this from bindgen

This also buttons up the API so less things are exposed which is good.

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:42 -07:00
William Casarin
f187f4f8f2 c: move cursor.h to nostrdb subdir 
everything will be in here soon
 2025-08-11 16:39:42 -07:00
William Casarin
4e9583ef54 nostrdb/stream: actually use file pointer in stream api 
Right now it's accidently hardcoded.

Fixes: 8376e5bca05c ("add "import -"")
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:42 -07:00
Yasuhiro Matsumoto
cc95d5df6e nostrdb/add "import -" 
Closes: https://github.com/damus-io/nostrdb/pull/21
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:42 -07:00
William Casarin
4ca156fd83 nostrdb/build: fix additional compiler errors 
When trying to build from rust

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:42 -07:00