tyiu/damus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
4,264 Commits 139 Branches 0 Tags
eb995845018f10adf77b1fa7f354a04aeb79402e
Commit Graph

4264 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
William Casarin
0ac03df841 nostrdb: build: fix compile warning 
A small size_t/uint64 conversion issue

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
db99b4f4d4 nostrdb: fix dubious looking parens logic 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
cc9585b6e3 nostrdb: plan: use a less efficient plan for author query plans 
This is less efficient for now but we don't have a small-author-list
query plan yet.

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
bd17dcfac6 nostrdb: plan: add created_at query plan 
This introduces the basic created_at query plan. We scan the created_at
+ id index in descending order looking for items that match a filter.
This is a very general query plan, but might not be very efficient for
anything other than local timelines.

Changelog-Added: Add general created_at query plan for timelines
Closes: https://github.com/damus-io/nostrdb/issues/26
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
25e91b386c nostrdb: cores: just set to 2 on unknown platforms 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
560e9e53cd nostrdb: fix a few note size compile issues 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
1c1e5fa2a0 nostrdb: random: add getrandom fallback for android 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
2d5f86b142 nostrdb: filter: make sure clone copies metadata 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
89686d758a nostrdb: filter: make sure to return clone errors 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
6c26add1da nostrdb: filter: add ndb_filter_clone 
Clone filters when moving them into subscriptions. This will allow us to
fix the double free issue on the rust side.

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
3c5a83392e nostrdb: filter: use relative data offsets for easy cloning 
Instead of storing exact pointers inside of our filter elements, just
store offsets. This will allow us to clone filters very easily without
having to mess around with fixing up the pointers afterwards.

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
1c63c3b9bb nostrdb: filter: add ndb_filter_end 
This is a pretty scary looking function that realloc our large variable
filter buffer into a compact one. This saves up a bunch of memory when
we are done building the filter.

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
0bd4717e01 nostrdb: query: include note size in query results 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
bebd531b58 nostrdb: return number of items popped when polling 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
5788c077c4 nostrdb: silence annoying debug 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
1b77b4f0e0 nostrdb: filters: copy filter metadata into subscription 
This fixes a few ownership issues

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
62625c6ff3 nostrdb: ndb: add ndb_poll_for_notes 
The polling variant of ndb_wait_for_notes. This makes more sense for
realtime apps like notedeck

Changelog-Added: Add ndb_poll_for_notes
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
c8d88058d4 nostrdb: queue: switch to prot_queue_try_pop_all 
This allows you to `try pop` multiple items instead of 1

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
b8bef86ea1 nostrdb: port kernelkind's to the new bech32 parser 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
b128330b2a nostrdb: tce: fix build for previous TCE change 
Fixes: 34093cd1 ("tce: add AUTH to-client-event")
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
934ea80f85 nostrdb: blocks: add word count interface 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
588cebd18d nostrdb: header: add ptr helpers for swift 
swift is kind of dumb when it comes to opaque pointers

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
William Casarin
ccca6e58ec nostrdb: strblock: add typedef 
I don't technically need this but it helps a lot on the swift side
of things since I already have code that uses this identifier of a
similar structure

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:40:00 -07:00
Charlie Fish
c1befa5221 nostrdb/tce: add AUTH to-client-event 
This was committed to damus, but this should be in nostrdb or else we
will lose it when we update.

Damus: 84cfeb1604 ("nip42: add initial relay auth support")
Link: https://groups.google.com/a/damus.io/g/patches/c/Zx3dk01e0yg/m/t59TsVkXAQAJ
Signed-off-by: Charlie Fish <contact@charlie.fish>
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
8b3c86c5de nostrdb/query: add tag index and tag queries 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
05c5a6dacb nostrdb/filter: don't end field if we don't have one active 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
1a6568deca nostrdb/perf: add some flamegraph helpers to makefile 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
1b2f4c41df nostrdb/fix macos build 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
25bcf9c243 nostrdb/ndb: measure query performance 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
3993679cc0 nostrdb/query: support until for kind query plans 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
e302bf37fa nostrdb/ndb: add inital query command 
still very early, but works for kinds!

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
a45f4d3087 nostrdb/Query Plans 
Instead of running queries off filters directly, we do some simple
heuristics and determine a reasonable query plan for the given filter.

To test this, also add a kind index query plan and add a test for it.

We still need tag, author, and created_at index scans. This is up next!

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
d598e178c1 nostrdb/index: make sure kind index is DUPSORT + INTEGERDUP 
We will probably need a migration for this?

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
77601e77ee nostrdb/filter: rename FILTER_GENERIC to FILTER_TAG 
it's a bit more intuitive

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
206efba58a nostrdb/cleanup: remove old dbscan stuff 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
a84749cd07 nostrdb/debug: add print_kind_keys helper 
I needed this for debugging kind queries

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
shuoer86
099b588be2 nostrdb/Fix typos 
Closes: https://github.com/damus-io/nostrdb/pull/25
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
75c7adddb8 nostrdb/query: implement kind queries 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
9f1b9ab945 nostrdb/Initial nostrdb queries 
Still a lot more work to do, but this is at least a proof of concept for
querying nostrdb using filters.

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
b2080a946e nostrdb/cursor: fix bug when pushing last element 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
942e47a720 nostrdb/query: extract ndb_cursor_start 
This is useful for positioning LMDB cursors at the start of a query. We
will be re-using this in the upcoming query code

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
6dbf3416b9 nostrdb/cursor: remove old array code 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
2b14acd62f nostrdb/filter: don't allow adding id elements on kinds 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
267a9ac54b nostrdb/ocd: small cleanup 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
8b03ed6175 nostrdb/filters: remove ndb_filter_group from public API 
We can just use a list of filters instead when subscribing

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
6cd7b945ca nostrdb/filter: use binary search for large contact list filters 
This is much more efficient than linear scans

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:59 -07:00
William Casarin
e5e6735129 nostrdb/filter: sort filter elements 
If they are sorted we can do binary search when matching filters like
how strfry does it.

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
9c2f7a931c nostrdb/subs: always fail when calling wait_for_notes on a subid of 0 
this is an invalid subscription id

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
b1bbf355de nostrdb/subs: notify on profile notes as well 
We missed this in the original subscription code

Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00
William Casarin
d7a2064786 nostrdb/debug: add a few more debug statement 
Signed-off-by: William Casarin <jb55@jb55.com>
 2025-08-11 16:39:43 -07:00