When ndb_filter_end processes an empty filter (no fields added), it calls
realloc(filter->elem_buf.start, 0) which frees the memory and returns NULL.
The existing code only updated the pointer if realloc
returned non-NULL, leaving elem_buf.start pointing to freed memory. This
caused a double-free crash when ndb_filter_destroy later called free() on
the dangling pointer.
Fix by explicitly setting filter->elem_buf.start to NULL when realloc
returns NULL due to zero-size allocation, and update the assertion to
allow NULL pointers for empty filters. ndb_filter_destroy already checks
for NULL before freeing.
Closes: https://github.com/damus-io/damus/issues/3634
Changelog-Fixed: Fix memory corruption crash when creating empty filters
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Three issues were causing invoices to not render or fetch:
1. bech32.c: Hardcoded MAX_PREFIX limited HRP length, but BOLT11 HRPs
can be arbitrarily long depending on amount. Now derives max HRP
length dynamically from input length (len-6 to match bolt11.c buffer).
2. content_parser.c: bolt11_decode_minimal was passed a pointer into
the content buffer without null-termination. When a note contained
multiple invoices, the decoder would read past the first invoice
into newlines and the second invoice, causing checksum failure.
Fixed by creating a null-terminated copy using strndup.
3. bolt11.c: bech32_decode_alloc allocated buffers using strlen(str)-6
and strlen(str)-8 without checking minimum length first. For inputs
shorter than 8 chars, this caused size_t underflow leading to huge
allocations and potential crash. Added early length guard.
IMPORTANT: bech32_decode callers must allocate hrp buffer of at least
strlen(input) - 6 bytes. This matches existing bolt11.c usage.
Changelog-Fixed: Fixed Lightning invoice parsing and fetching for all amounts
Closes: https://github.com/damus-io/damus/issues/3456
Closes: https://github.com/damus-io/damus/issues/3151
Signed-off-by: alltheseas <alltheseas@noreply.github.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit fixes the background crashes with termination code
0xdead10cc.
Those crashes were caused by the fact that NostrDB was being stored on
the shared app container (Because our app extensions need NostrDB
data), and iOS kills any process that holds a file lock after the
process is backgrounded.
Other developers in the field have run into similar problems in the past
(with shared SQLite databases or shared SwiftData), and they generally
recommend not to place those database in shared containers at all,
mentioning that 0xdead10cc crashes are almost inevitable otherwise:
- https://ryanashcraft.com/sqlite-databases-in-app-group-containers/
- https://inessential.com/2020/02/13/how_we_fixed_the_dreaded_0xdead10cc_cras.html
Since iOS aggressively backgrounds and terminates processes with tight
timing constraints that are mostly outside our control (despite using
Apple's recommended mechanisms, such as requesting more time to perform
closing operations), this fix aims to address the issue by a different
storage architecture.
Instead of keeping NostrDB data on the shared app container and handling
the closure/opening of the database with the app lifecycle signals, keep
the main NostrDB database file in the app's private container, and instead
take periodic read-only snapshots of NostrDB in the shared container, so as
to allow extensions to have recent NostrDB data without all the
complexities of keeping the main file in the shared container.
This does have the tradeoff that more storage will be used by NostrDB
due to file duplication, but that can be mitigated via other techniques
if necessary.
Closes: https://github.com/damus-io/damus/issues/2638
Closes: https://github.com/damus-io/damus/issues/3463
Changelog-Fixed: Fixed background crashes with error code 0xdead10cc
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
before we weren't checking this, meaning we were getting
results from other keys. oops.
Reported-by: Jeff Gardner
Fixes: #84
Signed-off-by: William Casarin <jb55@jb55.com>
Rogue relays could in theory attack nostrdb by replaying ids and
signatures from other notes. This fixes this weakness by calculating the
id again in ndb_note_verify.
There is no known relays exploiting this, but lets get ahead of it
before we switch to the outbox model in damus iOS/notedeck
Signed-off-by: William Casarin <jb55@jb55.com>
This adds some helpers for adding custom filtering logic
to nostr filters. These are just a callback and a closure.
There can only be one custom callback filter per filter.
Fixes: https://github.com/damus-io/nostrdb/issues/33
Signed-off-by: William Casarin <jb55@jb55.com>
The basic idea of this is to allow you to use the standard
nip50 query interface to search for profiles using our profile
index.
query: {"search":"jb55", "kinds":[0]}
will result in a profile_search query plan that searches kind0 profiles
for the corresponding `name` or `display_name`.
Signed-off-by: William Casarin <jb55@jb55.com>
Add support for relay-based filtering in nostr queries.
Filters can now include a "relays" field. Optimal performance when
you include a kind as well:
{"relays":["wss://pyramid.fiatjaf.com/"], "kinds":[1]}
This corresponds to a `ndb` query like so:
$ ndb query -r wss://pyramid.fiatjaf.com/ -k 1 -l 1
using filter '{"relays":["wss://pyramid.fiatjaf.com/"],"kinds":[1],"limit":1}'
1 results in 0.094929 ms
{"id":"277dd4ed26d0b44576..}
Signed-off-by: William Casarin <jb55@jb55.com>
This fixes a race condition where if multiple of the same note
is processed at the same time, we still manage to write the
note relays
Signed-off-by: William Casarin <jb55@jb55.com>
Add relay indexing for existing notes
This patch introduces a relay index for new notes and notes that have
already been stored, allowing the database to track additional relay
sources for a given note.
Changes:
- Added `NDB_WRITER_NOTE_RELAY` to handle relay indexing separately from
new note ingestion.
- Implemented `ndb_write_note_relay()` and
`ndb_write_note_relay_kind_index()` to store relay URLs.
- Modified `ndb_ingester_process_event()` to check for existing notes
and append relay info if necessary.
- Introduced `ndb_note_has_relay()` to prevent duplicate relay entries.
- Updated LMDB schema with `NDB_DB_NOTE_RELAYS` (note_id -> relay) and
`NDB_DB_NOTE_RELAY_KIND` (relay + kind + created_at -> note).
- Refactored `ndb_process_event()` to use `ndb_ingest_meta` for tracking
relay sources.
- Ensured proper memory management for relay strings in writer thread.
With this change, nostrdb can better track where notes are seen across
different relays, improving query capabilities for relay-based data
retrieval.
Signed-off-by: William Casarin <jb55@jb55.com>
This commit fixes a stack corruption issue caused by
an off-by-one error in one of the functions responsible
for parsing bech32 entities.
Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Previously two addresses from different memory regions were being
subtracted, which will lead to the incorrect number. This commit
improves the calculation.
Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
I wanted to not amend this since we've already applied
it in the nostrdb-update branch on damus and I don't want
to conflict
Signed-off-by: William Casarin <jb55@jb55.com>
The Address Sanitizer detected a heap buffer overflow during a memcpy operation
in nostrdb.c associated with note parsing.
It was found that not enough memory was being allocated to the buffer to
support all the content parsing.
Allocation size was increased to support the memory needed for the
parsing operations. However, the new number was not carefully calculated
as we will not run into this code path once we switch to the local relay
model.
Changelog-Fixed: Fixed memory error in nostrdb
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Add support for type KIND for bech32-encoded entities naddr and nevent
as specified in NIP-19.
Co-authored-by: kernelkind <kernelkind@gmail.com>
Signed-off-by: William Casarin <jb55@jb55.com>
This adds support for nip50 fulltext searches. This allows you to use
the nostrdb query interface for executing fulltext searches instead of
the typical `ndb_text_search` api. The benefits of this include a
standardized query interface that also further filters on other fields
in the filter.
Changelog-Added: Add nip50 search filters and queries
Signed-off-by: William Casarin <jb55@jb55.com>
