- Resend subscription requests to relays when websocket connection is
re-established
- More safeguard checks on whether Ndb is opened before accessing its
memory
- Cancel queued unsubscribe requests on app backgrounding to avoid race
conditions with subscribe requests when app enters the foreground
- Call Ndb re-open when Damus is active (not only on active notify), as
experimentally there have been instances where active notify code has
not been run. The operation is idempotent, so there should be no risk
of it being called twice.
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This should prevent background crashes caused by race conditions between
usages of Ndb and the Ndb/app lifecycle operations.
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit takes a step back from the full local relay model by
treating NostrDB as one of the many relays streamed from, instead of the
one exclusive relay that other classes rely on.
This was done to reduce regression risk from the local relay model
migration, without discarding the migration work already done.
The full "local relay model" behavior (exclusive NDB streaming) was
hidden behind a feature flag for easy migration later on.
Closes: https://github.com/damus-io/damus/issues/3225
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
before we weren't checking this, meaning we were getting
results from other keys. oops.
Reported-by: Jeff Gardner
Fixes: #84
Signed-off-by: William Casarin <jb55@jb55.com>
Rogue relays could in theory attack nostrdb by replaying ids and
signatures from other notes. This fixes this weakness by calculating the
id again in ndb_note_verify.
There is no known relays exploiting this, but lets get ahead of it
before we switch to the outbox model in damus iOS/notedeck
Signed-off-by: William Casarin <jb55@jb55.com>
This adds some helpers for adding custom filtering logic
to nostr filters. These are just a callback and a closure.
There can only be one custom callback filter per filter.
Fixes: https://github.com/damus-io/nostrdb/issues/33
Signed-off-by: William Casarin <jb55@jb55.com>
The basic idea of this is to allow you to use the standard
nip50 query interface to search for profiles using our profile
index.
query: {"search":"jb55", "kinds":[0]}
will result in a profile_search query plan that searches kind0 profiles
for the corresponding `name` or `display_name`.
Signed-off-by: William Casarin <jb55@jb55.com>
Add support for relay-based filtering in nostr queries.
Filters can now include a "relays" field. Optimal performance when
you include a kind as well:
{"relays":["wss://pyramid.fiatjaf.com/"], "kinds":[1]}
This corresponds to a `ndb` query like so:
$ ndb query -r wss://pyramid.fiatjaf.com/ -k 1 -l 1
using filter '{"relays":["wss://pyramid.fiatjaf.com/"],"kinds":[1],"limit":1}'
1 results in 0.094929 ms
{"id":"277dd4ed26d0b44576..}
Signed-off-by: William Casarin <jb55@jb55.com>
This fixes a race condition where if multiple of the same note
is processed at the same time, we still manage to write the
note relays
Signed-off-by: William Casarin <jb55@jb55.com>
Add relay indexing for existing notes
This patch introduces a relay index for new notes and notes that have
already been stored, allowing the database to track additional relay
sources for a given note.
Changes:
- Added `NDB_WRITER_NOTE_RELAY` to handle relay indexing separately from
new note ingestion.
- Implemented `ndb_write_note_relay()` and
`ndb_write_note_relay_kind_index()` to store relay URLs.
- Modified `ndb_ingester_process_event()` to check for existing notes
and append relay info if necessary.
- Introduced `ndb_note_has_relay()` to prevent duplicate relay entries.
- Updated LMDB schema with `NDB_DB_NOTE_RELAYS` (note_id -> relay) and
`NDB_DB_NOTE_RELAY_KIND` (relay + kind + created_at -> note).
- Refactored `ndb_process_event()` to use `ndb_ingest_meta` for tracking
relay sources.
- Ensured proper memory management for relay strings in writer thread.
With this change, nostrdb can better track where notes are seen across
different relays, improving query capabilities for relay-based data
retrieval.
Signed-off-by: William Casarin <jb55@jb55.com>
This commit improves NostrNetworkManager interfaces to be easier to use,
and with more options on how to read data from the Nostr network
This reduces the amount of duplicate logic in handling streams, and also
prevents possible common mistakes when using the standard subscribe method.
This fixes an issue with the mute list manager (which prompted for this
interface improvement, as the root cause is similar to other similar
issues).
Closes: https://github.com/damus-io/damus/issues/3221
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit implements nostr network subscriptions that survive between
sessions, as well as improved handling of RelayPool opening/closing with
respect to the app lifecycle.
This prevents stale data after users swap out and back into Damus.
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit fixes a crash that caused the app to crash when getting all
the follows from a profile.
This issue was caused by a use-after-free memory error on inherited
transactions after the original transaction is deinitialized.
The issue was fixed by introducing a reference count on all transactions
and only deallocating the C transaction when the ref count goes to zero.
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
The widespread usage of the SubscriptionManager caused new crashes to
occur when swapping apps.
This was caused due to an access to Ndb memory after Ndb has been closed
from the app background signal.
The issue was fixed with improved task management logic and ensuring all
subscription tasks are finished before closing Ndb.
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit introduces a verification step at the relay connection
level, to help ensure notes get validated at the source and prevent
security issues associated with untrusted relays.
`RelayConnection.swift` — the source that initially handles WebSocket
messages — was analyzed, and measures were put in place to prevent
(or at least minimize) unverified nostr event data being spread
throughout the app.
The following measures were taken:
1. A note verification step was added prior to the `self.handleEvent(.nostr_event(ev))` call (which sends a Nostr response to the rest of the app for logical handling).
a. From code analysis, there is only one such call in `RelayConnection.swift`.
2. `NostrConnectionEvent`, the object that gets passed to event handlers, had its interface modified to remove the "message" case, since:
a. that could be a source of unverified nostr events.
b. it is redundant an unneeded due to the `.nostr_event` case.
c. there were no usages of it around the codebase
3. The raw websocket event handler had its label renamed to "handleUnverifiedWSEvent", to make it clear to the caller about the verification status of the data.
a. Usages of this were inspected and no significant risk was detected.
4. A new `verify` method in NdbNote was created to verify Nostr notes, and unit tests were added to confirm tampering detections around all the major fields in a Nostr note.
5. Care was taken to ensure the performance regression is as little as
possible.
It is worth noting that we will not need this once the local relay model
architecture is introduced, since that architecture ensures note
validation before it reaches the rest of the application and the user.
In other words, this is a temporary fix.
However, since the migration to that new architecture is a major
undertaking that will take some time to be completed, this fix was written
in order to address security concerns while the migration is unfinished.
This fix was written in a way that attempts to be as effective as
possible in reducing security risks without a risky and lenghty
refactor of the code that would delay the fix from being published.
Changelog-Fixed: Improved security around note validation
Closes: https://github.com/damus-io/damus/issues/1341
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit fixes a crash that occurred when swapping between Damus and
other apps.
When Damus enters background mode, NostrDB is closed and its resources
released. When Damus re-enters foreground mode, NostrDB is reopened.
However, an issue with the transaction inheritance logic
caused a race condition where a side menu profile lookup would get an
obsolete transaction containing pointers that have been freedwhen
NostrDB was closed, causing a "use-after-free" memory error.
The issue was fixed by improving the transaction inheritance logic to
double-check if the "generation" counter (which auto increments when
Damus closes and re-opens) matches the generation marked on the
thread-specific transaction. This effectively prevents lookups from
inheriting an obsolete transaction from a previous NostrDB generation.
Closes: https://github.com/damus-io/damus/issues/3167
Changelog-Fixed: Fixed an issue where the app would crash when swapping between apps
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
