This commit implements a new Storage settings view that displays storage
usage statistics for NostrDB, snapshot database, and Kingfisher image cache.
Key features:
- Interactive pie chart visualization (iOS 17+) with tap-to-select functionality
- Pull-to-refresh gesture to recalculate storage
- Categorized list showing each storage type with size and percentage
- Total storage sum displayed at bottom
- Conditional compilation for iOS 16/17+ compatibility
- All calculations run on background thread to avoid blocking main thread
- NostrDB storage breakdown
Changelog-Added: Storage usage statistics view in Settings
Changelog-Changed: Moved clear cache button to storage settings
Closes: https://github.com/damus-io/damus/issues/3649
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
When ndb_filter_end processes an empty filter (no fields added), it calls
realloc(filter->elem_buf.start, 0) which frees the memory and returns NULL.
The existing code only updated the pointer if realloc
returned non-NULL, leaving elem_buf.start pointing to freed memory. This
caused a double-free crash when ndb_filter_destroy later called free() on
the dangling pointer.
Fix by explicitly setting filter->elem_buf.start to NULL when realloc
returns NULL due to zero-size allocation, and update the assertion to
allow NULL pointers for empty filters. ndb_filter_destroy already checks
for NULL before freeing.
Closes: https://github.com/damus-io/damus/issues/3634
Changelog-Fixed: Fix memory corruption crash when creating empty filters
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
NLLanguageRecognizer.processString() is an expensive NLP operation that
was moved to @MainActor in 5058fb33, causing UI jank when scrolling.
This moves language detection back to the async preload path where it
runs off the main thread. get_preload_plan is synchronous again and
defers language computation to preload_event. Translations still happen
on the first preload pass since preload_event now checks .havent_tried
directly rather than relying on the load_translations flag from the plan.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract and use relay hints from bech32 entities (nevent, nprofile, naddr)
and event tag references (e, q tags) to fetch events from hinted relays
not in the user's relay pool.
Changes:
- Parse relay hints from bech32 TLV data in URLHandler
- Pass relay hints through SearchType and NoteReference enums
- Add ensureConnected() to RelayPool for ephemeral relay connections
- Implement ephemeral relay lease management with race condition protection
- Add repostTarget() helper to extract relay hints from repost e tags
- Add QuoteRef struct to preserve relay hints from q tags (NIP-10/NIP-18)
- Support relay hints in replies with author pubkey in e-tags (NIP-10)
- Implement fallback broadcast when hinted relays don't respond
- Add comprehensive test coverage for relay hint functionality
- Add DEBUG logging for relay hint tracing during development
Implementation details:
- Connect to hinted relays as ephemeral, returning early when first connects
- Use total deadline to prevent timeout accumulation across hint attempts
- Decrement lease count before suspension points to ensure atomicity
- Fall back to broadcast if hints don't resolve or respond
Closes: https://github.com/damus-io/damus/issues/1147
Changelog-Added: Added relay hint support for nevent, nprofile, naddr links and event tag references (reposts, quotes, replies)
Signed-off-by: alltheseas
Signed-off-by: Daniel D'Aquino <daniel@daquino.me>
Co-authored-by: alltheseas <alltheseas@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Daniel D'Aquino <daniel@daquino.me
- Mentions.swift: convert_invoice_description now returns non-optional
InvoiceDescription, returning empty description for BOLT11 compliance
(both description and description_hash are optional per spec)
- Block.swift, NdbBlock.swift, NostrEvent.swift, NoteContent.swift:
Updated call sites to use non-optional invoice conversion
- InvoiceTests.swift: Added test for specific failing invoice
Signed-off-by: alltheseas <alltheseas@noreply.github.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Three issues were causing invoices to not render or fetch:
1. bech32.c: Hardcoded MAX_PREFIX limited HRP length, but BOLT11 HRPs
can be arbitrarily long depending on amount. Now derives max HRP
length dynamically from input length (len-6 to match bolt11.c buffer).
2. content_parser.c: bolt11_decode_minimal was passed a pointer into
the content buffer without null-termination. When a note contained
multiple invoices, the decoder would read past the first invoice
into newlines and the second invoice, causing checksum failure.
Fixed by creating a null-terminated copy using strndup.
3. bolt11.c: bech32_decode_alloc allocated buffers using strlen(str)-6
and strlen(str)-8 without checking minimum length first. For inputs
shorter than 8 chars, this caused size_t underflow leading to huge
allocations and potential crash. Added early length guard.
IMPORTANT: bech32_decode callers must allocate hrp buffer of at least
strlen(input) - 6 bytes. This matches existing bolt11.c usage.
Changelog-Fixed: Fixed Lightning invoice parsing and fetching for all amounts
Closes: https://github.com/damus-io/damus/issues/3456
Closes: https://github.com/damus-io/damus/issues/3151
Signed-off-by: alltheseas <alltheseas@noreply.github.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements an actor-based preloading system to efficiently fetch profile
metadata for note authors and referenced users. The EntityPreloader queues
requests and batches them intelligently (500 pubkeys or 1 second timeout)
to avoid network overload while improving UX by ensuring profiles are
available when rendering notes.
Key changes:
- Add EntityPreloader actor with queue-based batching logic
- Integrate with SubscriptionManager via PreloadStrategy enum
- Add lifecycle management (start/stop on app foreground/background)
- Skip preload for pubkeys already cached in ndb
- Include comprehensive test suite with 11 test cases covering batching,
deduplication, and edge cases
- Optimize ProfilePicView to load from ndb before first render
Closes: https://github.com/damus-io/damus/issues/gh-3511
Changelog-Added: Profile metadata preloading for improved timeline performance
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This makes negentropy optimizations available to the rest of the app via
Subscription Manager.
Changelog not needed because this should not have user-facing changes
Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This fixes an arithmetic overflow crash on iOS 17 caused by the fallback
lock.
In iOS 17, Swift Mutexes are not available, so we have a fallback class
for NdbUseLock that does not make use of them. This allows some thread
safety for iOS 17 users, but unfortunately not as much as iOS 18+ users.
This attempts to fix those remaining race conditions and subsequent
crashes by using `NSLock` in the fallback class, which is available on
iOS 17.
Closes: https://github.com/damus-io/damus/issues/3512
Changelog-Fixed: Fixed a crash on iOS 17 that would happen on startup
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit fixes the background crashes with termination code
0xdead10cc.
Those crashes were caused by the fact that NostrDB was being stored on
the shared app container (Because our app extensions need NostrDB
data), and iOS kills any process that holds a file lock after the
process is backgrounded.
Other developers in the field have run into similar problems in the past
(with shared SQLite databases or shared SwiftData), and they generally
recommend not to place those database in shared containers at all,
mentioning that 0xdead10cc crashes are almost inevitable otherwise:
- https://ryanashcraft.com/sqlite-databases-in-app-group-containers/
- https://inessential.com/2020/02/13/how_we_fixed_the_dreaded_0xdead10cc_cras.html
Since iOS aggressively backgrounds and terminates processes with tight
timing constraints that are mostly outside our control (despite using
Apple's recommended mechanisms, such as requesting more time to perform
closing operations), this fix aims to address the issue by a different
storage architecture.
Instead of keeping NostrDB data on the shared app container and handling
the closure/opening of the database with the app lifecycle signals, keep
the main NostrDB database file in the app's private container, and instead
take periodic read-only snapshots of NostrDB in the shared container, so as
to allow extensions to have recent NostrDB data without all the
complexities of keeping the main file in the shared container.
This does have the tradeoff that more storage will be used by NostrDB
due to file duplication, but that can be mitigated via other techniques
if necessary.
Closes: https://github.com/damus-io/damus/issues/2638
Closes: https://github.com/damus-io/damus/issues/3463
Changelog-Fixed: Fixed background crashes with error code 0xdead10cc
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This adds a sync mechanism in Ndb.swift to coordinate certain usage of
nostrdb.c calls and the need to close nostrdb due to app lifecycle
requirements. Furthermore, it fixes the order of operations when
re-opening NostrDB, to avoid race conditions where a query uses an older
Ndb generation.
This sync mechanism allows multiple queries to happen simultaneously
(from the Swift-side), while preventing ndb from simultaneously closing
during such usages. It also does that while keeping the Ndb interface
sync and nonisolated, which keeps the API easy to use from
Swift/SwiftUI and allows for parallel operations to occur.
If Swift Actors were to be used (e.g. creating an NdbActor), the Ndb.swift
interface would change in such a way that it would propagate the need for
several changes throughout the codebase, including loading logic in
some ViewModels. Furthermore, it would likely decrease performance by
forcing Ndb.swift operations to run sequentially when they could run in
parallel.
Changelog-Fixed: Fixed crashes that happened when the app went into background mode
Closes: https://github.com/damus-io/damus/issues/3245
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit redesigns the Ndb.swift interface with a focus on build-time
safety against crashes.
It removes the external usage of NdbTxn and SafeNdbTxn, restricting it
to be used only in NostrDB internal code.
This prevents dangerous and crash prone usages throughout the app, such
as holding transactions in a variable in an async function (which can
cause thread-based reference counting to incorrectly deinit inherited
transactions in use by separate callers), as well as holding unsafe
unowned values longer than the lifetime of their corresponding
transactions.
Closes: https://github.com/damus-io/damus/issues/3364
Changelog-Fixed: Fixed several crashes throughout the app
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
We have mechanisms in place to close NostrDB streams when the database needs to
close; however, there is a short time window where those streams are
closing down but the database still has its "open" property set to `true`,
which means that new NostrDB streams may open. If that happens, those
new streams will still be active when NostrDB gets closed down,
potentially causing memory crashes.
This was found by inspecting several crash logs and noticing that:
- most of the `ndb.close` calls are coming from the general
backgrounding task (not the last resort backgrounding task),
where all old tasks are guaranteed to have closed (we wait for all of
them to close before proceeding to closing NostrDB).
- the stack traces of the crashed threads show that, in most cases, the
stream crashes while they are in the query stage (which means that
those must have been very recently opened).
The issue was mitigated by signalling that NostrDB has closed (without
actually closing it) before cancelling any streaming tasks and officially
closing NostrDB. This way, new NostrDB streaming tasks will notice that
the database is closed and will wait for it to reopen.
No changelog entry is needed as this issue was introduced after our last public
release.
Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This fixes a crash that would occasionally occur when visiting profiles.
NdbTxn objects were being deinitialized on different threads from their
initialization, causing incorrect reference count decrements in thread-local
transaction dictionaries. This led to premature destruction of shared ndb_txn
C objects still in use by other tasks, resulting in use-after-free crashes.
The root cause is that Swift does not guarantee tasks resume on the same
thread after await suspension points, while NdbTxn's init/deinit rely on
thread-local storage to track inherited transaction reference counts.
This means that `NdbTxn` objects cannot be used in async functions, as
that may cause the garbage collector to deinitialize `NdbTxn` at the end
of such function, which may be running on a different thread at that
point, causing the issue explained above.
The fix in this case is to eliminate the `async` version of the
`NdbNoteLender.borrow` method, and update usages to utilize other
available methods.
Note: This is a rewrite of the fix in https://github.com/damus-io/damus/pull/3329
Note 2: This relates to the fix of an unreleased feature, so therefore no
changelog is needed.
Changelog-None
Co-authored-by: alltheseas <64376233+alltheseas@users.noreply.github.com>
Closes: https://github.com/damus-io/damus/issues/3327
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This PR adds Live Streaming and Live Chat to Damus via Damus Labs.
Changelog-Added: Added live stream timeline
Changelog-Added: Added live chat timeline
Changelog-Added: Added ability to create live chat event
Changelog-Added: Damus Labs Toggle
Signed-off-by: ericholguin <ericholguin@apache.org>
A race condition was identified where notes would get dropped if they
get indexed in the time window between when a query is made and the subscription is made.
The issue was fixed by making the subscribe call before making the query
call, to ensure we get all notes from that time when we perform the
query.
This dropped the failure rate for ndb subscription tests from about 20%
down to about 4%.
Local relay model issue was not publicly released, which is why the
changelog entry is "none".
Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This attempts to reduce race conditions coming from Ndb streaming
functions that could lead to lost notes or crashes.
It does so by making two improvements:
1. Instead of callbacks, now the callback handler uses async streams,
which reduces the chances of a callback being called before the last
item was processed by the consumer.
2. The callback handler will now queue up received notes if there are
no listeners yet. This is helpful because we need to issue the
subscribe call to nostrdb before getting the subscription id and
setting up a listener, but in between that time nostrdb may still
send notes which would effectively get dropped without this queuing
mechanism.
Changelog-Fixed: Improved robustness in the part of the code that streams notes from nostrdb
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This is a large refactor that aims to improve performance by offloading
RelayPool computations into a separate actor outside the main thread.
This should reduce congestion on the main thread and thus improve UI
performance.
Also, the internal subscription callback mechanism was changed to use
AsyncStreams to prevent race conditions newly found in that area of the
code.
Changelog-Fixed: Added performance improvements to timeline scrolling
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
- Resend subscription requests to relays when websocket connection is
re-established
- More safeguard checks on whether Ndb is opened before accessing its
memory
- Cancel queued unsubscribe requests on app backgrounding to avoid race
conditions with subscribe requests when app enters the foreground
- Call Ndb re-open when Damus is active (not only on active notify), as
experimentally there have been instances where active notify code has
not been run. The operation is idempotent, so there should be no risk
of it being called twice.
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This should prevent background crashes caused by race conditions between
usages of Ndb and the Ndb/app lifecycle operations.
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit takes a step back from the full local relay model by
treating NostrDB as one of the many relays streamed from, instead of the
one exclusive relay that other classes rely on.
This was done to reduce regression risk from the local relay model
migration, without discarding the migration work already done.
The full "local relay model" behavior (exclusive NDB streaming) was
hidden behind a feature flag for easy migration later on.
Closes: https://github.com/damus-io/damus/issues/3225
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
before we weren't checking this, meaning we were getting
results from other keys. oops.
Reported-by: Jeff Gardner
Fixes: #84
Signed-off-by: William Casarin <jb55@jb55.com>
Rogue relays could in theory attack nostrdb by replaying ids and
signatures from other notes. This fixes this weakness by calculating the
id again in ndb_note_verify.
There is no known relays exploiting this, but lets get ahead of it
before we switch to the outbox model in damus iOS/notedeck
Signed-off-by: William Casarin <jb55@jb55.com>
This adds some helpers for adding custom filtering logic
to nostr filters. These are just a callback and a closure.
There can only be one custom callback filter per filter.
Fixes: https://github.com/damus-io/nostrdb/issues/33
Signed-off-by: William Casarin <jb55@jb55.com>
The basic idea of this is to allow you to use the standard
nip50 query interface to search for profiles using our profile
index.
query: {"search":"jb55", "kinds":[0]}
will result in a profile_search query plan that searches kind0 profiles
for the corresponding `name` or `display_name`.
Signed-off-by: William Casarin <jb55@jb55.com>
Add support for relay-based filtering in nostr queries.
Filters can now include a "relays" field. Optimal performance when
you include a kind as well:
{"relays":["wss://pyramid.fiatjaf.com/"], "kinds":[1]}
This corresponds to a `ndb` query like so:
$ ndb query -r wss://pyramid.fiatjaf.com/ -k 1 -l 1
using filter '{"relays":["wss://pyramid.fiatjaf.com/"],"kinds":[1],"limit":1}'
1 results in 0.094929 ms
{"id":"277dd4ed26d0b44576..}
Signed-off-by: William Casarin <jb55@jb55.com>
