Extract and use relay hints from bech32 entities (nevent, nprofile, naddr)
and event tag references (e, q tags) to fetch events from hinted relays
not in the user's relay pool.
Changes:
- Parse relay hints from bech32 TLV data in URLHandler
- Pass relay hints through SearchType and NoteReference enums
- Add ensureConnected() to RelayPool for ephemeral relay connections
- Implement ephemeral relay lease management with race condition protection
- Add repostTarget() helper to extract relay hints from repost e tags
- Add QuoteRef struct to preserve relay hints from q tags (NIP-10/NIP-18)
- Support relay hints in replies with author pubkey in e-tags (NIP-10)
- Implement fallback broadcast when hinted relays don't respond
- Add comprehensive test coverage for relay hint functionality
- Add DEBUG logging for relay hint tracing during development
Implementation details:
- Connect to hinted relays as ephemeral, returning early when first connects
- Use total deadline to prevent timeout accumulation across hint attempts
- Decrement lease count before suspension points to ensure atomicity
- Fall back to broadcast if hints don't resolve or respond
Closes: https://github.com/damus-io/damus/issues/1147
Changelog-Added: Added relay hint support for nevent, nprofile, naddr links and event tag references (reposts, quotes, replies)
Signed-off-by: alltheseas
Signed-off-by: Daniel D'Aquino <daniel@daquino.me>
Co-authored-by: alltheseas <alltheseas@users.noreply.github.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Daniel D'Aquino <daniel@daquino.me
- Mentions.swift: convert_invoice_description now returns non-optional
InvoiceDescription, returning empty description for BOLT11 compliance
(both description and description_hash are optional per spec)
- Block.swift, NdbBlock.swift, NostrEvent.swift, NoteContent.swift:
Updated call sites to use non-optional invoice conversion
- InvoiceTests.swift: Added test for specific failing invoice
Signed-off-by: alltheseas <alltheseas@noreply.github.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit addresses a race condition that happened when the user
initializes the app on the universe view, where the loading function
would run before the relay list was fully loaded and connected, causing
the loading function to connect to an empty relay list.
The issue was fixed by introducing a call that allows callers to wait
for the app to connect to the network
Changelog-Fixed: Fixed issue where the app would occasionally launch an empty universe view
Closes: https://github.com/damus-io/damus/issues/3528
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Implements an actor-based preloading system to efficiently fetch profile
metadata for note authors and referenced users. The EntityPreloader queues
requests and batches them intelligently (500 pubkeys or 1 second timeout)
to avoid network overload while improving UX by ensuring profiles are
available when rendering notes.
Key changes:
- Add EntityPreloader actor with queue-based batching logic
- Integrate with SubscriptionManager via PreloadStrategy enum
- Add lifecycle management (start/stop on app foreground/background)
- Skip preload for pubkeys already cached in ndb
- Include comprehensive test suite with 11 test cases covering batching,
deduplication, and edge cases
- Optimize ProfilePicView to load from ndb before first render
Closes: https://github.com/damus-io/damus/issues/gh-3511
Changelog-Added: Profile metadata preloading for improved timeline performance
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit improves the ndb snapshot logic by only transferring desired
notes instead of copying the entire database, which could be as big as
10GB.
Closes: https://github.com/damus-io/damus/issues/3502
Changelog-Changed: Improved storage efficiency for NostrDB on extensions
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This makes negentropy optimizations available to the rest of the app via
Subscription Manager.
Changelog not needed because this should not have user-facing changes
Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This implements some useful functions to use negentropy from RelayPool,
but does not integrate them with the rest of the app.
No changelog for the negentropy support right now as it is not hooked up
to any user-facing feature
Changelog-Fixed: Fixed a race condition in the networking logic that could cause notes to get missed in certain rare scenarios
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit fixes a crash that occurred when clicking "follow all"
during onboarding.
This fix works by making `Contacts` and `PostBox` isolated into a
specific Swift Actor, and updating direct and indirect usages
accordingly.
Changelog-Fixed: Fixed a crash that occurred when clicking "follow all" during onboarding.
Closes: https://github.com/damus-io/damus/issues/3422
Co-authored-by: alltheseas <64376233+alltheseas@users.noreply.github.com>
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit fixes the background crashes with termination code
0xdead10cc.
Those crashes were caused by the fact that NostrDB was being stored on
the shared app container (Because our app extensions need NostrDB
data), and iOS kills any process that holds a file lock after the
process is backgrounded.
Other developers in the field have run into similar problems in the past
(with shared SQLite databases or shared SwiftData), and they generally
recommend not to place those database in shared containers at all,
mentioning that 0xdead10cc crashes are almost inevitable otherwise:
- https://ryanashcraft.com/sqlite-databases-in-app-group-containers/
- https://inessential.com/2020/02/13/how_we_fixed_the_dreaded_0xdead10cc_cras.html
Since iOS aggressively backgrounds and terminates processes with tight
timing constraints that are mostly outside our control (despite using
Apple's recommended mechanisms, such as requesting more time to perform
closing operations), this fix aims to address the issue by a different
storage architecture.
Instead of keeping NostrDB data on the shared app container and handling
the closure/opening of the database with the app lifecycle signals, keep
the main NostrDB database file in the app's private container, and instead
take periodic read-only snapshots of NostrDB in the shared container, so as
to allow extensions to have recent NostrDB data without all the
complexities of keeping the main file in the shared container.
This does have the tradeoff that more storage will be used by NostrDB
due to file duplication, but that can be mitigated via other techniques
if necessary.
Closes: https://github.com/damus-io/damus/issues/2638
Closes: https://github.com/damus-io/damus/issues/3463
Changelog-Fixed: Fixed background crashes with error code 0xdead10cc
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This adds a sync mechanism in Ndb.swift to coordinate certain usage of
nostrdb.c calls and the need to close nostrdb due to app lifecycle
requirements. Furthermore, it fixes the order of operations when
re-opening NostrDB, to avoid race conditions where a query uses an older
Ndb generation.
This sync mechanism allows multiple queries to happen simultaneously
(from the Swift-side), while preventing ndb from simultaneously closing
during such usages. It also does that while keeping the Ndb interface
sync and nonisolated, which keeps the API easy to use from
Swift/SwiftUI and allows for parallel operations to occur.
If Swift Actors were to be used (e.g. creating an NdbActor), the Ndb.swift
interface would change in such a way that it would propagate the need for
several changes throughout the codebase, including loading logic in
some ViewModels. Furthermore, it would likely decrease performance by
forcing Ndb.swift operations to run sequentially when they could run in
parallel.
Changelog-Fixed: Fixed crashes that happened when the app went into background mode
Closes: https://github.com/damus-io/damus/issues/3245
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
When a view subscribes to profile updates via streamProfile() or
streamProfiles(), the stream now immediately yields any existing
profile data from NostrDB before waiting for network updates.
Previously, subscribers had to wait up to ~1 second for the
subscriptionSwitcherTask to restart the profile listener before
receiving any data. During this window, views would display
abbreviated pubkeys (e.g., "npub1abc...") or robohash placeholders
instead of the cached profile name and picture.
The fix adds a simple NDB lookup when creating the stream. This has
negligible performance impact since:
- It's a one-time operation per subscription (not per update)
- The same lookup was already happening in view bodies anyway
- NDB lookups are fast local queries
A new `yieldCached` parameter (default: true) allows callers to opt
out of the initial cached emission. NoteContentView uses this to
avoid redundant artifact re-renders — it only needs network updates
since its initial render already uses cached profile data.
Furthermore, when a profile has no metadata, the display name now shows
"npub1yrse...q9ye" instead of "1yrsedhw:8q0pq9ye" for a better UX.
Closes: https://github.com/damus-io/damus/issues/3454
Closes: https://github.com/damus-io/damus/issues/3455
Changelog-Changed: Changed abbreviated pubkey format to npub1...xyz for better readability
Changelog-Fixed: Fixed instances where a profile would not display profile name and picture for a few seconds
Signed-off-by: alltheseas <64376233+alltheseas@users.noreply.github.com>
Co-authored-by: Daniel D’Aquino <daniel@daquino.me>
During profiling, I found that some large hangs were being caused by a
large number of `notify` calls (and their handling functions) keeping
the main thread overly busy.
We cannot move the `notify` mechanism to a background thread (It has to
be done on the main actor or else runtime warnings/errors appear), so
instead this commit removes a very large source of notify calls/handling around
NoteContentView, and replaces it with a background task that streams
for profile updates and only updates its view when a relevant profile is
updated.
Changelog-Changed: Improved performance around note content views to prevent hangs
Closes: https://github.com/damus-io/damus/issues/3439
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit removes an accidentally placed code comment. Behavior has
been tested during the original work related to that commit.
Fixes: b562b930cc
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit redesigns the Ndb.swift interface with a focus on build-time
safety against crashes.
It removes the external usage of NdbTxn and SafeNdbTxn, restricting it
to be used only in NostrDB internal code.
This prevents dangerous and crash prone usages throughout the app, such
as holding transactions in a variable in an async function (which can
cause thread-based reference counting to incorrectly deinit inherited
transactions in use by separate callers), as well as holding unsafe
unowned values longer than the lifetime of their corresponding
transactions.
Closes: https://github.com/damus-io/damus/issues/3364
Changelog-Fixed: Fixed several crashes throughout the app
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
We have mechanisms in place to close NostrDB streams when the database needs to
close; however, there is a short time window where those streams are
closing down but the database still has its "open" property set to `true`,
which means that new NostrDB streams may open. If that happens, those
new streams will still be active when NostrDB gets closed down,
potentially causing memory crashes.
This was found by inspecting several crash logs and noticing that:
- most of the `ndb.close` calls are coming from the general
backgrounding task (not the last resort backgrounding task),
where all old tasks are guaranteed to have closed (we wait for all of
them to close before proceeding to closing NostrDB).
- the stack traces of the crashed threads show that, in most cases, the
stream crashes while they are in the query stage (which means that
those must have been very recently opened).
The issue was mitigated by signalling that NostrDB has closed (without
actually closing it) before cancelling any streaming tasks and officially
closing NostrDB. This way, new NostrDB streaming tasks will notice that
the database is closed and will wait for it to reopen.
No changelog entry is needed as this issue was introduced after our last public
release.
Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This fixes a crash that would occasionally occur when visiting profiles.
NdbTxn objects were being deinitialized on different threads from their
initialization, causing incorrect reference count decrements in thread-local
transaction dictionaries. This led to premature destruction of shared ndb_txn
C objects still in use by other tasks, resulting in use-after-free crashes.
The root cause is that Swift does not guarantee tasks resume on the same
thread after await suspension points, while NdbTxn's init/deinit rely on
thread-local storage to track inherited transaction reference counts.
This means that `NdbTxn` objects cannot be used in async functions, as
that may cause the garbage collector to deinitialize `NdbTxn` at the end
of such function, which may be running on a different thread at that
point, causing the issue explained above.
The fix in this case is to eliminate the `async` version of the
`NdbNoteLender.borrow` method, and update usages to utilize other
available methods.
Note: This is a rewrite of the fix in https://github.com/damus-io/damus/pull/3329
Note 2: This relates to the fix of an unreleased feature, so therefore no
changelog is needed.
Changelog-None
Co-authored-by: alltheseas <64376233+alltheseas@users.noreply.github.com>
Closes: https://github.com/damus-io/damus/issues/3327
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This PR adds Live Streaming and Live Chat to Damus via Damus Labs.
Changelog-Added: Added live stream timeline
Changelog-Added: Added live chat timeline
Changelog-Added: Added ability to create live chat event
Changelog-Added: Damus Labs Toggle
Signed-off-by: ericholguin <ericholguin@apache.org>
This commit adds a new event subscription task in HomeModel, one which
streams low volume but important filters from NostrDB.
This was done to address an issue where the contact filters in the
general handler task could yield too many notes from NostrDB, hitting
its limits and clipping off important events such as mute-lists, leading
to downstream issues such as unintended mute-list overrides.
This issue was not present since the last public release, therefore no
changelog entry is needed.
Changelog-None
Closes: https://github.com/damus-io/damus/issues/3256
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This may negatively impact performance, but improves accuracy and
prevents profile loading issues
Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This fixes a hang during sign-up, which was caused by a change in
RelayPool handling code that would only send data to handlers with
matching subscription IDs.
It so happens that some handlers want to receive all notes, and they set
the filters to `nil` to achieve that.
Furthermore, some sign-up networking code was moved to prevent race conditions.
No changelog entry because the behaviour was not changed since the last
public release.
Changelog-None
Closes: https://github.com/damus-io/damus/issues/3254
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This is a large refactor that aims to improve performance by offloading
RelayPool computations into a separate actor outside the main thread.
This should reduce congestion on the main thread and thus improve UI
performance.
Also, the internal subscription callback mechanism was changed to use
AsyncStreams to prevent race conditions newly found in that area of the
code.
Changelog-Fixed: Added performance improvements to timeline scrolling
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This improves upon a temporary fix we had for the RelayPool race
condition that would cause timeline staleness.
The root cause was that during app launch, the HomeModel would subscribe
to some filters, and the subscribe function would filter out any relays
not yet connected to avoid unnecessary waiting for EOSEs from disconnected relays.
However, that filtering would cause the subscribe request to not be
queued up or sent back to the relays once connected, causing the relays
to never receive those subscription requests and causing timeline
staleness.
This was fixed by separating the relay list used for the subcription
request from the relay list used for waiting for network EOSEs. This
allows other mechanisms to ensure the subscription will go through even
when the app is initializing and relays are not yet fully connected.
Fixes: 61eb833239
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Previously, we combined the ndb and network stream within a "session
subscription" stream, which was teared down and rebuilt every time the
app went into the background and back to the foreground (This was done to
prevent crashes related to access to Ndb memory when Ndb is closed).
However, this caused complications and instability on the network
stream, leading to timeline staleness.
To address this, the pipeline was modified to merge the ndb and network
streams further upstream, on the multi-session stage, allowing the
session subscription streams to be completely split between Ndb and the
network.
For the ndb stream, we still tear it down and bring it up along the app
foreground state, to prevent memory crashes. However, the network stream
is kept intact between sessions, since RelayPool will now automatically
handle resubscription on websocket reconnection. This prevents
complexity and potential race conditions that could lead to timeline
staleness.
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
- Resend subscription requests to relays when websocket connection is
re-established
- More safeguard checks on whether Ndb is opened before accessing its
memory
- Cancel queued unsubscribe requests on app backgrounding to avoid race
conditions with subscribe requests when app enters the foreground
- Call Ndb re-open when Damus is active (not only on active notify), as
experimentally there have been instances where active notify code has
not been run. The operation is idempotent, so there should be no risk
of it being called twice.
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Use Apple's unified logging system, and specify proper privacy levels
for each piece of information.
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit adds more safeguards to prevent RUNNINGBOARD 0xdead10cc
crashes, by:
1. Using the `beginBackgroundTask(withName:expirationHandler:)` to
request additional background execution time before completely
suspending the app. See https://developer.apple.com/documentation/xcode/sigkill
2. Reorganizing app closing/cleanup tasks to be done in parallel when
possible to decrease time needed to cleanup resources.
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This should prevent RUNNINGBOARD 0xdead10cc crashes related to
ProfileManager and app background states.
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Through some local experimentation, it seems that network relays can support higher subscription limits.
Increase internal limits to avoid hitting issues with subscriptions
waiting on subscription pool to clear and appearing stale.
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit improves the loading speed for the home timeline (and likely
other areas of the app) by employing various techniques and changes:
- Network EOSE timeout reduced from 10 seconds down to 5 seconds
- Network EOSE does not wait on relays with broken connections
- Offload HomeModel handler event processing to separate tasks to
avoid a large backlog
- Give SubscriptionManager streamers more fine-grained EOSE signals for
local optimization
- Only wait for Ndb EOSE on the home timeline for faster loading
- Add logging with time elapsed measurements for easier identification of
loading problems
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This is done to prevent hang ups when the device is offline.
Changelog-Added: Added the ability to load saved notes if device is offline
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit takes a step back from the full local relay model by
treating NostrDB as one of the many relays streamed from, instead of the
one exclusive relay that other classes rely on.
This was done to reduce regression risk from the local relay model
migration, without discarding the migration work already done.
The full "local relay model" behavior (exclusive NDB streaming) was
hidden behind a feature flag for easy migration later on.
Closes: https://github.com/damus-io/damus/issues/3225
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit improves NostrNetworkManager interfaces to be easier to use,
and with more options on how to read data from the Nostr network
This reduces the amount of duplicate logic in handling streams, and also
prevents possible common mistakes when using the standard subscribe method.
This fixes an issue with the mute list manager (which prompted for this
interface improvement, as the root cause is similar to other similar
issues).
Closes: https://github.com/damus-io/damus/issues/3221
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit implements nostr network subscriptions that survive between
sessions, as well as improved handling of RelayPool opening/closing with
respect to the app lifecycle.
This prevents stale data after users swap out and back into Damus.
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
