This adds a sync mechanism in Ndb.swift to coordinate certain usage of
nostrdb.c calls and the need to close nostrdb due to app lifecycle
requirements. Furthermore, it fixes the order of operations when
re-opening NostrDB, to avoid race conditions where a query uses an older
Ndb generation.
This sync mechanism allows multiple queries to happen simultaneously
(from the Swift-side), while preventing ndb from simultaneously closing
during such usages. It also does that while keeping the Ndb interface
sync and nonisolated, which keeps the API easy to use from
Swift/SwiftUI and allows for parallel operations to occur.
If Swift Actors were to be used (e.g. creating an NdbActor), the Ndb.swift
interface would change in such a way that it would propagate the need for
several changes throughout the codebase, including loading logic in
some ViewModels. Furthermore, it would likely decrease performance by
forcing Ndb.swift operations to run sequentially when they could run in
parallel.
Changelog-Fixed: Fixed crashes that happened when the app went into background mode
Closes: https://github.com/damus-io/damus/issues/3245
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit redesigns the Ndb.swift interface with a focus on build-time
safety against crashes.
It removes the external usage of NdbTxn and SafeNdbTxn, restricting it
to be used only in NostrDB internal code.
This prevents dangerous and crash prone usages throughout the app, such
as holding transactions in a variable in an async function (which can
cause thread-based reference counting to incorrectly deinit inherited
transactions in use by separate callers), as well as holding unsafe
unowned values longer than the lifetime of their corresponding
transactions.
Closes: https://github.com/damus-io/damus/issues/3364
Changelog-Fixed: Fixed several crashes throughout the app
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This PR adds Live Streaming and Live Chat to Damus via Damus Labs.
Changelog-Added: Added live stream timeline
Changelog-Added: Added live chat timeline
Changelog-Added: Added ability to create live chat event
Changelog-Added: Damus Labs Toggle
Signed-off-by: ericholguin <ericholguin@apache.org>
This commit introduces a verification step at the relay connection
level, to help ensure notes get validated at the source and prevent
security issues associated with untrusted relays.
`RelayConnection.swift` — the source that initially handles WebSocket
messages — was analyzed, and measures were put in place to prevent
(or at least minimize) unverified nostr event data being spread
throughout the app.
The following measures were taken:
1. A note verification step was added prior to the `self.handleEvent(.nostr_event(ev))` call (which sends a Nostr response to the rest of the app for logical handling).
a. From code analysis, there is only one such call in `RelayConnection.swift`.
2. `NostrConnectionEvent`, the object that gets passed to event handlers, had its interface modified to remove the "message" case, since:
a. that could be a source of unverified nostr events.
b. it is redundant an unneeded due to the `.nostr_event` case.
c. there were no usages of it around the codebase
3. The raw websocket event handler had its label renamed to "handleUnverifiedWSEvent", to make it clear to the caller about the verification status of the data.
a. Usages of this were inspected and no significant risk was detected.
4. A new `verify` method in NdbNote was created to verify Nostr notes, and unit tests were added to confirm tampering detections around all the major fields in a Nostr note.
5. Care was taken to ensure the performance regression is as little as
possible.
It is worth noting that we will not need this once the local relay model
architecture is introduced, since that architecture ensures note
validation before it reaches the rest of the application and the user.
In other words, this is a temporary fix.
However, since the migration to that new architecture is a major
undertaking that will take some time to be completed, this fix was written
in order to address security concerns while the migration is unfinished.
This fix was written in a way that attempts to be as effective as
possible in reducing security risks without a risky and lenghty
refactor of the code that would delay the fix from being published.
Changelog-Fixed: Improved security around note validation
Closes: https://github.com/damus-io/damus/issues/1341
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Currently NostrDB does not seem to handle encryption/decryption of DMs.
Since NostrDB now controls the block parsing process and fetches note
contents directly from the database, we have to add a specific condition
that injects decrypted content directly to the ndb content parser.
This is done in conjunction with some minor refactoring to `NdbBlocks`
and associated structs, as in C those are separated between the content
string and the offsets for each block, but in Swift this is more
ergonomically represented as a standalone/self-containing object.
No changelog entry is added because the previously broken version was
never released to the public, and therefore this fix produces no
user-facing changes compared to the last released version.
Changelog-None
Closes: https://github.com/damus-io/damus/issues/3106
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit introduces new interfaces for working with NostrDB from
Swift, including `NostrFilter` conversion, subscription streaming via
AsyncStreams and lookup/wait functions.
No user-facing changes.
Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit implements rendering comments from the `["comment",
<COMMENT_TEXT>]` tag in a highlight note.
Comment contents get rendered like a kind 1 note's "content" field
This commit also adds the `r` "reference" tag as a standard tag reference type
Changelog-Added: Add support for rendering highlights with comments
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This patch allows highlights to be included in posts as well as removes context
when creating a highlight. Highlights now route as the root and selecting the
highlight in root routes to the highlighted event.
Signed-off-by: ericholguin <ericholguin@apache.org>
This removes EventRefs alltogether and uses the form we use in Damus
Android.
This simplifies our ThreadReply logic and fixes a reply-to-root bug
Reported-by: NotBiebs <justinbieber@stemstr.app>
Changelog-Fixed: Fix thread bug where a quote isn't picked up as a reply
Signed-off-by: William Casarin <jb55@jb55.com>
This should drastically increase compatibility for damus replies in
other clients.
Also filter non-pubkey references when replying so we don't run into the
q-tag bug.
Changelog-Added: Added nip10 marker replies
Changelog-Fixed: Fixed issue where some replies were including the q tag
Fixes: https://github.com/damus-io/damus/issues/2239
Fixes: https://github.com/damus-io/damus/issues/2233
Signed-off-by: William Casarin <jb55@jb55.com>
This patch adds a maybe_get_content method to NdbNote which returns an
optional string instead of “*failed to decrypt content*” on DM
decryption failure.
This method will be used by the MutelistManager in future patches.
Lighting-Address: fishcharlie@strike.me
Signed-off-by: Charlie Fish <contact@charlie.fish>
Reviewed-by: William Casarin <jb55@jb55.com>
Link: 20240210163650.42884-2-contact@charlie.fish
Signed-off-by: William Casarin <jb55@jb55.com>
This patch depends on: Adding new structs/enums for new mute list
- Rewrites Lists.swift to use new mute list option
- This leads to a lot of changes for changing the type from RefId to the new MuteItem
- Update & relay new mute list in AddMuteItemView.swift (fixing previous patch TODO)
- Renames `list` to `list_deprecated`
- We need to keep this since existing users might have an old mute list
Related: https://github.com/damus-io/damus/issues/1718
Related: https://github.com/damus-io/damus/issues/856
Lighting Address: fishcharlie@strike.me
Signed-off-by: Charlie Fish <contact@charlie.fish>
Reviewed-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
This patch depends on: Adding new structs/enums for new mute list
- Rewrites Lists.swift to use new mute list option
- This leads to a lot of changes for changing the type from RefId to the new MuteItem
- Update & relay new mute list in AddMuteItemView.swift (fixing previous patch TODO)
- Renames `list` to `list_deprecated`
- We need to keep this since existing users might have an old mute list
Related: https://github.com/damus-io/damus/issues/1718
Related: https://github.com/damus-io/damus/issues/856
Lighting Address: fishcharlie@strike.me
Signed-off-by: Charlie Fish <contact@charlie.fish>
Reviewed-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
This commit brings key local notification logic into the notification
extension target to allow the extension to reuse much of the
functionality surrounding the processing and formatting of
notifications. More specifically, the functions
`process_local_notification` and `create_local_notification` were
brought into the extension target.
This will enable us to reuse much of the pre-existing notification logic
(and avoid having to reimplement all of that)
However, those functions had high dependencies on other parts of the
code, so significant refactorings were needed to make this happen:
- `create_local_notification` and `process_local_notification` had its
function signatures changed to avoid the need to `DamusState` (which
pulls too many other dependecies)
- Other necessary dependencies, such as `Profiles`, `UserSettingsStore`
had to be pulled into the extension target. Subsequently,
sub-dependencies of those items had to be pulled in as well
- In several cases, files were split to avoid pulling too many
dependencies (e.g. Some Model files depended on some functions in View
files, so in those cases I moved those functions into their own
separate file to avoid pulling in view logic into the extension
target)
- Notification processing logic was changed a bit to remove dependency
on `EventCache` in favor of using ndb directly (As instructed in a
TODO comment in EventCache, and because EventCache has too many other
dependencies)
tldr: A LOT of things were moved around, a bit of logic was changed
around local notifications to avoid using `EventCache`, but otherwise
this commit is meant to be a no-op without any new features or
user-facing functional changes.
Testing
-------
Device: iPhone 15 Pro
iOS: 17.0.1
Damus: This commit
Coverage:
1. Ran unit tests to check for regressions (none detected)
2. Launched the app and navigated around and did some interactions to
perform a quick functional smoke test (no regressions found)
3. Sent a few push notifications to check they still work as expected (PASS)
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: Terry Yiu <git@tyiu.xyz>
Reviewed-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Changelog-Fixed: Add workaround to fix note language recognition and reduce wasteful translation requests
Many different parts of the codebase could be opening transactions when
somewhere higher in the heirarchy on the main thread might already have
an active transaction. This can lead to failed transaction opening which
is bad.
Instead of relying on passing down the transaction to subviews, lets
keep track of the active transactions in a thread-local dictionary. That
way whenever we create a new transaction we can inherit the one that is
already active in the current thread.
Inherited transactions don't end the query when they are garbage
collected, we still expect the first-opened query to do this.
This change includes several source files related to NostrDB into the extension target as well, so that we can use it from that context (and thus enable more advanced push notification formatting and suppression)
To make this change possible, I had to split some source files as well as to move some functions to different files, to ensure we don't have to pull too much unnecessary code into the extension.
Testing
-------
PASS
Device: iPhone 15 Pro simulator
iOS: 17.0.1
Damus: This commit
Test steps:
1. Build DamusNotificationService. Should succeed. PASS
2. Build Damus (the app). PASS
3. Run app, scroll around some notes, go to a few different views, post a note. Should work as normal. PASS
This change addresses an issue where notes with quotes sometimes are not loaded correctly because the quoted note was not available in the same relay. Now whenever a user posts a note with a quoted note, the quoted note is also broadcast to the user's selected relays.
Issue repro
-----------
ISSUE REPRODUCED
Device: iPhone 14 Pro Simulator
iOS: 17.0
Damus: `1fabd4c0fe98d1f47b1fa0f76984ad78095bd49c`
Setup:
- Make sure you have a debugger connected
- Have a test note that you can quote
Steps:
1. Start Damus and let logs settle
2. Observe where the last log is
3. Quote the test note
4. Copy newly generated logs and paste on a text editor.
5. Analyze those logs. Pay attention to the new note id, as well as the note id of the quoted event (`["q", <QUOTED_NOTE_ID>]`)
Results: Logs show that the newly posted event is being flushed to the relays, but not the note that is being quoted.
Testing of the fix
------------------
PASS
Device: iPhone 14 Pro Simulator
iOS: 17.0
Damus: This commit
Setup:
- Make sure you have a debugger connected
- Have a test note that you can quote
Steps:
1. Start Damus and let logs settle
2. Observe where the last log is
3. Quote the test note
4. Copy newly generated logs and paste on a text editor.
5. Analyze those logs. Pay attention to the new note id, as well as the note id of the quoted event (`["q", <QUOTED_NOTE_ID>]`)
Results:
- Logs show the new event being flushed to the relays. PASS
- Logs show the quoted event also being flushed to the relays. PASS
Closes: https://github.com/damus-io/damus/issues/1495
Changelog-Fixed: Broadcast quoted notes when posting a note with quotes
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Reviewed-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Collapse the various standalone functions for parsing block data, and
refactor consumers to initialize a Block with given data and access its
members as needed.
Closes: https://github.com/damus-io/damus/pull/1496
Signed-off-by: William Casarin <jb55@jb55.com>
This is a refactor of the codebase to use a more memory-efficient
representation of notes. It should also be much faster at decoding since
we're using a custom C json parser now.
Changelog-Changed: Improved memory usage and performance when processing events
This is a non-behavioral change in preparation for the actual switchover
from Strings to Ids. The purpose of this kit is to reduce the size of
the switchover commit which is going to be very large.
Since we can't mutate NdbNotes, let's update the existing codebase to
generate and sign ids on NostrEvent constructions. This will allow us to
match NdbNote's constructor
