This commit introduces a verification step at the relay connection
level, to help ensure notes get validated at the source and prevent
security issues associated with untrusted relays.
`RelayConnection.swift` — the source that initially handles WebSocket
messages — was analyzed, and measures were put in place to prevent
(or at least minimize) unverified nostr event data being spread
throughout the app.
The following measures were taken:
1. A note verification step was added prior to the `self.handleEvent(.nostr_event(ev))` call (which sends a Nostr response to the rest of the app for logical handling).
a. From code analysis, there is only one such call in `RelayConnection.swift`.
2. `NostrConnectionEvent`, the object that gets passed to event handlers, had its interface modified to remove the "message" case, since:
a. that could be a source of unverified nostr events.
b. it is redundant an unneeded due to the `.nostr_event` case.
c. there were no usages of it around the codebase
3. The raw websocket event handler had its label renamed to "handleUnverifiedWSEvent", to make it clear to the caller about the verification status of the data.
a. Usages of this were inspected and no significant risk was detected.
4. A new `verify` method in NdbNote was created to verify Nostr notes, and unit tests were added to confirm tampering detections around all the major fields in a Nostr note.
5. Care was taken to ensure the performance regression is as little as
possible.
It is worth noting that we will not need this once the local relay model
architecture is introduced, since that architecture ensures note
validation before it reaches the rest of the application and the user.
In other words, this is a temporary fix.
However, since the migration to that new architecture is a major
undertaking that will take some time to be completed, this fix was written
in order to address security concerns while the migration is unfinished.
This fix was written in a way that attempts to be as effective as
possible in reducing security risks without a risky and lenghty
refactor of the code that would delay the fix from being published.
Changelog-Fixed: Improved security around note validation
Closes: https://github.com/damus-io/damus/issues/1341
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit fixes a crash that occurred when swapping between Damus and
other apps.
When Damus enters background mode, NostrDB is closed and its resources
released. When Damus re-enters foreground mode, NostrDB is reopened.
However, an issue with the transaction inheritance logic
caused a race condition where a side menu profile lookup would get an
obsolete transaction containing pointers that have been freedwhen
NostrDB was closed, causing a "use-after-free" memory error.
The issue was fixed by improving the transaction inheritance logic to
double-check if the "generation" counter (which auto increments when
Damus closes and re-opens) matches the generation marked on the
thread-specific transaction. This effectively prevents lookups from
inheriting an obsolete transaction from a previous NostrDB generation.
Closes: https://github.com/damus-io/damus/issues/3167
Changelog-Fixed: Fixed an issue where the app would crash when swapping between apps
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
It was decided on a standup meeting that this feature is not important
and failing tests can be disabled.
Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit fixes a stack corruption issue caused by
an off-by-one error in one of the functions responsible
for parsing bech32 entities.
Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Previously two addresses from different memory regions were being
subtracted, which will lead to the incorrect number. This commit
improves the calculation.
Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Currently NostrDB does not seem to handle encryption/decryption of DMs.
Since NostrDB now controls the block parsing process and fetches note
contents directly from the database, we have to add a specific condition
that injects decrypted content directly to the ndb content parser.
This is done in conjunction with some minor refactoring to `NdbBlocks`
and associated structs, as in C those are separated between the content
string and the offsets for each block, but in Swift this is more
ergonomically represented as a standalone/self-containing object.
No changelog entry is added because the previously broken version was
never released to the public, and therefore this fix produces no
user-facing changes compared to the last released version.
Changelog-None
Closes: https://github.com/damus-io/damus/issues/3106
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
NostrDB relies on manual memory management, so it is a good idea to
enable the address sanitizer on debug configurations, as it helps find
memory-related issues on the app, which will allow us to identify memory
issues and potential crashes earlier in the development process.
Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
This commit introduces new interfaces for working with NostrDB from
Swift, including `NostrFilter` conversion, subscription streaming via
AsyncStreams and lookup/wait functions.
No user-facing changes.
Changelog-None
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
I wanted to not amend this since we've already applied
it in the nostrdb-update branch on damus and I don't want
to conflict
Signed-off-by: William Casarin <jb55@jb55.com>
The Address Sanitizer detected a heap buffer overflow during a memcpy operation
in nostrdb.c associated with note parsing.
It was found that not enough memory was being allocated to the buffer to
support all the content parsing.
Allocation size was increased to support the memory needed for the
parsing operations. However, the new number was not carefully calculated
as we will not run into this code path once we switch to the local relay
model.
Changelog-Fixed: Fixed memory error in nostrdb
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Add support for type KIND for bech32-encoded entities naddr and nevent
as specified in NIP-19.
Co-authored-by: kernelkind <kernelkind@gmail.com>
Signed-off-by: William Casarin <jb55@jb55.com>
This adds support for nip50 fulltext searches. This allows you to use
the nostrdb query interface for executing fulltext searches instead of
the typical `ndb_text_search` api. The benefits of this include a
standardized query interface that also further filters on other fields
in the filter.
Changelog-Added: Add nip50 search filters and queries
Signed-off-by: William Casarin <jb55@jb55.com>
Update fulltext search queries to include an optional filter. This can
be used to narrow down the fulltext search. This is another step towards
nip50 support in nostrdb.
I noticed the code was exiting dubiously in certain situations... so we
fix that as well. It's possible we were missing search results because
of this.
Signed-off-by: William Casarin <jb55@jb55.com>
Add a helper for sorting search words from largest to smallest. This
should help search performance. For example, let's say our search index
is like so:
"the pokemon is cool"
the
the
the
...
* 1000
Our root word search would have to start 1000 new recursive queries. By
sorting by the largest word:
pokemon
pokemon
pokemon
...
* 10
We only have to do 10 recursive searches, assuming larger words are less
common, which will likely be the case most of the time
Signed-off-by: William Casarin <jb55@jb55.com>
This fixes an allocation issue with ndb_filter_init_with for small
page sizes. instead of allocating the buffer around pages, we allocate
based on total buffer size.
Fixes: f7aac3215575 ("filter: introduce ndb_filter_init_with")
Signed-off-by: William Casarin <jb55@jb55.com>
We forgot to move one DEBUG instance to NDB_LOG
Fixes: b4c2ff3d270a ("Only log to stdout if NDB_LOG is defined")
Signed-off-by: William Casarin <jb55@jb55.com>
Just a static function for now for creating smaller filter sizes. We
will use this for filters that we know are small so that we don't have
to allocate so many pages at once. It's likely the OS will only allocate
a single page anyways, but its nice to be explicit.
Changelog-Added: Add ndb_filter_init_with
Signed-off-by: William Casarin <jb55@jb55.com>
Make fulltext indices and note blocks optional. This will be useful for
quickly building databases when testing, since more stuff in the write
queue when writing can slow things down.
Signed-off-by: William Casarin <jb55@jb55.com>
We should be specifying that we've matched the id here, not authors. Not
that this would have any effect.. but still.
Signed-off-by: William Casarin <jb55@jb55.com>
This function can be used to check if a db is an index or not. We
will use this in future functions that rebuild indices.
Signed-off-by: William Casarin <jb55@jb55.com>
technically more accurate. we are about to introduce a new type called:
ndb_ts_u64_id
which would be confusing if we didn't do this
Signed-off-by: William Casarin <jb55@jb55.com>
This was the only thing that wasn't threadsafe. Add a simple mutex
instead of a queue so that polling is quick.
This also means we can't really return the internal subscriptions
anymore, so we remove that for now until we have a safer
interface.
Fixes: https://github.com/damus-io/nostrdb/issues/55
Signed-off-by: William Casarin <jb55@jb55.com>
subset testing for filters. Can be used to see if one subset is
redundant in the presence of a another in the local relay model
Changelog-Added: Add ndb_filter_is_subset_of
Signed-off-by: William Casarin <jb55@jb55.com>
filter equality testing. this works because field elements are sorted
Changelog-Added: Add ndb_filter_eq for filter equality testing
Signed-off-by: William Casarin <jb55@jb55.com>
Expose a way to get the set of filters for a subscription. On the rust
side, we should likely ndb_filter_clone each filter asap, because the
result of this function will only be valid up until the subscription
ends.
Signed-off-by: William Casarin <jb55@jb55.com>
This will find strings which match the beginning of other strings,
which seems wrong.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
Sure, this format would be nice, but it's not what the code does.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
If we make unknown_field simply discard, we can remove decoders and
have them discard those fields.
Now we can cut down struct bolt11 to only the fields needed by
invoice.c, and also speed up parsing a little.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
Copy the latest, which has parsing fixes. We make a new explicit
"bolt11_decode_minimal" which doesn't check sigs, rather than neutering
the bolt11_decode logic.
As a bonus, this now correctly parses "LIGHTNING:BECH32..." format
(upper case, with prefix).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
I wondered by `make check` was giving strange errors, until I realized it wasn't fully rebuilding.
Also, remove leftover CCAN files I missed previously.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
Only change for us: CCAN_TAL_NEVER_RETURN_NULL can be defined if
we don't override tal error handling.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
This is the version of CCAN which CLN was using at the time these
were taken. Unfortunately lots of whitespace has been changed,
but AFAICT no source changes.
Here's the command I ran (with ../ccan checked out to 1ae4c432):
```
make update-ccan CCAN_NEW="alignof array_size build_assert check_type container_of cppmagic likely list mem short_types str structeq take tal tal/str typesafe_cb utf8 endian crypto/sha256"
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
It isn't actually in the CCAN module (though it probably should be!).
So it breaks when we update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
This lets them be updated/bugfixed together. I just copied them for now,
didn't change anything else.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
This will find strings which match the beginning of other strings,
which seems wrong.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
Sure, this format would be nice, but it's not what the code does.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
If we make unknown_field simply discard, we can remove decoders and
have them discard those fields.
Now we can cut down struct bolt11 to only the fields needed by
invoice.c, and also speed up parsing a little.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
Copy the latest, which has parsing fixes. We make a new explicit
"bolt11_decode_minimal" which doesn't check sigs, rather than neutering
the bolt11_decode logic.
As a bonus, this now correctly parses "LIGHTNING:BECH32..." format
(upper case, with prefix).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
I wondered by `make check` was giving strange errors, until I realized it wasn't fully rebuilding.
Also, remove leftover CCAN files I missed previously.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
It isn't actually in the CCAN module (though it probably should be!).
So it breaks when we update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: William Casarin <jb55@jb55.com>
When creating filters, sometimes IDs are pushed as strings, so if there
is ever a 0 byte, the id prematurely ends, causing the filter to not
match
Fixes: https://github.com/rust-nostr/nostr/issues/454
Signed-off-by: William Casarin <jb55@jb55.com>
can't figure out why this is happening, but let's disable it for now
while we test. we shouldn't hit this code path anyways once we switch
over to local notes in damus ios
Signed-off-by: William Casarin <jb55@jb55.com>
Since Damus iOS is not an immediate-mode UI like android, we would
rather not poll for results. Instead we need a way to register a
callback function that is called when we get new subscription results.
This is also useful on the android side, allowing us to request a new
frame to draw when we have new results, instead of drawing every second.
Signed-off-by: William Casarin <jb55@jb55.com>
We didn't have a way to unsubscribe from subscriptions. Now we do!
Apps like notecrumbs may open up many local subscriptions based on
incoming requests. We may need to make the MAX_SUBSCRIPTIONS size much
larger, but this should be okish for now.
Changelog-Added: Add ndb_unsubscribe to unsubscribe from subscriptions
Signed-off-by: William Casarin <jb55@jb55.com>
This introduces the basic created_at query plan. We scan the created_at
+ id index in descending order looking for items that match a filter.
This is a very general query plan, but might not be very efficient for
anything other than local timelines.
Changelog-Added: Add general created_at query plan for timelines
Closes: https://github.com/damus-io/nostrdb/issues/26
Signed-off-by: William Casarin <jb55@jb55.com>
Clone filters when moving them into subscriptions. This will allow us to
fix the double free issue on the rust side.
Signed-off-by: William Casarin <jb55@jb55.com>
Instead of storing exact pointers inside of our filter elements, just
store offsets. This will allow us to clone filters very easily without
having to mess around with fixing up the pointers afterwards.
Signed-off-by: William Casarin <jb55@jb55.com>
This is a pretty scary looking function that realloc our large variable
filter buffer into a compact one. This saves up a bunch of memory when
we are done building the filter.
Signed-off-by: William Casarin <jb55@jb55.com>
The polling variant of ndb_wait_for_notes. This makes more sense for
realtime apps like notedeck
Changelog-Added: Add ndb_poll_for_notes
Signed-off-by: William Casarin <jb55@jb55.com>
I don't technically need this but it helps a lot on the swift side
of things since I already have code that uses this identifier of a
similar structure
Signed-off-by: William Casarin <jb55@jb55.com>
Instead of running queries off filters directly, we do some simple
heuristics and determine a reasonable query plan for the given filter.
To test this, also add a kind index query plan and add a test for it.
We still need tag, author, and created_at index scans. This is up next!
Signed-off-by: William Casarin <jb55@jb55.com>
Still a lot more work to do, but this is at least a proof of concept for
querying nostrdb using filters.
Signed-off-by: William Casarin <jb55@jb55.com>
This is useful for positioning LMDB cursors at the start of a query. We
will be re-using this in the upcoming query code
Signed-off-by: William Casarin <jb55@jb55.com>
This makes it a bit more flexible, but maybe we can add quoting in the
future that re-enables this. Or maybe a search option
Signed-off-by: William Casarin <jb55@jb55.com>
This adds some initial code for the nostrdb relay subscription monitor.
When new notes are written to the database, they are checked against
active subscriptions. If any of the subscriptions are matched, the note
primary key is written to the inbox queue for that subscription.
We also add an ndb_wait_for_notes() method that simply waits for notes
to be written by the subscription monitor.
Changelog-Added: Added filter subscriptions
Signed-off-by: William Casarin <jb55@jb55.com>
so we don't need heap allocation. we will be calling this a lot in tight
render loops, we don't want to be allocating on each frame.
Signed-off-by: William Casarin <jb55@jb55.com>
In some situations we will need to have owned note blocks. For
example, when we try to fetch note blocks from the database and it's not
there yet. We will need to parse the content on the spot and return an
owned copy, since it will not be immediately available in the database.
Add a new flag field to note blocks that lets us know if it's owned by
malloc or nostrdb.
We the add a free function that checks this flag and frees the object if
its set. If it is not set then it doesn nothing because it likely came
from the database.
Signed-off-by: William Casarin <jb55@jb55.com>
Fix parsing URL when encountering a period at the end of the url by
setting it as disallowed from being present at the end of a
URL.
Some characters are disallowed to be present at the end of URLs.
Presently, the period character is the only disallowed character.
A character is the last character in the URL if it is followed by
is_whitespace() or if it's the last character in the string.
Signed-off-by: kernelkind <kernelkind@gmail.com>
Tested-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb5.com>
Signed-off-by: William Casarin <jb55@jb55.com>
We need to pull the data out as well! Let's add some initial decoders.
We still need tests to make sure it's working.
Signed-off-by: William Casarin <jb55@jb55.com>
This adds some initial code for nostrdb content parsing.
We still need to write tests for encoding and decoding, so this is
likely not working yet.
Signed-off-by: William Casarin <jb55@jb55.com>
We will be storing raw nostr bech32 buffers directly into nostrdb, so
adapt our bech32 code to reflect this.
When doing our content parsing pass, we will only look for strings and we
won't allocate any intermediate buffers. Only when we write this string
block to nostrdb will we actually allocate in our nostrdb output buffer
(no mallocs!)
Signed-off-by: William Casarin <jb55@jb55.com>
This is the start of our rust library for nostrdb. Implement idiomatic
interfaces for Ndb and NdbConfig.
Changelog-Added: Add initial rust library
Signed-off-by: William Casarin <jb55@jb55.com>
rust doesn't like packed structures, so hide this from bindgen
This also buttons up the API so less things are exposed which is good.
Signed-off-by: William Casarin <jb55@jb55.com>
- Fix aspect ratio, use fit
- Remove fixed height on image frame to align close button on image
- Use overlay instead of ZStack to reduce complexity
- Add background to close button to get better contrast in light mode
- Change close-image to be a button for better accessibility
Changelog-Fixed: Fix aspect ratio on pasted or uploaded images
Signed-off-by: Askeew <askeew@hotmail.com>
Closes: https://github.com/damus-io/damus/issues/2913
Huge refactor to add better structure to the project.
Separating features with their associated view and model structure.
This should be better organization and will allow us to improve the
overall architecture in the future.
I forsee many more improvements that can follow this change. e.g. MVVM Arch
As well as cleaning up duplicate, unused, functionality.
Many files have global functions that can also be moved or be renamed.
damus/
├── Features/
│ ├── <Feature>/
│ │ ├── Views/
│ │ └── Models/
├── Shared/
│ ├── Components/
│ ├── Media/
│ ├── Buttons/
│ ├── Extensions/
│ ├── Empty Views/
│ ├── ErrorHandling/
│ ├── Modifiers/
│ └── Utilities/
├── Core/
│ ├── Nostr/
│ ├── NIPs/
│ ├── DIPs/
│ ├── Types/
│ ├── Networking/
│ └── Storage/
Signed-off-by: ericholguin <ericholguin@apache.org>
Changelog-Fixed: Fixed note content rendering to not remove whitespace before hashtag
Closes: https://github.com/damus-io/damus/issues/3122
Fixes: f436291209 ("Fix note content rendering to not remove whitespace before hashtag")
Signed-off-by: Terry Yiu <git@tyiu.xyz>
Put the views into ScrollView
Fixed banner offset in Geometry reader
Changelog-Fixed: Fixed stretchy banner header in Edit profile
Signed-off-by: Swift Coder <scoder1747@gmail.com>
This commit enhances the ImageCarousel component to maintain a consistent
height when navigating between images of different aspect ratios. The
changes prevent the UI from "jumping" during carousel navigation, which
improves the overall user experience.
Key improvements:
- Added `first_image_fill` property to store dimensions of the first image
- Modified height calculation to prioritize the first image's dimensions
- Refactored image fill calculation into a reusable `compute_item_fill` method
- Added proper view clipping to prevent content overflow
- Simplified filling behavior for more predictable layout
These changes provide a smoother, more stable carousel experience by
maintaining consistent dimensions throughout image navigation.
Changelog-Changed: Improved the image sizing behavior on the image carousel for a smoother experience
Closes: https://github.com/damus-io/damus/issues/2724
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Damus stores npub as both Strings and URLs in NSAttributedString.Key.link when a note is saved as a draft. Make Damus correctly handle both when we retrieve and store drafts.
Changelog-Changed: Handle npub correctly in draft notes
Signed-off-by: Askeew <askeew@hotmail.com>
Closes: https://github.com/damus-io/damus/issues/2923
[](https://github.com/damus-io/damus/actions/workflows/run-tests.yaml)
user_visible_description:NSLocalizedString("You opened an invalid link. The link you tried to open refers to \"nsec\", which is not supported.",comment:"User-visible error description for a user who tries to open an unsupported \"nsec\" link."),
tip:NSLocalizedString("Please contact the person who provided the link, and ask for another link. Also, this link may have sensitive information, please use caution before sharing it.",comment:"User-visible tip on what to do if a link contains an unsupported \"nsec\" reference."),
Text("\(Text(verbatim:bar.relays.formatted()).font(.body.bold()))\(noun)",comment:"Sentence composed of 2 variables to describe how many relays a note was found on. In source English, the first variable is the number of relays, and the second variable is 'Relay' or 'Relays'.")
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.