Chunk home filters to avoid hitting max filter item limits

When a user is following several accounts, they may get a stale feed
caused by the subscription request being rejected by relays (due to max filter item limits).

This commit implements a fix that gets around the issue by
creating several chunked filters for the home feed event and contact
metadata subscriptions.

This is a short to medium-term practical fix, where we get around the
practical limitations imposed by most relays. In the future we should
work on longer-term solutions, which will likely require protocol improvements

Main Test
---------

Procedure:
1. Login with Elsat's npub (Or some account that follows about 2K people)
2. Check the home feed. There should be fresh notes.

REPRO:
Device: iPhone 15 simulator
iOS: 17.4
Damus: 1.9 (3) (0d9954290a)
Results:
- No fresh notes, most recent post is from several hours ago (Feed is stale)

FIX TEST:
Device: iPhone 15 simulator
iOS: 17.4
Damus: This commit
Results:
- Fresh notes appear, most recent post is from a few seconds ago.

Other testing:
--------------

- New automated test passing
- All other automated tests passing
- Tested scrolling down the feed on these conditions:
  - Device: iPhone 13 Mini
  - iOS: 17.4.1
  - Accounts:
    - One with about 160 contacts and 10 relays (Daniel D’Aquino)
    - One with about 1K+ contacts and 9 relays (Freedom Smuggler)
    - One with about 981 contacts and 6 relays (jb55)
    - Elsat's account (2K+ accounts and 8 relays)
  - Result: None of those were stale

Changelog-Fixed: Fix stale feed issue when follow list is too big
Closes: https://github.com/damus-io/damus/issues/2194
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Reviewed-by: William Casarin <jb55@jb55.com>
This commit is contained in:
Daniel D’Aquino
2024-05-10 17:03:01 -07:00
parent 52aefc8d64
commit 46185c55d1
5 changed files with 182 additions and 2 deletions

View File

@@ -54,4 +54,68 @@ struct NostrFilter: Codable, Equatable {
public static func filter_hashtag(_ htags: [String]) -> NostrFilter {
NostrFilter(hashtag: htags.map { $0.lowercased() })
}
/// Splits the filter on a given filter path/axis into chunked filters
///
/// - Parameter path: The path where chunking should be done
/// - Parameter chunk_size: The maximum size of each chunk.
/// - Returns: An array of arrays, where each contained array is a chunk of the original array with up to `size` elements.
func chunked(on path: ChunkPath, into chunk_size: Int) -> [Self] {
let chunked_slices = self.get_slice(from: path).chunked(into: chunk_size)
var chunked_filters: [NostrFilter] = []
for chunked_slice in chunked_slices {
var chunked_filter = self
chunked_filter.apply_slice(chunked_slice)
chunked_filters.append(chunked_filter)
}
return chunked_filters
}
/// Gets a slice from a NostrFilter on a given path/axis
///
/// - Parameter path: The path where chunking should be done
/// - Parameter chunk_size: The maximum size of each chunk.
/// - Returns: An array of arrays, where each contained array is a chunk of the original array with up to `size` elements.
func get_slice(from path: ChunkPath) -> Slice {
switch path {
case .pubkeys:
return .pubkeys(self.pubkeys)
case .authors:
return .authors(self.authors)
}
}
/// Overrides one member/axis of a NostrFilter using a specific slice
/// - Parameter slice: The slice to be applied on this NostrFilter
mutating func apply_slice(_ slice: Slice) {
switch slice {
case .pubkeys(let pubkeys):
self.pubkeys = pubkeys
case .authors(let authors):
self.authors = authors
}
}
/// A path to one of the axes of a NostrFilter.
enum ChunkPath {
case pubkeys
case authors
// Other paths/axes not supported yet
}
/// Represents the value of a single axis of a NostrFilter
enum Slice {
case pubkeys([Pubkey]?)
case authors([Pubkey]?)
func chunked(into chunk_size: Int) -> [Slice] {
switch self {
case .pubkeys(let array):
return (array ?? []).chunked(into: chunk_size).map({ .pubkeys($0) })
case .authors(let array):
return (array ?? []).chunked(into: chunk_size).map({ .authors($0) })
}
}
}
}