Chunk home filters to avoid hitting max filter item limits

When a user is following several accounts, they may get a stale feed
caused by the subscription request being rejected by relays (due to max filter item limits).

This commit implements a fix that gets around the issue by
creating several chunked filters for the home feed event and contact
metadata subscriptions.

This is a short to medium-term practical fix, where we get around the
practical limitations imposed by most relays. In the future we should
work on longer-term solutions, which will likely require protocol improvements

Main Test
---------

Procedure:
1. Login with Elsat's npub (Or some account that follows about 2K people)
2. Check the home feed. There should be fresh notes.

REPRO:
Device: iPhone 15 simulator
iOS: 17.4
Damus: 1.9 (3) (0d9954290a)
Results:
- No fresh notes, most recent post is from several hours ago (Feed is stale)

FIX TEST:
Device: iPhone 15 simulator
iOS: 17.4
Damus: This commit
Results:
- Fresh notes appear, most recent post is from a few seconds ago.

Other testing:
--------------

- New automated test passing
- All other automated tests passing
- Tested scrolling down the feed on these conditions:
  - Device: iPhone 13 Mini
  - iOS: 17.4.1
  - Accounts:
    - One with about 160 contacts and 10 relays (Daniel D’Aquino)
    - One with about 1K+ contacts and 9 relays (Freedom Smuggler)
    - One with about 981 contacts and 6 relays (jb55)
    - Elsat's account (2K+ accounts and 8 relays)
  - Result: None of those were stale

Changelog-Fixed: Fix stale feed issue when follow list is too big
Closes: https://github.com/damus-io/damus/issues/2194
Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
Reviewed-by: William Casarin <jb55@jb55.com>
This commit is contained in:
Daniel D’Aquino
2024-05-10 17:03:01 -07:00
parent 52aefc8d64
commit 46185c55d1
5 changed files with 182 additions and 2 deletions

View File

@@ -42,6 +42,10 @@ enum HomeResubFilter {
}
class HomeModel: ContactsDelegate {
// The maximum amount of contacts placed on a home feed subscription filter.
// If the user has more contacts, chunking or other techniques will be used to avoid sending huge filters
let MAX_CONTACTS_ON_FILTER = 500
// Don't trigger a user notification for events older than a certain age
static let event_max_age_for_notification: TimeInterval = EVENT_MAX_AGE_FOR_NOTIFICATION
@@ -545,7 +549,8 @@ class HomeModel: ContactsDelegate {
notifications_filter.limit = 500
var notifications_filters = [notifications_filter]
var contacts_filters = [contacts_filter, our_contacts_filter, our_blocklist_filter, our_old_blocklist_filter]
let contacts_filter_chunks = contacts_filter.chunked(on: .authors, into: MAX_CONTACTS_ON_FILTER)
var contacts_filters = contacts_filter_chunks + [our_contacts_filter, our_blocklist_filter, our_old_blocklist_filter]
var dms_filters = [dms_filter, our_dms_filter]
let last_of_kind = get_last_of_kind(relay_id: relay_id)
@@ -598,7 +603,7 @@ class HomeModel: ContactsDelegate {
home_filter.authors = friends
home_filter.limit = 500
var home_filters = [home_filter]
var home_filters = home_filter.chunked(on: .authors, into: MAX_CONTACTS_ON_FILTER)
let followed_hashtags = Array(damus_state.contacts.get_followed_hashtags())
if followed_hashtags.count != 0 {