This commit makes several improvements to video coordination, and implements a new video control view. The video support stack in Damus has been re-architected to achieve this. The new architecture can be summarized as follows: 1. `DamusVideoCoordinator` is a singleton object in `DamusState`, and it is responsible for deciding which video should have the "main stage" focus, based on main stage requests that video player views make when they become visible. Having "main stage" focus means that the coordinator will auto-play that video and pause others, and is used throughout the app to determine which video to talk to or control, in the case of app-wide controls (analogous to how Apple Music needs to know which song is playing for displaying playback controls on the iOS home screen) Having a singleton take care of this establishes clear ownership and prevents conflicts such as double-playing video. This coordinator also holds a pool of video media items (`DamusVideoPlayer`), with exactly ONE `DamusVideoPlayer` per URL, to reduce bandwidth and ensure perfect syncing of the same video in different contexts. 2. `DamusVideoPlayer` objects hold the actual media item (video data, playback state), much like `AVPlayer`. In fact, `DamusVideoPlayer` can be described as a wrapper for `AVPlayer`, except it has an interface that is much more SwiftUI friendly, enabling playback state syncing with minimal effort. `DamusVideoPlayer` is NOT a view. And there is only ONE `DamusVideoPlayer` per URL — held by the coordinator. However, when the app needs to display that same video in multiple places, the app can instantiate multiple video player VIEWS of the same `DamusVideoPlayer` 3. `DamusVideoPlayer.BaseView` is the most basic video player view for a `DamusVideoPlayer` item. It has basically no features other than showing the video itself. 4. `DamusVideoPlayerView` is the standard, batteries-included, video player view for `DamusVideoPlayer` items, that is used throughout the app. It also tries to detect its own visibility, and makes requests to `DamusVideoCoordinator` to take over the main stage when it becomes visible. 5. `DamusVideoControlsView` is a view that presents video playback controls (play/pause, mute, scrubbing) for a `DamusVideoPlayer` object. How a `DamusVideoPlayerView` gains and loses main stage focus: 1. `DamusVideoPlayerView` uses `VisibilityTracker` to find out when it becomes visible or not 2. When it becomes visible, it makes a request to the video coordinator to take main stage focus. The request also specifies which layer the video view is in (Full screen layer? Normal app layer?), which the video player view gets from the `\.view_layer_context` environment variable set by `damus_full_screen_cover` 3. The coordinator (`DamusVideoCoordinator`) keeps all of these requests, and uses its own internal logic and info to determine which video should get the main stage. The logic also depends on whether or not the app finds itself in full screen mode. Once the main stage is given to a different video, the previous video is paused, the main-staged-video is played, and the requestor receives a callback. 4. Once the video disappears from view, it tells the coordinator that it is giving up the main stage, and the coordinator then picks another main stage request again. On top of this, several of other small changes and improvements were made, such as video gesture improvements Note: This commit causes some breakage over the image carousel sizing logic, which will be addressed separately in the next commit. Changelog-Fixed: Fixed iOS 18 gesture issues that would take user to the thread view when clicking on a video or unmuting it Changelog-Fixed: Fixed several issues that would cause video to automatically play or pause incorrectly Changelog-Fixed: Fixed issue where full screen video would disappear when going to landscape mode Changelog-Added: Added new easy to use video controls for full screen video Changelog-Changed: Improved video syncing and bandwidth usage when switching between timeline video and full screen mode Signed-off-by: Daniel D’Aquino <daniel@daquino.me>
348 lines
12 KiB
Swift
348 lines
12 KiB
Swift
//
|
|
// ImageCarousel.swift
|
|
// damus
|
|
//
|
|
// Created by William Casarin on 2022-10-16.
|
|
//
|
|
|
|
import SwiftUI
|
|
import Kingfisher
|
|
|
|
// TODO: all this ShareSheet complexity can be replaced with ShareLink once we update to iOS 16
|
|
struct ShareSheet: UIViewControllerRepresentable {
|
|
typealias Callback = (_ activityType: UIActivity.ActivityType?, _ completed: Bool, _ returnedItems: [Any]?, _ error: Error?) -> Void
|
|
|
|
let activityItems: [URL?]
|
|
let callback: Callback? = nil
|
|
let applicationActivities: [UIActivity]? = nil
|
|
let excludedActivityTypes: [UIActivity.ActivityType]? = nil
|
|
|
|
func makeUIViewController(context: Context) -> UIActivityViewController {
|
|
let controller = UIActivityViewController(
|
|
activityItems: activityItems as [Any],
|
|
applicationActivities: applicationActivities)
|
|
controller.excludedActivityTypes = excludedActivityTypes
|
|
controller.completionWithItemsHandler = callback
|
|
return controller
|
|
}
|
|
|
|
func updateUIViewController(_ uiViewController: UIActivityViewController, context: Context) {
|
|
// nothing to do here
|
|
}
|
|
}
|
|
|
|
// Custom UIPageControl
|
|
struct PageControlView: UIViewRepresentable {
|
|
@Binding var currentPage: Int
|
|
var numberOfPages: Int
|
|
|
|
func makeCoordinator() -> Coordinator {
|
|
Coordinator(self)
|
|
}
|
|
|
|
func makeUIView(context: Context) -> UIPageControl {
|
|
let uiView = UIPageControl()
|
|
uiView.backgroundStyle = .minimal
|
|
uiView.currentPageIndicatorTintColor = UIColor(Color("DamusPurple"))
|
|
uiView.pageIndicatorTintColor = UIColor(Color("DamusLightGrey"))
|
|
uiView.currentPage = currentPage
|
|
uiView.numberOfPages = numberOfPages
|
|
uiView.addTarget(context.coordinator, action: #selector(Coordinator.valueChanged), for: .valueChanged)
|
|
return uiView
|
|
}
|
|
|
|
func updateUIView(_ uiView: UIPageControl, context: Context) {
|
|
uiView.currentPage = currentPage
|
|
uiView.numberOfPages = numberOfPages
|
|
}
|
|
}
|
|
|
|
extension PageControlView {
|
|
final class Coordinator: NSObject {
|
|
var parent: PageControlView
|
|
|
|
init(_ parent: PageControlView) {
|
|
self.parent = parent
|
|
}
|
|
|
|
@objc func valueChanged(sender: UIPageControl) {
|
|
let currentPage = sender.currentPage
|
|
withAnimation {
|
|
parent.currentPage = currentPage
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
|
|
enum ImageShape {
|
|
case square
|
|
case landscape
|
|
case portrait
|
|
case unknown
|
|
|
|
static func determine_image_shape(_ size: CGSize) -> ImageShape {
|
|
guard size.height > 0 else {
|
|
return .unknown
|
|
}
|
|
let imageRatio = size.width / size.height
|
|
switch imageRatio {
|
|
case 1.0: return .square
|
|
case ..<1.0: return .portrait
|
|
case 1.0...: return .landscape
|
|
default: return .unknown
|
|
}
|
|
}
|
|
}
|
|
|
|
class CarouselModel: ObservableObject {
|
|
var current_url: URL?
|
|
var fillHeight: CGFloat
|
|
var maxHeight: CGFloat
|
|
var firstImageHeight: CGFloat?
|
|
|
|
@Published var open_sheet: Bool
|
|
@Published var selectedIndex: Int
|
|
@Published var video_size: CGSize?
|
|
@Published var image_fill: ImageFill?
|
|
|
|
init(image_fill: ImageFill?) {
|
|
self.current_url = nil
|
|
self.fillHeight = 350
|
|
self.maxHeight = UIScreen.main.bounds.height * 1.2 // 1.2
|
|
self.firstImageHeight = nil
|
|
self.open_sheet = false
|
|
self.selectedIndex = 0
|
|
self.video_size = nil
|
|
self.image_fill = image_fill
|
|
}
|
|
}
|
|
|
|
// MARK: - Image Carousel
|
|
@MainActor
|
|
struct ImageCarousel<Content: View>: View {
|
|
var urls: [MediaUrl]
|
|
|
|
let evid: NoteId
|
|
|
|
let state: DamusState
|
|
@ObservedObject var model: CarouselModel
|
|
let content: ((_ dismiss: @escaping (() -> Void)) -> Content)?
|
|
|
|
init(state: DamusState, evid: NoteId, urls: [MediaUrl]) {
|
|
self.urls = urls
|
|
self.evid = evid
|
|
self.state = state
|
|
let media_model = state.events.get_cache_data(evid).media_metadata_model
|
|
self._model = ObservedObject(initialValue: CarouselModel(image_fill: media_model.fill))
|
|
self.content = nil
|
|
}
|
|
|
|
init(state: DamusState, evid: NoteId, urls: [MediaUrl], @ViewBuilder content: @escaping (_ dismiss: @escaping (() -> Void)) -> Content) {
|
|
self.urls = urls
|
|
self.evid = evid
|
|
self.state = state
|
|
let media_model = state.events.get_cache_data(evid).media_metadata_model
|
|
self._model = ObservedObject(initialValue: CarouselModel(image_fill: media_model.fill))
|
|
self.content = content
|
|
}
|
|
|
|
var filling: Bool {
|
|
model.image_fill?.filling == true
|
|
}
|
|
|
|
var height: CGFloat {
|
|
model.firstImageHeight ?? model.image_fill?.height ?? model.fillHeight
|
|
}
|
|
|
|
func Placeholder(url: URL, geo_size: CGSize, num_urls: Int) -> some View {
|
|
Group {
|
|
if num_urls > 1 {
|
|
// jb55: quick hack since carousel with multiple images looks horrible with blurhash background
|
|
Color.clear
|
|
} else if let meta = state.events.lookup_img_metadata(url: url),
|
|
case .processed(let blurhash) = meta.state {
|
|
Image(uiImage: blurhash)
|
|
.resizable()
|
|
.frame(width: geo_size.width * UIScreen.main.scale, height: self.height * UIScreen.main.scale)
|
|
} else {
|
|
Color.clear
|
|
}
|
|
}
|
|
}
|
|
|
|
func Media(geo: GeometryProxy, url: MediaUrl, index: Int) -> some View {
|
|
Group {
|
|
switch url {
|
|
case .image(let url):
|
|
Img(geo: geo, url: url, index: index)
|
|
.onTapGesture {
|
|
model.open_sheet = true
|
|
}
|
|
case .video(let url):
|
|
DamusVideoPlayerView(url: url, coordinator: state.video, style: .preview(on_tap: { model.open_sheet = true }))
|
|
.onChange(of: model.video_size) { size in
|
|
guard let size else { return }
|
|
|
|
let fill = ImageFill.calculate_image_fill(geo_size: geo.size, img_size: size, maxHeight: model.maxHeight, fillHeight: model.fillHeight)
|
|
|
|
print("video_size changed \(size)")
|
|
if self.model.image_fill == nil {
|
|
print("video_size firstImageHeight \(fill.height)")
|
|
self.model.firstImageHeight = fill.height
|
|
state.events.get_cache_data(evid).media_metadata_model.fill = fill
|
|
}
|
|
|
|
self.model.image_fill = fill
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
func Img(geo: GeometryProxy, url: URL, index: Int) -> some View {
|
|
KFAnimatedImage(url)
|
|
.callbackQueue(.dispatch(.global(qos:.background)))
|
|
.backgroundDecode(true)
|
|
.imageContext(.note, disable_animation: state.settings.disable_animation)
|
|
.image_fade(duration: 0.25)
|
|
.cancelOnDisappear(true)
|
|
.configure { view in
|
|
view.framePreloadCount = 3
|
|
}
|
|
.imageFill(for: geo.size, max: model.maxHeight, fill: model.fillHeight) { fill in
|
|
state.events.get_cache_data(evid).media_metadata_model.fill = fill
|
|
// blur hash can be discarded when we have the url
|
|
// NOTE: this is the wrong place for this... we need to remove
|
|
// it when the image is loaded in memory. This may happen
|
|
// earlier than this (by the preloader, etc)
|
|
DispatchQueue.main.asyncAfter(deadline: .now() + 1.0) {
|
|
state.events.lookup_img_metadata(url: url)?.state = .not_needed
|
|
}
|
|
self.model.image_fill = fill
|
|
if index == 0 {
|
|
self.model.firstImageHeight = fill.height
|
|
//maxHeight = firstImageHeight ?? maxHeight
|
|
} else {
|
|
//maxHeight = firstImageHeight ?? fill.height
|
|
}
|
|
}
|
|
.background {
|
|
Placeholder(url: url, geo_size: geo.size, num_urls: urls.count)
|
|
}
|
|
.aspectRatio(contentMode: filling ? .fill : .fit)
|
|
.kfClickable()
|
|
.position(x: geo.size.width / 2, y: geo.size.height / 2)
|
|
.tabItem {
|
|
Text(url.absoluteString)
|
|
}
|
|
.id(url.absoluteString)
|
|
.padding(0)
|
|
|
|
}
|
|
|
|
var Medias: some View {
|
|
TabView(selection: $model.selectedIndex) {
|
|
ForEach(urls.indices, id: \.self) { index in
|
|
GeometryReader { geo in
|
|
Media(geo: geo, url: urls[index], index: index)
|
|
}
|
|
}
|
|
}
|
|
.tabViewStyle(PageTabViewStyle(indexDisplayMode: .never))
|
|
.fullScreenCover(isPresented: $model.open_sheet) {
|
|
if let content {
|
|
FullScreenCarouselView<Content>(video_coordinator: state.video, urls: urls, settings: state.settings, selectedIndex: $model.selectedIndex) {
|
|
content({ // Dismiss closure
|
|
model.open_sheet = false
|
|
})
|
|
}
|
|
}
|
|
else {
|
|
FullScreenCarouselView<AnyView>(video_coordinator: state.video, urls: urls, settings: state.settings, selectedIndex: $model.selectedIndex)
|
|
}
|
|
}
|
|
.frame(height: height)
|
|
.onChange(of: model.selectedIndex) { value in
|
|
model.selectedIndex = value
|
|
}
|
|
}
|
|
|
|
var body: some View {
|
|
VStack {
|
|
if #available(iOS 18.0, *) {
|
|
Medias
|
|
} else {
|
|
// An empty tap gesture recognizer is needed on iOS 17 and below to suppress other overlapping tap recognizers
|
|
// Otherwise it will both open the carousel and go to a note at the same time
|
|
Medias.onTapGesture { }
|
|
}
|
|
|
|
|
|
if urls.count > 1 {
|
|
PageControlView(currentPage: $model.selectedIndex, numberOfPages: urls.count)
|
|
.frame(maxWidth: 0, maxHeight: 0)
|
|
.padding(.top, 5)
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
// MARK: - Image Modifier
|
|
extension KFOptionSetter {
|
|
/// Sets a block to get image size
|
|
///
|
|
/// - Parameter block: The block which is used to read the image object.
|
|
/// - Returns: `Self` value after read size
|
|
public func imageFill(for size: CGSize, max: CGFloat, fill: CGFloat, block: @escaping (ImageFill) throws -> Void) -> Self {
|
|
let modifier = AnyImageModifier { image -> KFCrossPlatformImage in
|
|
let img_size = image.size
|
|
let geo_size = size
|
|
let fill = ImageFill.calculate_image_fill(geo_size: geo_size, img_size: img_size, maxHeight: max, fillHeight: fill)
|
|
DispatchQueue.main.async { [block, fill] in
|
|
try? block(fill)
|
|
}
|
|
return image
|
|
}
|
|
options.imageModifier = modifier
|
|
return self
|
|
}
|
|
}
|
|
|
|
|
|
public struct ImageFill {
|
|
let filling: Bool?
|
|
let height: CGFloat
|
|
|
|
static func calculate_image_fill(geo_size: CGSize, img_size: CGSize, maxHeight: CGFloat, fillHeight: CGFloat) -> ImageFill {
|
|
let shape = ImageShape.determine_image_shape(img_size)
|
|
|
|
let xfactor = geo_size.width / img_size.width
|
|
let scaled = img_size.height * xfactor
|
|
|
|
//print("calc_img_fill \(img_size.width)x\(img_size.height) xfactor:\(xfactor) scaled:\(scaled)")
|
|
|
|
// calculate scaled image height
|
|
// set scale factor and constrain images to minimum 150
|
|
// and animations to scaled factor for dynamic size adjustment
|
|
switch shape {
|
|
case .portrait, .landscape:
|
|
let filling = scaled > maxHeight
|
|
let height = filling ? fillHeight : scaled
|
|
return ImageFill(filling: filling, height: height)
|
|
case .square, .unknown:
|
|
return ImageFill(filling: nil, height: scaled)
|
|
}
|
|
}
|
|
}
|
|
|
|
// MARK: - Preview Provider
|
|
struct ImageCarousel_Previews: PreviewProvider {
|
|
static var previews: some View {
|
|
let url: MediaUrl = .image(URL(string: "https://jb55.com/red-me.jpg")!)
|
|
let test_video_url: MediaUrl = .video(URL(string: "http://cdn.jb55.com/s/zaps-build.mp4")!)
|
|
ImageCarousel<AnyView>(state: test_damus_state, evid: test_note.id, urls: [test_video_url, url])
|
|
.environmentObject(OrientationTracker())
|
|
}
|
|
}
|
|
|