libvlc client can set a block that will be called each time picture in picture is
started or closed.
`VLCPictureInPictureWindowControlling` implementation is also moved from the display
module to the picture in picture module
The recent addition of the new display to support natively decoded
CVSampleBufferRef frames provides straightforward Metal rendering
but may still suffer from unsupported features or unstable code.
Being able to fallback to any legacy OpenGL based display might be
helpful to get around unexpected issues with the new display.
In libvlc, vout displays are opened by order of priority and can't
be set using the `--vout` option.
Libvlc users had no way to use an alternate vout display if the
samplebufferdisplay doesn't work as expected.
This change is introducing a new option for libvlc users to force
a fallback to legacy OpenGL based displays.
This adds a new module to handle picture in picture on iOS and tvOS.
Picture in picture is only enabled if the drawable ns-object conforms to the
new `VLCPictureInPictureDrawable` protocol.
`VLCPictureInPictureDrawable` protocol provide new apis to start/stop picture
in picture and various callbacks to handle user interactions from the picture
in picture window's overlay.
Implementing `VLCPictureInPictureDrawable.pictureInPictureReady` block is
mandatory to be notified once picture in picture initialization is done.
This block will pass a `id<VLCPictureInPictureWindowControlling>` object
allowing to start and stop picture in picture.
It will also provide a way to invalidate the state of the playback by calling
`invalidatePlaybackState`, forcing the picture in picture to fetch new media
infos like current time or media length from the
`VLCPictureInPictureDrawable.mediaController`.
This module uses Apple public APIs hence it won't work for macOS at this time.
Separate work will have to be done in order to provide support for picture in
picture private APIs and allow the use of picture in picture on macOS with
libvlc.
Transform image buffer's pixels to display frames with proper video orientation.
API used to transform the picture depends on the host operating system version.
Given OS is at least iOS/iPadOS/tvOS 16.0 or macOS 13.0, an implementation backed
by VideoToolbox APIs will be used.
If not available, it will fallback to a CoreImage backed implementation.
So far the CoreImage implementation doesn't seems to perform as fast as the one
backed by the VideoToolbox API.
Fixes a warning about the implicit conversion of enum type:
implicit conversion from enumeration type 'enum CGImageAlphaInfo'
to different enumeration type 'CGBitmapInfo'
If the display filling mode of the video changed, the video placement has changed as well.
Display modules don't actually care about the aspect ratio, just where they're supposed to stretch the video.
If the zoom of the video changed, the video placement has changed as well.
Display modules don't actually care about the aspect ratio, just where they're supposed to stretch the video.
We already use the picture pitch.
We map the picture to a texture that is rendered using the buffer dimensions.
The other format is mostly used for positioning.
This separates the region type from its container.
The drawback is that it makes the structure bigger. But we get plenty
of helpers. There might be other containers suitable but we use a mix of
append and prepend.
This new module provides the ability to render Core Video pixel buffers pictures into an AVSampleBufferDisplayLayer.
Pictures decoded by a software decoder are converted to Core Video pixel buffers and picture decoded by the VideoToolbox decoder are rendered directly.
With this native API, rendering will be able to handle HDR tone-mapping natively without using OpenGL/libplacebo as it will be backed by Metal.
It also opens the ability to handle Picture in Picture mode on supported Darwin platforms with upcoming additional work.
It currently have the limitation to not support spherical video as it doesn't seems to be feasible at this time.
This module has the highest priority and will be the first vout display to be opened.
Alexandre Janniaux
Steve Lhomme
Maxime Chapelet
Claudio Cambra
Marvin Scholz