{"id":130428,"date":"2021-02-12T21:24:10","date_gmt":"2021-02-12T21:24:10","guid":{"rendered":"https:\/\/developer.apple.com\/news\/?id=rwbholxw"},"modified":"2021-02-12T21:24:10","modified_gmt":"2021-02-12T21:24:10","slug":"support-hdr-video-playback-editing-and-export-in-your-app","status":"publish","type":"post","link":"https:\/\/sickgaming.net\/blog\/2021\/02\/12\/support-hdr-video-playback-editing-and-export-in-your-app\/","title":{"rendered":"Support HDR video playback, editing, and export in your app"},"content":{"rendered":"
\"Four<\/div>\n

You can help people create more vivid and true-to-life video when you support high dynamic range (HDR) in your app. And when you support HDR with Dolby Vision, people with iPhone 12 or iPhone 12 Pro can go even further and shoot, edit, and play cinema-grade videos right from their device. Dolby Vision tuning is provided dynamically to each frame, preserving the intended look of the original shots.<\/p>\n

Here\u2019s how you can provide the best HDR video playback, editing, and export experience.<\/p>\n

Get started with HDR video<\/h3>\n

Your app needs to support iOS 14.1 or later to take advantage of HDR video. To begin, we recommend reviewing a few WWDC sessions which provide a good overview of the process.<\/p>\n

\n
\n
<\/a> <\/section>\n
<\/p>\n

Export HDR media in your app with AVFoundation<\/h4>\n

Discover how to author and export high dynamic range (HDR) content in your app using AVFoundation. Learn about high dynamic range and how you can take advantage of it in your app. We\u2019ll show you how to implement feature sets that allow people to export HDR content, go over supported HDR formats,…<\/p>\n

<\/a> <\/section>\n<\/section>\n<\/section>\n

\n
\n
<\/a> <\/section>\n
<\/p>\n

Edit and play back HDR video with AVFoundation<\/h4>\n

Find out how you can support HDR editing and playback in your macOS app, and how you can determine if a specific hardware configuration is eligible for HDR playback. We’ll show you how to use AVMutableVideoComposition with the built-in compositor and easily edit HDR content, explain how you can use…<\/p>\n

<\/a> <\/section>\n<\/section>\n<\/section>\n

\n

Note: iPhone 12 and iPhone 12 Pro record HDR video in Dolby Vision Profile 8.4, Cross-compatibility ID 4 (HLG) format, using an HEVC (10-bit) codec. This format is designed to be backwards compatible with HLG, allowing existing HEVC decoders to decode as HLG. Video is recorded by the Camera app as a QuickTime File Format (QTFF) movie (.mov extension). Signaling for Dolby Vision in a QTFF movie is similar to signaling in Dolby Vision Streams within the ISO base media file format.<\/em><\/p>\n

\n

Learn more about Dolby Vision Profiles<\/a><\/p>\n

Learn more about Dolby Vision Levels<\/a><\/p>\n

Learn more about Dolby Vision Streams<\/a><\/p>\n

Support HDR video playback in your app<\/h3>\n

Both iOS and macOS support HDR video playback on all eligible devices. Use eligibleForHDRPlayback<\/code> on AVPlayer<\/code> to check for HDR playback support on the current device. In general, the classes AVPlayer<\/code>, AVPlayerlayer<\/code>, or AVSampleBufferDisplayLayer<\/code> can be used to play Dolby Vision video. If your app uses AVPlayer<\/code>, you don\u2019t need to add anything additional to your code: The AVFoundation<\/code> framework automatically sets up an HDR playback pipeline to handle Dolby Vision Profile 8.4 if it detects an asset in Dolby Vision and the device supports HDR playback.<\/p>\n

If your app usesAVSampleBufferDisplayLayer<\/code> to render video, make sure any sample buffers passed to the sample buffer display layer are in formats suitable for HDR and carry Dolby Vision Profile 8.4 per-frame metadata. These sample buffers need to have 10-bit or higher bit-depth. A commonly used 10-bit format is 4:2:0 Y\u2019CbCr video range, represented by kCVPixelFormatType_420YpCbCr10BiPlanarVideoRange<\/code>. The associated OSType<\/code> for this pixel format is \u2019x420\u2019<\/code>.<\/p>\n

If your sample buffers are decoded using VTDecompressionSession<\/code>, you can carry the Dolby Vision Profile 8.4 per-frame metadata in the buffers by using kVTDecompressionPropertyKey_PropagatePerFrameHDRDisplayMetadata<\/code>. This value is true by default.<\/p>\n

Asset inspection<\/strong>
\nAVMediaCharacteristic<\/code> provides options for specifying media type characteristics, including whether a video includes HDR metadata. You can use the Swift media characteristic containsHDRVideo<\/code> to identify whether any segment of a track contains HDR so that your app can render it correctly. In Objective-C, you can use AVMediaCharacteristicContainsHDRVideo<\/code>, defined in AVMediaFormat.h<\/code>. <\/p>\n

After loading the tracks property using the Swift method loadValuesAsynchronously(forKeys:completionHandler:)<\/code>, you can get HDR tracks using tracks(withMediaCharacteristic:)<\/code>. Here\u2019s how you might get all desired HDR tracks:<\/p>\n

let<\/span> hdrTracks =<\/span> asset.tracks(withMediaCharacteristic: .containsHDRVideo)<\/code><\/pre>\n
In a similar fashion, you can use the Objective-C method loadValuesAsynchronouslyForKeys:completionHandler:<\/code> to load the tracks property and obtain the HDR tracks with the method tracksWithMediaCharacteristic:<\/code>, like so:<\/p>\n
NSArray<\/span><AVAssetTrack<\/span> *> *hdrTracks = [asset tracksWithMediaCharacteristic:AVMediaCharacteristicContainsHDRVideo<\/span>];<\/code><\/pre>\n
The hasMediaCharacteristic(_:)<\/code> method can be used to track media characteristics, such as HDR media type, format descriptions, or explicit tagging. For example:<\/p>\n
if<\/span> track.hasMediaCharacteristic(.containsHDRVideo){ }<\/code><\/pre>\n
In Objective-C, you can use the same hasMediaCharacteristic:<\/code> method for explicit tagging, as demonstrated here:<\/p>\n
if<\/span>([track hasMediaCharacteristic:AVMediaCharacteristicContainsHDRVideo<\/span>]){ }<\/code><\/pre>\n
Support HDR video editing and previewing in your app<\/h3>\n
To add HDR content editing to your application, use AVVideoComposition<\/code>. If you\u2019re using the built-in compositor, you can also use the Swift initializer init(asset:applyingCIFiltersWithHandler:)<\/code> or the Objective-C initializer videoCompositionWithAsset:applyingCIFiltersWithHandler:<\/code> with built-in CIFilters<\/code> to easily incorporate an HDR editing pipeline in your app.<\/p>\n
Custom compositors can support HDR content, too: You can use the supportsHDRSourceFrames<\/code> property to indicate HDR capability. For Objective-C, the supportsHDRSourceFrames<\/code> property is a part of the AVVideoCompositing<\/code> protocol defined in AVVideoCompositing.h<\/code>.<\/p>\n
If your custom compositor needs to operate in 10-bit HDR pixel formats, you\u2019ll need to select pixel buffer attributes that your compositor can accept as input by using the sourcePixelBufferAttributes<\/code> property. For Objective-C, this property is found in AVVideoCompositing.h<\/code>. The value of this property is a dictionary which contains pixel buffer attribute keys defined in CoreVideo<\/code> header file CVPixelBuffer.h<\/code>.<\/p>\n
Additionally, to create new buffers for processing, you\u2019ll need the correct pixel buffer attributes required by the video compositor. For this particular purpose, use the property requiredPixelBufferAttributesForRenderContext<\/code>.<\/p>\n
If your app offers video previewing during editing, modifying the pixel values may invalidate the video\u2019s existing dynamic metadata and its usage. Because the Dolby Vision Profile 8.4 metadata is completely transparent, you can use AVPlayerItem<\/code> to drop any invalid metadata during preview-only scenarios, as well as update dynamic metadata during export to reflect changes in the video content. <\/p>\n
To configure HDR settings, you can use the appliesPerFrameHDRDisplayMetadata<\/code> property from AVPlayerItem<\/code> , which defaults to true. In Objective-C, the property defaults to YES and can be found in AVPlayerItem.h<\/code>.<\/p>\n
By default, AVFoundation<\/code> will attempt to use Dolby Vision metadata if present for a video, but you can tell your app to ignore it: Just set the appliesPerFrameHDRDisplayMetadata<\/code> property from AVPlayerItem<\/code> to false in Swift, or NO in Objective-C. If your application is using VTDecompressionSession<\/code> APIs from VideoToolbox<\/code>, you can turn off Dolby Vision tone mapping with kVTDecompressionPropertyKey_PropagatePerFrameHDRDisplayMetadata<\/code>. To use this property in C or Objective-C, make sure to include VideoToolbox in the framework header VTDecompressionProperties.h<\/code>.<\/p>\n
Support HDR export in your app<\/h3>\n
You can support HDR video export in your app when you work with AVAssetWriter<\/code> and HEVC presets.<\/p>\n
Discover presets and AVAssetExportSession<\/strong>
\nAll HEVC presets have been upgraded to support HDR. The output format will match the source format, so if the source file is Dolby Vision Profile 8.4, the exported movie will also be Dolby Vision Profile 8.4. If you need to change the output format, you can use AVAssetWriter<\/code>.<\/p>\n
\n
Note: H.264 presets will convert HDR to Standard Dynamic Range (SDR).<\/em><\/p>\n
\n
In order to preserve Dolby Vision Profile 8.4 during export using AVAssetWriter<\/code>, you must choose a suitable output format, color properties that support Dolby Vision, and a 10-bit profile level. <\/p>\n
To start, note that querying supportedOutputSettingsKeys(for:)<\/code> in Swift or supportedOutputSettingsKeysForConnection:<\/code> in Objective-C provides a list of output settings keys supported for the current device. <\/p>\n
For Dolby Vision export, the video output settings dictionary key AVVideoCompressionPropertiesKey<\/code> allows you to control bit rate, B-frame delivery, I-frame interval, and codec quality. The value associated with this key is an instance of NSDictionary<\/code>. For Objective-C, this key is found in AVVideoSettings.h<\/code>.<\/p>\n
For example, a video output settings dictionary for Dolby Vision in Swift would contain these key\/value pairs:<\/p>\n
let<\/span> videoOutputSettings: [String<\/span>: Any<\/span>] =<\/span> [ AVVideoCodecKey<\/span>: AVVideoCodecType<\/span>.hevc, AVVideoProfileLevelKey<\/span>: kVTProfileLevel_HEVC_Main10_AutoLevel, AVVideoColorPropertiesKey<\/span>: [ AVVideoColorPrimariesKey<\/span>: AVVideoColorPrimaries_ITU_R_2020<\/span>, AVVideoTransferFunctionKey<\/span>: AVVideoTransferFunction_ITU_R_2100_HLG<\/span>, AVVideoYCbCrMatrixKey<\/span>: AVVideoYCbCrMatrix_ITU_R_2020<\/span> ], AVVideoCompressionPropertiesKey<\/span>: [ kVTCompressionPropertyKey_HDRMetadataInsertionMode: kVTHDRMetadataInsertionMode_Auto ]\n]<\/code><\/pre>\n
With Objective-C, your video output settings dictionary would contain the same key\/value pairs:<\/p>\n
NSDictionary<\/span><NSString<\/span>*, id<\/span>>* videoOutputSettings = @{ AVVideoCodecKey<\/span>: AVVideoCodecTypeHEVC<\/span>, AVVideoProfileLevelKey<\/span>: (__bridge NSString<\/span>*)kVTProfileLevel_HEVC_Main10_AutoLevel, AVVideoColorPropertiesKey<\/span>: @{ AVVideoColorPrimariesKey<\/span>: AVVideoColorPrimaries_ITU_R_2020<\/span>, AVVideoTransferFunctionKey<\/span>: AVVideoTransferFunction_ITU_R_2100_HLG<\/span>, AVVideoYCbCrMatrixKey<\/span>: AVVideoYCbCrMatrix_ITU_R_2020<\/span> }, AVVideoCompressionPropertiesKey<\/span>: @{ (__bridge NSString<\/span>*)kVTCompressionPropertyKey_HDRMetadataInsertionMode: (__bridge NSString<\/span>*)kVTHDRMetadataInsertionMode_Auto }\n};<\/code><\/pre>\n
In Objective-C, the key kVTCompressionPropertyKey_HDRMetadataInsertionMode<\/code> and the value kVTHDRMetadataInsertionMode_Auto<\/code> are found in VTDecompressionProperties.h<\/code>.<\/p>\n
In addition to defining key\/value pairs, make sure that the pixel buffers presented to AVAssetWriterInput<\/code> are a 10-bit 4:2:0 Y\u2019CbCr video range represented by ‘x420’ OSType.<\/p>\n
You may elect to use a separate AVAssetReader<\/code> or AVAssetWriter<\/code> model for export. In that case, you can use the VideoToolbox property kVTCompressionPropertyKey_PreserveDynamicHDRMetadata<\/code> and set it to kCFBooleanFalse or false for C\/Objective-C or Swift respectively. When you set the VideoToolbox property, AVAssetWriter<\/code> will automatically recompute the Dolby Vision Profile 8.4 metadata for exporting the file. This should be done as your app modifies the output frames from the AVAssetReader<\/code>.<\/p>\n
Resources<\/h3>\n
Learn more about AVFoundation<\/a><\/p>\n
AVFoundation<\/a><\/p>\n
Video Toolbox<\/a><\/p>\n
Learn more about Dolby Vision Profiles<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
You can help people create more vivid and true-to-life video when you support high dynamic range (HDR) in your app. And when you support HDR with Dolby Vision, people with iPhone 12 or iPhone 12 Pro can go even further and shoot, edit, and play cinema-grade videos right from their device. Dolby Vision tuning is […]<\/p>\n","protected":false},"author":2,"featured_media":130429,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[55],"tags":[],"class_list":["post-130428","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-apple-developer-news"],"_links":{"self":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/130428","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/comments?post=130428"}],"version-history":[{"count":0,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/130428\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/media\/130429"}],"wp:attachment":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/media?parent=130428"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/categories?post=130428"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/tags?post=130428"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}