Venues and Events
Project Summary
Testing event and venue video captures and optimization methods for best quality/performance across various XR devices.
Project Details
!! update publisher hints with findings!
In native app, 60Mbps h265 8k (7680x3840) is working (using upscaled 5K->8K source via topaz)
Native 8k Insta360 x4 workflow: export from Studio @ 120Mbps hvec -> Topaz nyx3 HD cleanup hvec 60mbps-> Premier for editing -> h.264@60-80Mbps 30fps vbit -> ffmep h265@60Mbps:
ffmpeg -i in.mp4 -c:v hevc_videotoolbox -x265-params lossless=1 -c:a copy out-lossless-hvec.mp4
*hevc_videotoolbox is macOS accellerated lib
ref:
https://developer-docs.magicleap.cloud/docs/guides/features/media-codecs/
Quest supports up to 8K60 courtesy Snapdragon 2 chipset; ML2 published limit is 4K30
raw Premier output (lowbit/med quality default h.264 @preset (35-40Mbps)) runs smooth if precompiled into app (scene?)
Premier -> ffmpeg h.264 output (faststart flag change only) with defaults works with stutter; halves bitrate from 45mbps (above), to 20; quality is lower
Video straight from Premier in h.264 and h.265 does not play. It's possible a VBR <14mbps will fix see notes below; but h.265(HVEC) is still preferred for compression efficiency. H.264 is preferred for preview and compat but will produce a slighlty fatter result (hint: it appears that >4K h.264 is using HVEC encapsulation via newer codec standards but not absolutley sure); h.265 may still be preferred for HDR/10-bit support (?unsubstantiated).
We were previously pressing 1.5K@8Mbps with noticeable visual loss.
Error in app with bad formats is:
Preparing . . . Android video error. Restart app to fix
(doesn't actually fix)
h.265 not makeSEA vid preview compatible but is opti for decoding
Now testing premier h.265 @35(lowbit) -> ffmpeg h.265 crf 20 (35 worked, quality was poor; range is 0-50; 0 is lossless)
ffmpeg -i /in.mp4 -c:v libx265 -crf 25 -c:a copy -movflags faststart /out_360_faststart-hvec-ffmpeg.mp4 is fairly stable for a low/medium action scene
<crf starts to stutter on Quest 3; 20 fails !!this may actually be due to photon and other overhead in full Catapult not just bitrate; anything >@9Mbps causes stutter on Quest 3
!!ffmpeg conversion required for web loading but not precompiled even w/o Unity transcode (dunno why?).
Venues-specific optimizations:
try 5.7k ai scrubbed video (on thumb?)
8k looks great even @5.7k mesh but crashes oculus :( if 5.7k is gooder us it and move on!)
premier sharpened video is better but not as good as ai
5.7 AI processed at same res has weird artifacts
crashes happening when using topaz on mac with gpu
crashes happening w/Premeir+ Media when using VR denoise; looks like a low-level Apple bug w/eGPU. Does not crash using only internal graphics
Remote Assets