Immersive audio-visual systems extend conventional viewing by letting audiences look around a scene, choose a vantage point or interact with programme content. For broadcast engineers and technical decision-makers, the challenge is not simply producing a 360-degree image. Visuals, audio, tracking, processing and delivery must work together; when they do not, the sense of presence can weaken and viewers may experience discomfort.
Virtual reality (VR) has been a particular focus for broadcasters, while augmented reality (AR) overlays or combines digital material with the physical world. The approaches differ, but both make content presentation and user interaction part of the system design.
What makes immersive audio-visual systems different?
Immersion depends on several connected qualities: visual fidelity, sound quality and intuitive interaction. A viewer using a head-mounted display (HMD) may turn their head to inspect a scene, so the displayed image and spatial audio need to respond appropriately. Motion tracking and low system latency help keep the experience aligned with the viewer’s movements.
Presentation mode shapes how much interaction is possible. An HMD can provide a VR view that obscures the surrounding physical environment, or an AR view that keeps it visible. A “magic window” lets users explore 360-degree video on a phone, computer or television by dragging the image or moving a sensor-equipped device. It offers directional exploration without the full immersion of an HMD. A second-screen experience can instead complement a regular television programme with synchronized alternative viewpoints or information.
Degrees of freedom define how viewers move
Immersive programme formats can be distinguished by the movement they support:
- 3DoF: The viewer can rotate their head to look in different directions—using yaw, pitch and roll—but remains at one observation point.
- 3DoF+: Rotation is supplemented by limited head translation, which can support effects such as motion parallax.
- 6DoF: The viewer can rotate and move through the scene, with position changes represented along the x, y and z axes.
These distinctions affect capture, rendering and interaction requirements. A fixed observation point can suit a seated 360-degree experience, while movement through a scene requires the system to represent changes in position as well as orientation. Multiple vantage points offer another approach: viewers choose between discrete captured locations rather than moving freely through continuous space.
Broadcast formats range from live events to virtual screens
One live-event concept is the “infinite seat” broadcast. Camera and microphone rigs are placed at different locations, with each location providing a distinct viewing and listening position. Viewers may select a seat and look around from it. Producers can also add elements such as commentary, statistics, replays or cuts between viewpoints.
Other use cases include cinematic VR, in which a viewer follows a linear story from a fixed position, and free-viewpoint television, which allows a three-dimensional scene to be viewed from different positions. HMDs can also present conventional two-dimensional programmes on a virtual screen, with an optional virtual environment around it. These formats can extend existing content, but they do not all require the same production or delivery approach.
Production workflows must account for viewer control
Conventional video production gives editors and directors substantial control over what the audience sees and when. In immersive content, viewers may choose where to look. That changes how producers guide attention: rapid cuts, fast camera movements and sudden viewpoint changes can be uncomfortable, so familiar cinematic techniques may need to be used carefully.
A 360-degree frame also leaves less room to conceal equipment and crew. Production teams may need to hide them behind set elements or address visibility during post-production. Workflow stages can include footage ingestion, stitching, asset creation, audio production, tracking, layout, rendering, review, delivery and archiving. Immersive projects can involve substantial quantities of media data, and linear productions have different authoring needs from experiences that change in response to user interaction.
Audio is part of the scene, not an independent afterthought. Sound may need to remain spatially anchored as a viewer turns, or it may be designed as an overlay that moves with the display. The distinction affects how sound is authored and experienced. Production and playback systems therefore need to preserve the intended relationship between image, sound and head movement.
Delivery and playback remain significant constraints
High-resolution immersive video can place heavy demands on network throughput and consumer-device processing. Buffering interrupts continuity, while limited processing resources can affect playback. Some streaming approaches send higher-quality video for the viewer’s current field of view and lower-quality video for the periphery. For this method to work smoothly, the system must switch between viewports without a visible disruption as the viewer turns.
Broadcast delivery presents a different constraint: transmitting the complete panoramic image may be more appropriate than sending an individually selected viewport to each viewer. The ITU report also describes the Omnidirectional Media Format (OMAF), standardized as ISO/IEC 23090-2, which specifies a format for immersive media and IP-based delivery methods using MMT and MPEG-DASH.
Playback quality is not just a resolution question. Frame rate, motion-to-photon latency and display characteristics can affect realism and comfort. HMDs also vary, so the same content may not be presented consistently across devices. Viewer sensitivity, exposure duration and the motion in the content can all contribute to sensory sickness; no single technical adjustment eliminates that concern.
Practical implications for technical teams
Immersive production trials have covered sports, news, documentaries, music and special events. The experience can give audiences a strong sense of presence and control over their viewpoint, but trials have also highlighted requests for higher image quality and the need to consider motion sickness, younger viewers and device limitations.
For teams assessing a broadcast VR or AR project, the main decisions are interconnected:
- Choose a presentation mode and degree of freedom that fit the programme.
- Plan camera placement, audio capture and storytelling around viewer-controlled attention.
- Define how the content will be stitched, authored, encoded and distributed.
- Test playback across intended devices, with attention to latency, image quality and buffering.
- Consider comfort and provide suitable alternatives where an HMD is not appropriate.
Immersive audio-visual systems create new options for presenting broadcast content, but they also extend the engineering problem across the full production chain. A compelling result depends on aligning capture, authoring, delivery and playback with the interaction the programme promises. For technical decision-makers, early coordination across those stages is essential to balancing viewer freedom with quality, reliability and comfort.