Opus (audio format)


Opus is a lossy audio coding format developed by the Xiph.Org Foundation and standardized by the Internet Engineering Task Force, designed to efficiently code speech and general audio in a single format, while remaining low-latency enough for real-time interactive communication and low-complexity enough for low-end embedded processors. Opus replaces both Vorbis and Speex for new applications, and several blind listening tests have ranked it higher-quality than any other standard audio format at any given bitrate until transparency is reached, including MP3, AAC, and HE-AAC.
Opus combines the speech-oriented LPC-based SILK algorithm and the lower-latency MDCT-based CELT algorithm, switching between or combining them as needed for maximal efficiency. Bitrate, audio bandwidth, complexity, and algorithm can all be adjusted seamlessly in each frame. Opus has the low algorithmic delay necessary for use as part of a real-time communication link, networked music performances, and live lip sync; by trading-off quality or bitrate, the delay can be reduced down to 5 ms. Its delay is exceptionally low compared to competing codecs, which require well over 100 ms, yet Opus performs very competitively with these formats in terms of quality per bitrate.
As an open format standardized through RFC 6716, a reference implementation called libopus is available under the New BSD License. The reference has both fixed-point and floating-point optimizations for low- and high-end devices, with SIMD optimizations on platforms that support them. All known software patents that cover Opus are licensed under royalty-free terms. Opus is widely used as the voice-over-IP codec in applications such as WhatsApp and the PlayStation 4.

Features

Opus supports constant and variable bitrate encoding from 6 kbit/s to 510 kbit/s, frame sizes from 2.5 ms to 60 ms, and five sampling rates from 8 kHz to 48 kHz. An Opus stream can support up to 255 audio channels, and it allows channel coupling between channels in groups of two using mid-side coding.
Opus has very short latency, which makes it suitable for real-time applications such as telephony, Voice over IP and videoconferencing; research by Xiph led to the CELT codec, which allows the highest quality while maintaining low delay. In any Opus stream, the bitrate, bandwidth, and delay can be continually varied without introducing any distortion or discontinuity; even mixing packets from different streams will cause a smooth change, rather than the distortion common in other codecs. Unlike Vorbis, Opus does not require large codebooks for each individual file, making it more efficient for short clips of audio and more resilient.
As an open standard, the algorithms are openly documented, and a reference implementation is published. Broadcom and the Xiph.Org Foundation own software patents on some of the CELT algorithms, and Skype Technologies/Microsoft own some on the SILK algorithms; each offers a royalty-free perpetual for use with Opus, reserving only the right to make use of their patents to defend against infringement suits of third parties. Qualcomm, Huawei, France Telecom, and Ericsson have claimed that their patents may apply, which Xiph's legal counsel denies, and none have pursued any legal action. The Opus license automatically and retroactively terminates for any entity that attempts to file a patent suit.
of Opus-encoded audio as bitrate rises clearly shows lowpass behavior and better preservation of the band energy with CELT.
The Opus format is based on a combination of the full-bandwidth CELT format and the speech-oriented SILK format, both heavily modified: CELT is based on the modified discrete cosine transform that most music codecs use, using CELP techniques in the frequency domain for better prediction, while SILK uses linear predictive coding and an optional Long-Term Prediction filter to model speech. In Opus, both were modified to support more frame sizes, as well as further algorithmic improvements and integration, such as using CELT's range encoder for both types. To minimize overhead at low bitrates, if latency is not as pressing, SILK has support for packing multiple 20 ms frames together, sharing context and headers; SILK also allows Low Bit-Rate Redundancy frames, allowing low-quality packet loss recovery. CELT includes both spectral replication and noise generation, similar to AAC's SBR and PNS, and can further save bits by filtering out all harmonics of tonal sounds entirely, then replicating them in the decoder. Better tone detection is an ongoing project to improve quality.
The format has three different modes: speech, hybrid, and CELT. When compressing speech, SILK is used for audio frequencies up to 8 kHz. If wider bandwidth is desired, a hybrid mode uses CELT to encode the frequency range above 8 kHz. The third mode is pure-CELT, designed for general audio. SILK is inherently VBR and cannot hit a bitrate target, while CELT can always be encoded to any specific number of bytes, enabling hybrid and CELT mode when CBR is required.
SILK supports frame sizes of 10, 20, 40 and 60 ms. CELT supports frame sizes of 2.5, 5, 10 and 20 ms. Thus, hybrid mode only supports frame sizes of 10 and 20 ms; frames shorter than 10 ms will always use CELT mode. A typical Opus packet contains a single frame, but packets of up to 120 ms are produced by combining multiple frames per packet. Opus can transparently switch between modes, frame sizes, bandwidths, and channel counts on a per-packet basis, although specific applications may choose to limit this.
The reference implementation is written in C and compiles on hardware architectures with or without a floating-point unit, although floating-point is currently required for audio bandwidth detection and most speed optimizations.

Containers

Opus packets are not self-delimiting, but are designed to be used inside a container of some sort which supplies the decoder with each packet's length. Opus was originally specified for encapsulation in Ogg containers, specified as audio/ogg; codecs=opus, and for Ogg Opus files the .opus filename extension is recommended. Opus streams are also supported in Matroska, WebM, MPEG-TS, and MP4.
Alternatively, each Opus packet may be wrapped in a network packet which supplies the packet length. Opus packets may be sent over an ordered datagram protocol such as RTP.
An optional self-delimited packet format is defined in an appendix to the specification. This uses one or two additional bytes per packet to encode the packet length, allowing packets to be concatenated without encapsulation.

Bandwidth and sampling rate

Opus allows the following bandwidths during encoding. Opus compression does not depend on the input sample rate; timestamps are measured in 48 kHz units even if the full bandwidth is not used. Likewise, the output sample rate may be freely chosen. For example, audio can be input at 16 kHz yet be set to encode only narrowband audio.
AbbreviationAudio
bandwidth		Effective
sample rate
NB 4 kHz8 kHz
MB 6 kHz12 kHz
WB 8 kHz16 kHz
SWB 12 kHz24 kHz
FB 20 kHz48 kHz

History

Opus was proposed for the standardization of a new audio format at the IETF, which was eventually accepted and granted by the codec working group. It is based on two initially separate standard proposals from the Xiph.Org Foundation and Skype Technologies S.A.. Its main developers are Jean-Marc Valin, Koen Vos, and Timothy B. Terriberry. Among others, Juin-Hwey Chen, Gregory Maxwell, and Christopher Montgomery were also involved.
The development of the CELT part of the format goes back to thoughts on a successor for Vorbis under the working name Ghost. As a newer speech codec from the Xiph.Org Foundation, Opus replaces Xiph's older speech codec Speex, an earlier project of Jean-Marc Valin. CELT has been worked on since November 2007.
The SILK part has been under development at Skype since January 2007 as the successor of their SVOPC, an internal project to make the company independent from third-party codecs like iSAC and iLBC and respective license payments.
In March 2009, Skype suggested the development and standardization of a wideband audio format within the IETF. Nearly a year passed with much debate on the formation of an appropriate working group. Representatives of several companies which were taking part in the standardization of patent-encumbered competing formats stated objections against the start of the standardization process for a royalty-free format: representatives of Polycom and Ericsson—the creators and licensors of G.719—as well as France Télécom, Huawei and the Orange Labs, which were involved in the creation of G.718. The working group finally formed in February 2010, and even the corresponding Study Group 16 from the ITU-T pledged to support its work.
In July 2010, a prototype of a hybrid format was presented that combined the two proposed format candidates SILK and CELT. In September 2010, Opus was submitted to the IETF as proposal for standardization. For a short time the format went under the name of Harmony before it got its present name in October 2010. At the beginning of February 2011, the bitstream format was tentatively frozen, subject to last changes. Near the end of July 2011, Jean-Marc Valin was hired by the Mozilla Corporation to continue working on Opus. In November 2011, the working group issued the last call for changes on the bitstream format. The bitstream has been frozen since January 8, 2012. On July 2, 2012, Opus was approved by the IETF for standardization. The reference software entered release candidate state on August 8, 2012. The final specification was released as RFC 6716 on September 10, 2012. and versions 1.0 and 1.0.1 of the reference implementation libopus were released the day after.
On July 11, 2013, libopus 1.0.3 brought bug fixes and a new Surround sound API that improves channel allocation and quality, especially for LFE.

1.1

On December 5, 2013, libopus 1.1 was released, incorporating overall speed improvements and significant encoder quality improvements: Tonality estimation boosts bitrate and quality for previously , like harpsichords; automated speech/music detection improves quality in mixed audio; mid-side stereo reduces the bitrate needs of many songs; band precision boosting for improved transients; and DC rejection below 3 Hz. Two new VBR modes were added: unconstrained for more consistent quality, and temporal VBR that boosts louder frames and generally improves quality.
libopus 1.1.1 was released on November 26, 2015, and 1.1.2 on January 12, 2016, both adding speed optimizations and bug fixes. July 15, 2016 saw the release of version 1.1.3 and includes bug fixes, optimizations, documentation updates and experimental Ambisonics work. libopus 1.2 Beta was released on May 24, 2017.

1.2

libopus 1.2 was released on June 20, 2017. Improvements brought in 1.2 allow it to create fullband music at bit rates as low as 32 kbit/s, and wideband speech at just 12 kbit/s.
libopus 1.2 includes optional support for the decoder specification changes made in drafts of RFC 8251, improving the quality of output from such low-rate streams.

1.3

libopus 1.3 was released on October 18, 2018. The Opus 1.3 major release again brings quality improvements, new features, and bug fixes. Changes since 1.2.x include:
Notable bug fixes include:
libopus 1.3.1 was released on April 12, 2019. This Opus 1.3.1 minor release fixes an issue with the analysis on files with digital silence, especially on x86 builds. It also includes two new features:
Opus has been shown to have excellent quality, and at higher bit rates, it turns out to be competitive with audio formats with much higher delay, such as HE-AAC and Vorbis.
In listening tests around 64 kbit/s, Opus shows superior quality compared to HE-AAC codecs, which were previously dominant due to their use of the patented spectral band replication technology. In listening tests around 96 kbit/s, Opus shows slightly superior quality compared to AAC and significantly better quality compared to Vorbis and MP3.
Opus has very low algorithmic delay, a necessity for use as part of a low-audio-latency communication link, which can permit natural conversation, networked music performances, or lip sync at live events. Total algorithmic delay for an audio format is the sum of delays that must be incurred in the encoder and the decoder of a live audio stream regardless of processing speed and transmission speed, such as buffering audio samples into blocks or frames, allowing for window overlap and possibly allowing for noise-shaping look-ahead in a decoder and any other forms of look-ahead, or for an MP3 encoder, the use of bit reservoir.
Total one-way latency below 150 ms is the preferred target of most VoIP systems, to enable natural conversation with turn-taking little affected by delay. Musicians typically feel in-time with up to around 30 ms audio latency, roughly in accord with the fusion time of the Haas effect, though matching playback delay of each user's own instrument to the round-trip latency can also help. It is suggested for lip sync that around 45–100 ms audio latency may be acceptable.
Opus permits trading-off reduced quality or increased bitrate to achieve an even smaller algorithmic delay. While the reference implementation's default Opus frame is 20.0 ms long, the SILK layer requires a further 5.0 ms lookahead plus 1.5 ms for resampling, giving a default delay of 26.5 ms. When the CELT layer is active, it requires 2.5 ms lookahead for window overlap to which a matching delay of 4.0 ms is added by default to synchronize with the SILK layer. If the encoder is instantiated in the special restricted low delay mode, the 4.0 ms matching delay is removed and the SILK layer is disabled, permitting the minimal algorithmic delay of 5.0 ms.

Support

The format and algorithms are openly documented and the reference implementation is published as free software. Xiph's reference implementation is called libopus and a package called opus-tools provides command-line encoder and decoder utilities. It is published under the terms of a BSD-like license. It is written in C and can be compiled for hardware architectures with or without a floating-point unit. The accompanying diagnostic tool opusinfo reports detailed technical information about Opus files, including information on the standard compliance of the bitstream format. It is based on ogginfo from the vorbis-tools and therefore — unlike the encoder and decoder — is available under the terms of version 2 of the GPL.

Implementations

contains a complete source code for the reference implementation written in C. RFC contains errata.
The FFmpeg project has encoder and decoder implementations not derived from the reference library.
The libopus reference library has been ported to both C# and Java as part of a project called Concentus. These ports sacrifice performance for the sake of being easily integrated into cross-platform applications.

Software

– a digital radio format for AM frequencies – can broadcast and receive Opus audio using Dream software-defined radio.
The Wikimedia Foundation sponsored a free and open source online JavaScript Opus encoder for browsers supporting the required HTML5 features.
Signal switched from Speex to Opus audio codec for better audio quality in the beginning of 2017.

Operating system support

Most end-user software relies on multimedia frameworks provided by the operating system. Native Opus codec support is implemented in most major multimedia frameworks for Unix-like operating systems, including GStreamer, FFmpeg, and Libav libraries.
Google added native support for Opus audio playback in Android 5.0 "Lollipop". However, it was limited to Opus audio encapsulated in Matroska containers, such as .mkv and .webm files. Android 7.0 "Nougat" introduced support for Opus audio encapsulated in .ogg containers. Android 10 finally added native support for .opus extensions.
Due to the addition of WebRTC support in Apple's WebKit rendering engine, macOS High Sierra and iOS 11 come with native playback support for Opus audio encapsulated in Core Audio Format containers.
On Windows 10, version 1607, Microsoft provided native support for Opus audio encapsulated in Matroska and WebM files. On version 1709, support for Opus audio encapsulated in .ogg containers was made available through a pre-installed add-on called Web Media Extensions. On Windows 10 version 1903, native support for the .opus container was added. On Windows 8.1 and older, third-party decoders, such as LAV Filters, are available to provide support for the format.
WindowsmacOSLinuxAndroidiOS
Codec support
Container supportOn Windows 10 Anniversary Update :
WebM
Matroska

On Windows 10 Fall Creators Update with Web Media Extensions add-on:
Ogg

On Windows 10 October 2018 Update :
WebM

On Windows 10 May 2019 Update :

Ogg

On macOS High Sierra:
Core Audio Format 		Ogg
WebM
Matroska
MPEG-TS 		On Android 5:
WebM
Matroska

On Android 7:
Ogg

On Android 10:

Ogg

On iOS 11:
Core Audio Format
NotesOn Windows 10:
- On Anniversary Update, limited support is available in Microsoft Edge and Universal Windows Platform apps. Windows Media Player does not support Ogg; only WebM and Matroska.
- On April 2018 Update with preinstalled, Microsoft Edge supports Opus audio embedded in 		Only Opus audio in a CAF container is supported.–- Up till Android 9, the filename extension.opus was not recognised Only Opus audio in a CAF container is supported.

Media player support

While support in multimedia frameworks automatically enables Opus support in software which is built on top of such frameworks, several applications developers made additional efforts for supporting the Opus audio format in their software. Such support was added to AIMP, Amarok, cmus, Music Player Daemon, foobar2000, Mpxplay, MusicBee, SMplayer, VLC media player, Winamp and Xmplay audio players; Icecast, Airtime audio streaming software; and Asunder audio CD ripper, CDBurnerXP CD burner, FFmpeg, Libav and MediaCoder media encoding tools. Streaming Icecast radio trials are live since September 2012 and January 2013. SteamOS uses Opus or Vorbis for streaming audio.

Browser support

Opus support is mandatory for WebRTC implementations. Opus is supported in Mozilla Firefox, Chromium and Google Chrome, Blink-based Opera, as well as all browsers for Unix-like systems relying on GStreamer for multimedia formats support. Although Internet Explorer will not provide Opus playback natively, support for the format is built into the Edge browser, along with VP9, for full WebM support. Safari supports Opus as of iOS 11 and macOS High Sierra.

VoIP support

Due to its abilities, Opus gained early interest from voice-over-IP software vendors. Several SIP clients, including Acrobits Softphone, CSipSimple, Empathy, Jitsi, Tuenti, Line2, Linphone, Phoner and PhonerLite, SFLphone, Telephone, Mumble, Discord and TeamSpeak 3 voice chat software also support Opus. TrueConf supports Opus in its VoIP products. Asterisk lacked builtin Opus support for legal reasons, but a third-party patch was available for download and official support via a binary blob was added in September 2016. Tox P2P videoconferencing software uses Opus exclusively. Classified-ads distributed messaging app sends raw opus frames inside TLS socket in its VoIP implementation.
Opus is widely used as the voice codec in WhatsApp, which has over 1.5billion users worldwide. WhatsApp uses Opus at 816 kHz sampling rates, with the Real-time Transport Protocol. The PlayStation 4 video game console also uses the CELT/Opus codec for its PlayStation Network system party chat.

Hardware

Since version 3.13, Rockbox enables Opus playback on supported portable media players, including some products from the iPod series by Apple, devices made by iriver, Archos and Sandisk, and on Android devices using "Rockbox as an Application". All recent Grandstream IP phones support Opus audio both for encoding and decoding. OBihai OBi1062, OBi1032 and OBi1022 IP phones all support Opus. Recent BlueSound wireless speakers support Opus playback. Devices running Hiby OS, like the Hiby R3, are capable of decoding Opus files natively.
Many broadcast IP codecs include Opus such as those manufactured by Comrex, GatesAir and Tieline.