FFmpeg


FFmpeg is a free and open-source software project consisting of a large suite of libraries and programs for handling video, audio, and other multimedia files and streams. At its core is the FFmpeg program itself, designed for command-line-based processing of video and audio files, and widely used for format transcoding, basic editing, video scaling, video post-production effects, and standards compliance.
FFmpeg includes libavcodec, an audio/video codec library used by many commercial and free software products, libavformat, an audio/video container mux and demux library, and the core ffmpeg command-line program for transcoding multimedia files.
FFmpeg is part of the workflow of hundreds of other software projects, and its libraries are a core part of software media players such as VLC, and has been included in core processing for YouTube and iTunes. Codecs for the encoding and/or decoding of most audio and video file formats is included, making it highly useful for the transcoding of common and uncommon media files into a single common format.
The name of the project is inspired by the MPEG video standards group, together with "FF" for "fast forward". The logo uses a zigzag pattern that shows how MPEG video codecs handle entropy encoding.
FFmpeg is published under the GNU Lesser General Public License 2.1+ or GNU General Public License 2+.

History

The project was started by Fabrice Bellard in 2000, and was led by Michael Niedermayer from 2004 until 2015. Some FFmpeg developers were also part of the MPlayer project.
On January 10, 2014, two Google employees announced that over 1000 bugs had been fixed in FFmpeg during the previous two years by means of fuzz testing.
In January 2018, the ffserver command-line program – a long-time component of FFmpeg – was removed. The developers had previously deprecated the program citing high maintenance efforts due to its use of internal application programming interfaces.
The project publishes a new release every three months on average. While release versions are available from the website for download, FFmpeg developers recommend that users compile the software from source using the latest build from their source code Git version control system.

Codec history

Two video coding formats with corresponding codecs and one container format have been created within the FFmpeg project so far. The two video codecs are the lossless FFV1, and the lossless and lossy Snow codec. Development of Snow has stalled, while its bit-stream format has not been finalized yet, making it experimental since 2011. The multimedia container format called NUT is no longer being actively developed, but still maintained.
In summer 2010, Fiona Glaser, Ronald Bultje, and David Conrad of the FFmpeg Team announced the ffvp8 decoder. Through testing, they determined that ffvp8 was faster than Google's own libvpx decoder. Starting with version 0.6, FFmpeg also supported WebM and VP8.
In October 2013, a native VP9 and the OpenHEVC decoder, an open source High Efficiency Video Coding decoder, were added to FFmpeg. In 2016 the native AAC encoder was considered stable, removing support for the two external AAC encoders from VisualOn and FAAC. FFmpeg 3.0 retained build support for the Fraunhofer FDK AAC encoder. Since version 3.4 "Cantor"  FFmpeg supported the FITS image format. Since November 2018 in version 4.1 "al-Khwarizmi"  AV1 can be muxed in MP4 and Matroska incl. WebM.

Forks

On March 13, 2011, a group of FFmpeg developers decided to fork the project under the name "Libav". The event was related to an issue in project management, in which developers disagreed with the leadership of FFmpeg.

Components

Command line tools

CPUs

FFmpeg encompasses software implementations of video and audio compressing and decompressing algorithms. These can be compiled and run on diverse instruction sets.
Many widespread instruction sets are supported by FFmpeg, including x86, PPC, ARM, DEC Alpha, SPARC, and MIPS.

Special purpose hardware

Various application-specific integrated circuits related to video and audio compression and decompression exist. Such ASICs can perform the computation for audio/video decompression or compression partly or fully to offload these from the host CPU. To make use of such an ASIC, instead of a complete implementation of some algorithm, only the API is required. There are numerous ASICs and APIs available, of which several are supported by FFmpeg.
FirmASICpurposesupported by FFmpegDetails
AMDUVDdecodingvia VDPAU API and VAAPI
AMDVCEencodingvia VAAPI, considered experimental
AmlogicAmlogic Video Enginedecoding?
BlackMagicDeckLinkencoding/decodingreal-time ingest and playout
BroadcomCrystal HDdecoding
IntelIntel Clear Videodecoding
IntelIntel Quick Sync Videoencoding/decoding
NvidiaPureVideo / NVDECdecodingvia the VDPAU API as of FFmpeg v1.2
via CUVID API as of FFmpeg v3.1
NvidiaNVENCencodingas of FFmpeg v2.6

Use with the FFmpeg Utility

Internal hardware acceleration decoding is enabled through the -hwaccel option. It starts decoding normally, but if a decodable stream is detected in hardware, then the decoder designates all significant processing to that hardware, thus accelerating the decoding process. Whereas if no decodable streams are detected, hardware acceleration will be skipped and it will still be decoded in software. -hwaccel_device option is applied when the hardware requires a particular device to function especially there are several graphic cards are available.

Supported codecs and formats

Image formats

FFmpeg supports many common and some uncommon image formats.
The PGMYUV image format is a homebrewn variant of the binary PGM Netpbm format. FFmpeg also supports 16-bit depths of the PGM and PPM formats, and the binary PAM format with or without alpha channel, depth 8 bit or 16 bit for pix_fmts monob, gray, gray16be, rgb24, rgb48be, ya8, rgba, rgb64be.

Supported formats

In addition to FFV1 and Snow formats, which were created and developed from within FFmpeg, the project also supports the following formats:
GroupFormat typeFormat name
ISO/IEC/ITU-TVideoMPEG-1 Part 2, H.261, H.262/MPEG-2 Part 2, H.263, MPEG-4 Part 2, H.264/MPEG-4 AVC, HEVC/H.265, Motion JPEG, IEC DV video and CD+G
ISO/IEC/ITU-TAudioMP1, MP2, MP3, AAC, HE-AAC, MPEG-4 ALS, G.711 μ-law, G.711 A-law, G.721, G.722, G.722.2, G.723, G.723.1, G.726, G.729, G.729D, IEC DV audio and Direct Stream Transfer
ISO/IEC/ITU-TSubtitleMPEG-4 Timed Text
ISO/IEC/ITU-TImageJPEG, Lossless JPEG, JPEG-LS, JPEG 2000, PNG, CCITT G3 and CCITT G4
Alliance for Open MediaVideoAV1
EIASubtitleEIA-608
CEASubtitleCEA-708
SMPTEVideoSMPTE 314M, SMPTE 370M, VC-1, VC-2, VC-3
SMPTEAudioSMPTE 302M
SMPTEAudioFull Rate, AC-3, Enhanced AC-3 and DTS Coherent Acoustics
SMPTEImageDPX
ATSC/ETSI/DVBSubtitleDVB Subtitling
DVD Forum/DolbyAudioMLP / Dolby TrueHD
DVD Forum/DolbySubtitleDVD-Video subtitles
DTS, Inc/QDesignAudioDTS Coherent Acoustics, DTS Extended Surround, DTS 96/24, DTS-HD High Resolution Audio, DTS Express, DTS-HD Master Audio, QDesign Music Codec 1 and 2
Blu-ray Disc AssociationSubtitlePGS
3GPPAudioAMR-NB, AMR-WB
3GPP2AudioQCELP-8, QCELP-13 and Enhanced Variable Rate Codec
World Wide Web ConsortiumVideoAnimated GIF
World Wide Web ConsortiumSubtitleWebVTT
World Wide Web ConsortiumImageGIF
IETFAudioiLBC, Opus and Comfort noise
International Voice AssociationAudioDSS-SP
SACVideoAVS video
MicrosoftVideoMicrosoft RLE, Microsoft Video 1, Cinepak, Indeo, Microsoft MPEG-4 v1, v2 and v3, Windows Media Video, WMV Screen and Mimic codec
MicrosoftAudioWindows Media Audio, XMA, MS-GSM and MS-ADPCM
MicrosoftSubtitleSAMI
MicrosoftImageWindows Bitmap, WMV Image and DirectDraw Surface
Interactive Multimedia AssociationAudioIMA ADPCM
Digital Video InteractiveVideoRTV 2.1
Digital Video InteractiveAudioDVI4 audio codec
RealNetworksVideoRealVideo Fractal Codec, 1, 2, 3 and 4
RealNetworksAudioRealAudio v1 – v10
RealNetworksSubtitleRealText
AppleVideoCinepak, ProRes, Sorenson 3 Codec, QuickTime Animation, QuickTime Graphics, Apple Video, Apple Intermediate Codec and Pixlet
AppleAudioALAC
Adobe Flash Player VideoScreen video, Screen video 2, Sorenson Spark and VP6
Adobe Flash Player AudioAdobe SWF ADPCM and Nellymoser Asao
Aldus / AdobeImageTIFF and PSD
Xiph.OrgVideoTheora
Xiph.OrgAudioSpeex, Vorbis, Opus and FLAC
Xiph.OrgSubtitleOgg Writ
SonyAudioAdaptive Transform Acoustic Coding and PSX ADPCM
NTTAudioTwinVQ
On2 / GIPS / GoogleVideoDuck TrueMotion 1, Duck TrueMotion 2, Duck TrueMotion 2.0 Real Time, VP3, VP4, VP5, VP6, VP7, VP8, VP9 and animated WebP
On2 / GIPS / GoogleAudioDK ADPCM Audio 3/4, On2 AVC and iLBC
On2 / GIPS / GoogleImageWebP
RAD Game ToolsVideoSmacker video and Bink video
DSP GroupAudioTruespeech
RenderWareVideoTXD
NetpbmImagePBM, PGM, PPM, PNM, PAM and PFM
MIT/X Consortium/The Open GroupImageXBM, XPM and xwd
Silicon GraphicsVideoSilicon Graphics RLE 8-bit video, Silicon Graphics MVC1/2
Silicon GraphicsImageSilicon Graphics Image
Oracle/Sun MicrosystemsImageSun Raster
IBMVideoIBM UltiMotion
Avid Technology / TruevisionVideoAvid 1:1x, Avid Meridien, Avid DNxHD and DNxHR
Avid Technology / TruevisionImageTarga
Autodesk / AliasVideoAutodesk Animator Studio Codec and FLIC
Autodesk / AliasImageAlias PIX
Grass Valley / CanopusVideoHQ, HQA, HQX and Lossless
NewTekVideoSpeedHQ
Industrial Light & Magic / LucasfilmImageOpenEXR
Mozilla CorporationVideoAPNG
MatroxVideoMatrox Uncompressed SD / HD
AMD/ATIVideoATI VCR1/VCR2
AsusVideoASUS V1/V2 codec
Spruce TechnologiesSubtitleSpruce subtitle

Muxers

Output formats in FFmpeg are called "muxers". FFmpeg supports, among others, the following:
FFmpeg supports many pixel formats. Some of these formats are only supported as input formats. The command ffmpeg -pix_fmts provides a list of supported pixel formats.
FFmpeg does not support IMC1-IMC4, AI44, CYMK, RGBE, Log RGB and other formats. It also does not yet support ARGB 1:5:5:5, 2:10:10:10, or other BMP bitfield formats that are not commonly used.

Supported protocols

Open standards

FFmpeg supports, among others, the following filters.

Audio

Legal aspects

FFmpeg contains more than 100 codecs, most of which use compression techniques of one kind or another. Many such compression techniques may be subject to legal claims relating to software patents. Such claims may be enforceable in countries like the United States which have implemented software patents, but are considered unenforceable or void in member countries of the European Union, for example. Patents for many older codecs, including AC3 and all MPEG-1 and MPEG-2 codecs, have expired.
FFmpeg is licensed under the LGPL license, however, if a particular build of FFmpeg is linked against any GPL libraries, then the entire binary is licensed under the GPL.

Projects using FFmpeg

FFmpeg is used by software such as VLC media player, xine, Shotcut, Cinelerra-GG video editor, Plex, Kodi, Blender, HandBrake, YouTube, and MPC-HC; it handles video and audio playback in Google Chrome, and Linux version of Firefox. Graphical user interface front-ends for FFmpeg have been developed, including XMedia Recode.
FFmpeg is used by ffdshow, LAV Filters, GStreamer FFmpeg plug-in, Perian, OpenMAX IL, and FFmpegInterop to expand the encoding and decoding capabilities of their respective multimedia platform.