ffmpeg - the multimedia hub


ffmpeg is an audio/video converter. The basic use is to convert an input file/stream to a different output file/stream. Sounds simple? Yeah, but it's also far more complicated.

Multimedia libs in linux is not unique. You have probably (even if not noticed it) been in touch with gstreamer, sox and libav. Diversity is a good thing, but for a minimalistic distro like Puppy, it means we have to take the hard decisions what to support and what to leave.

These days, ffmpeg matures into something more than just one of the other libs. Simply because some major projects make it their preferred multimedia lib.
- Ubuntu and Debian used ffmpeg some years ago, but switched to libav when it forked from ffmpeg. Now they have switched back to ffmpeg. Puppy based on ie. Ubuntu Tahr had major issues with Puppy-tools built upon ffmpeg, like FFconvert and pBurn. pMusic was never included into Puppy Tahr. Slackware on the other side kept ffmpeg, so Puppy Slacko never had these issues.
- Firefox has from version 44 switched from gstreamer to ffmpeg for multimedia support. In a standard Puppy, gstreamer is only used by Firefox. So in a future release, including Firefox will require less dependencies. Gstreamer is not lightweight, so the iso-shrink will be noticeable.

As mentioned, Puppy already includes apps using ffmpeg, but it could also give us more interesting ffmpeg-based apps. Here follows some load thoughts of different ways to use ffmpeg for new projects. That means new apps/features that not require any new backends/dependencies. But let's start to see how it already is in use:

- The original target for ffmpeg was to convert input file to another output file. This is what we see in FFconvert.
ffmpeg -i /path/input_file.mp3 [options] /path/output_file.ogg

This is also what is used by pBurn to make your audio files compatible before burning an audio-CD and any video file for your Video-DVD.

- Depending on the ffmpeg-package, it may or may not contain the ffmpeg media player - ffplay. This is a simple but very useful mediaplayer for video and audio files and streams. It could for sure have been the default media player Puppy, but atm it does not has any control-gui, or a signal-system for building an external gui. Interesting to see how this evolves in the future.
ffplay -i /path/input_file.mp3


- Instead of sending the converted output audio/video stream to an output file, it is possible to send it further to another command. Technically that means sending the stream to stdout and pipe it to next command. This way of using ffmpeg is what pMusic is based upon: Take whatever audio-file format and convert it to raw audio before sending it to the simple audioplayer (aplay) shipped with the audio driver system - alsa.
ffmpeg -i /path/input_file.mp3" -f au - | aplay

This is another solution than using ffplay, and it is likely to think that this has to be hard on the resources on your system. It is not. And there are 3 main benefits: 1.) ffplay is not always bound to the ffmpeg pack. 2.) We can control the soundcard more accurate through aplay than ffplay. 3.) Only very recent versions of ffplay allows sound-filtering of the input stream.

- The input or output can also be a server (a livestream). This webcam recorder is an example of such use. But in the following example I send the output to a local server.
ffmpeg -i /path/input_file.mp3 -f rtp rtp://127.0.0.1:1234

This ffmpeg stream can not be heard until you connect to the server. So, you can (dis)connect to your stream without restarting the audio-file. I have set up this structure for the next pMusic release (5.1.0) to support visualization. ffmpeg has filters to convert audio into a generated video like waves or similar, but managing visualization should not interfere with the main audio stream. Solved by sending the output both the the server and to stdout.
ffmpeg -i /path/input_file.mp3 -f rtp rtp://127.0.0.1:1234 -f au - | aplay

Realize than ffmpeg can mix and split inputs and outputs. It's getting complicated, and I stop here...

- ffmpeg supports more inputs than files and streams. It is capable to grab the output of the X-server (What you see on your screen). That means it is simple to build a screen recorder for ie. youtube howtos. I have wondered why no one have found this to be a fun project for their coding. It's all based on one single command like:
ffmpeg -video_size 1024x768 -framerate 25 -f x11grab -i :0.0+100,200 output.mp4

including the sound from your mic.
ffmpeg -video_size 1024x768 -framerate 25 -f x11grab -i :0.0+100,200 -f alsa -ac 2 -i hw:0 output.mkv


- The last example is interesting because it also grabs the sound from the soundcard. Combine this with a local audio server as shown above and you can tweak (soundfilters like /eq/pitch/vibrato/..) the sound in realtime without interfering with the input soundcard stream. Are we seeing a pRack...

- And I have not mentioned that ffmpeg supports basic video editing like croping, scaling, fading, effects, etc.


These are only a few examples of the usage of this awesome tool. And personally, I find it pleasing to see that it most likely will be included into future Puppies. Here are much unused potential. So, girls and guys - go coding.


Posted on 31 Mar 2016, 06:51 by zigbert - Categories: Development
Edit - Delete


Comments:

Posted on 30 Mar 2016, 10:03 by 01micko
"Nice insight"
It is often little known the power of ffmpeg. I have only ever scratched the surface.

A very nice insight! Thank you.
Delete



Add Comment

Title
Author
 
Content
Show Smilies
Security Code 2320944
The Puppy Linux mascot?
Password (to protect your identity)