dav1d shifts up a gear : 0.2 is out!

Jean-Baptiste Kempf

tl;dr: dav1d has its second release

If you want a quick summary of this post, about our AV1 decoder:

• dav1d is really ready for production,
• dav1d has impressive benchmarks on ARM devices,
• dav1d is now fast on 32-bit desktop processors (SSSE3).

Read the following for more details...

A few reminders about dav1d

If you follow this blog, you should know everything about dav1d.

AV1 is a new video codec by the Alliance for Open Media, composed of most of the important Web companies (Google, Facebook, Netflix, Amazon, Microsoft, Mozilla...). AV1 has the potential to be up to 20% better than the HEVC codec, but the patents license is totally free, while HEVC patents licenses are insanely high and very confusing.

The VideoLAN, VLC and FFmpeg communities have started to work on a new decoder, sponsored by the Alliance for Open Media, in order to create the reference optimized decoder for AV1.

Second Release

Today, we release the second version of dav1d, called 0.2.1, Antelope.
You can now safely use the decoder on all platforms, with excellent performance.

For the first release, we showed impressive benchmarks for AVX-2 processors, with up to 5x speedups compared to the reference decoder.

In this release, the focus has been toward ARM devices (32-bit and 64-bit) and desktop processors that did not support AVX-2.

It is important to know that the ARM and SSSE3 optimizations are not finished yet. You should expect more performance in the future.

ARM devices

For the ARM devices, we've been doing both ARMv7 and ARMv8 acceleration. We've been testing on iOS, Windows and Android to be sure that it works fine on all OSes.

On ARMv8, we achieve between 150% and 200% of the speed of aomdec:

On ARMv7, we achieve up to 400% on the SnapDragon 410:

It's interesting to see that dav1d is faster in ARMv7 mode than the reference decoder in ARMv8 mode on the same machine.

iOS and iPhones

The playback on iOS is quite important, since those are the fastest ARM devices, and quite widespread:

Depending on the samples, we have achieve 1080p at 75fps on Summer sample, and 40fps on more complex samples, like Chimera.

Desktop

For the desktop, we focused on SSSE3 optimizations, because they should cover 98% active of the desktop processors.

We also did optimizations for both 32-bit and 64-bit architectures, and not only 64bits, as we did for AVX-2.

In multi-thread scenarios, we show between 2x and 3x gains compared to aomdec:

Get it

You can get the tarball on our FTP: dav1d 0.2.1.

You can get the code and report issues on our gitlab project.

You can also join the project, or sponsor our work, by contacting me

Conclusion

Dav1d 0.2 is now faster than aomdec on all the 4 important architectures (x86/SSSE3, x64/AVX-2, ARMv7, ARMv8).

The speedups we see goes from x2 and x5, and on ARM devices, we are now approaching 1080p60 in software.

We're going to continue acceleration work on SSSE3 and ARM devices, in the next few releases.

March 13, 2019 09:32 PM

Android runtime permissions in one (suspending) function

Geoffrey Métais

This post offers a basic implementation of a single suspending function managing the runtime permission process.

Runtime permission API

If you are reading this, you should already know this API. It is only composed of requestPermissions method, and its related callback onRequestPermissionsResult. But this is a View-level API, not really convenient.

We still are lucky, this API is accessible from fragments, and that will allow us to make it way easier to use.

Headless Fragment

The trick is to make it a headless fragment, i.e. a fragment without a view. It’s still bound to its activity and can fire dialogs, that’s what we want.

override fun onCreate(savedInstanceState: Bundle?) {
    retainInstance = true
    requestPermissions(arrayOf(Manifest.permission.READ_EXTERNAL_STORAGE), PERMISSION_STORAGE_CODE)
}

// No onCreateView() overriding

override fun onRequestPermissionsResult(requestCode: Int, permissions: Array<String>, grantResults: PermissionResults) {
    when (requestCode) {
        PERMISSION_STORAGE_CODE -> {
            if (grantResults.granted()) {
                deferredGrant.complete(true)
                exit()
                return
            } else if (shouldShowRequestPermissionRationale(...)) {
                // We insist
                return
            }
            deferredGrant.complete(false)
            exit()
        }
    }
}
protected fun exit() {
    retainInstance = false
    activity?.run {
        if (!isFinishing) supportFragmentManager
            .beginTransaction()
            .remove(this@BaseHeadlessFragment)
            .commitAllowingStateLoss()
    }
}

Our runtime permission implementation has become modular, it can be called from any activity just by starting this fragment. That’s a first step!

The permission grant result is handled by deferredGrant.complete(boolean), that’s how we can get a suspend function controlling our fragment.

Deferred behavior

We will now leverage it, and create a single function to launch this permission granting process and get its result.
For this, we use a CompletableDeferred, which is a Deferred like the one returned by the async function, but we can complete it by ourselves once we have our result.

All we have to do is to launch our fragment, prepare a CompletableDeferred and await for it.

Voilà! We have our function:

protected val deferredGrant = CompletableDeferred<Boolean>()

suspend fun awaitGrant() = deferredGrant.await()

companion object {
    suspend fun FragmentActivity.getStoragePermission() : Boolean {
        if (canReadStorage()) return true
        return launchFragment().awaitGrant()
    }
}

Profit

Usage is straghtforward:
From a coroutine, any activity can call getStoragePermission(), and it suspends until user has made its choice.
No callback anymore, and it’s callable from any Activity or Fragment in your app \o/

if (getStoragePermission()) {
    proceed()
} else {
    retreat()
}

While the user is asked for the permission you want for your app, coroutine execution is suspended by getStoragePermission() condition. Then, the relevant branch of this if expression will be executed.

March 09, 2019 12:00 AM

How to use AV1 with open source tools

Jean-Baptiste Kempf

AV1 and muxing

If you follow this blog, you should know everything about AV1.

AV1 is a new video codec by the Alliance for Open Media, composed of most of the important Web companies (Google, Facebook, Netflix, Amazon, Microsoft, Mozilla...) and VideoLAN.

AV1 has the potential to be up to 20% better than the HEVC codec, but the patents' license is totally free, while HEVC patents licenses are insanely high and very confusing.

This is the first time, where the Open community is ahead of the MPEG community, notably because AV1 and Opus are both great codecs.

AV1 has mappings to wrap it inside MP4 or MKV. And other mappings are coming, notably for RTP or TS.

So, of course, the open source community has developed tools to support AV1. This post is about how to use those tools.

Tools

FFmpeg

For FFmpeg, integration with libaom was done for both encoding and decoding (and now also dav1d for decoding).

To encode, it is important to activate the --enable-libaom option at ./configure time.
You can get all the options for encoding by using ffmpeg -h encoder=libaom-av1.

simple encode

To encode any file that is played by ffmpeg, just use the -c:v libaom-av1 option:
ffmpeg -i input.mp4 -c:v libaom-av1 -crf 30 -b:v 0 -strict experimental av1_test.mkv

This works, of course, for the mp4 and mkv output formats.

2-pass encode

For 2-pass encoding, use the usual commands, but with the -c:v libaom-av1:
ffmpeg -i input.mp4 -c:v libaom-av1 -strict experimental -b:v 2M -pass 1 -an -f matroska /dev/null && ffmpeg -i input.mp4 -c:v libaom-av1 -strict experimental -b:v 2M -pass 2 -c:a libopus output.mkv

To know more about AV1 in FFmpeg, please use the help or the official documentation.

GPAC

For GPAC, the integration of libaom was done too, and it is quite simple.

For example, to add an av1 stream inside an MP4, just use MP4Box:
MP4Box -add file.av1 file_av1.mp4

And, you can even prepare those AV1/MP4 files for DASH streaming:
MP4Box -dash 1000 -profile onDemand file_av1.mp4

If you want more details, or try encrypting of those streams, please read the GPAC blog.

GStreamer

Gstreamer made several releases supporting the AV1 plugins.

To play an MP4 AV1 file, just use gst-play-1.0 av1.mp4.

To do a simple encode and mux it in MP4:
gst-launch-1.0 videotestsrc num-buffers=300 ! video/x-raw, framerate=30/1, width=320, height=240 ! av1enc ! mp4mux ! filesink location=av1file.mp4

Or, if you want to transmux from MKV to MP4, or vice-versa:
gst-launch-1.0 filesrc location=av1.mkv ! matroskademux ! mp4mux ! filesink location=av1.mp4 gst-launch-1.0 filesrc location=av1.mp4 ! qtdemux ! matroskamux ! filesink location=av1.mkv

Finally, to transcode to AV1 and mux in MP4:
gst-launch-1.0 uridecodebin uri=file:///home/toto/file.avi ! av1enc ! mp4mux ! location=video-av1.mp4

VLC

Of course, VLC has full decoding integration, with libaom, and with dav1d (starting in 3.0.5, in a few days). This will work on all platforms, starting with desktop releases first.

But VLC 4.0 also has full encoding and muxing, in both MP4 and MKV, in the nightly builds. To use, you can try this:
vlc file.avi --sout "#transcode{vcodec=av01}:std{access=file,mux=mkv,dst='output.mkv'}"

MediaInfo

It's quite important to mention that Mediainfo already supports AV1, since version 18.08.

You can use the GUI, or the CLI: mediainfo av1.mkv.

MKVToolnix

Last, but not least, MKVtoolnix, supports AV1 muxing, since v28.0.0.

Conclusion

Please try those tools, to create and play AV1/OPUS files everywhere.

Also, please report any bug you would find in those tools.

December 19, 2018 02:37 PM

First release of dav1d, the AV1 decoder

Jean-Baptiste Kempf

tl;dr: dav1d has now a first release

If you want a quick summary of this post, about our AV1 decoder:

• dav1d is good enough that it has an official release,
• dav1d now covers all the spec and features of AV1 (including 12bits),
• dav1d is very fast on modern desktop and it is getting faster on mobile chips.

Read the following for more details...

A few reminders about dav1d

If you follow this blog, you should know everything about dav1d.

AV1 is a new video codec by the Alliance for Open Media, composed of most of the important Web companies (Google, Facebook, Netflix, Amazon, Microsoft, Mozilla...).

AV1 has the potential to be up to 20% better than the HEVC codec, but the patents license is totally free, while HEVC patents licenses are insanely high and very confusing.

The reference decoder for AV1 is great, but it's a research codebase, so it has a lot to improve.

Therefore, the VideoLAN, VLC and FFmpeg communities have started to work on a new decoder, sponsored by the Alliance of Open Media, in order to create the reference optimized decoder for AV1.

Release

Today, we release the first usable version of dav1d, called 0.1.0, Gazelle.
It means you can use the API, ship the decoder, and expect a bit of support on it.

We launched dav1d, more than 2 months ago, during VDD, and we've been working a lot since:

• we support all the features of AV1, even the less known ones,
• we support, 8, 10, 12 bits, and all chroma subsamplings,
• we support all AV1 files that were shared to us,
• we spent a lot of time to make dav1d very fast, while keeping the binary size manageable.

On modern desktop, dav1d is very fast, compared to other decoders:

You can see more details on my previous post.

More performance to come.

But, since the previous blogpost, we've added more assembly for desktop, and we've merged some assembly for ARMv8, and for older machines (SSSE3).

We're now as fast as libaom, in single-thread, on ARMv8, and faster with more threads.

We've been also merging more SSSE3 code. (I haven't had enough time to bench it).
Which means that we will soon be faster than other decoders, on all platforms.

And, we've been experimenting with shaders, notably for the Film Grain feature.

Get it

You can get the tarball on our FTP: dav1d 0.1.0.

You can get the code and report issues on our gitlab project.

You can also join the project, or sponsor our work, by contacting me

December 11, 2018 02:30 PM

dav1d: performance and completion of the first release

Jean-Baptiste Kempf

tl;dr: dav1d is in a very good shape

If you want a quick summary of this post:

• dav1d now covers all the spec and features of AV1, for 8bits and 10bits depth,
• dav1d is very fast, up to 400% faster (more fps) than the libaom decoder, and very often 100% faster.

Now is the right time to integrate it, in your products!

Read the following for more details...

A few reminders about dav1d

AV1 is a new video codec by the Alliance for Open Media, composed of most of the important Web companies (Google, Facebook, Netflix, Amazon, Microsoft, Mozilla...).

AV1 has the potential to be up to 20% better than the HEVC codec, but the patents license is totally free, while HEVC patents licenses are insanely high and very confusing.

The reference decoder for AV1 is great, but it's a research codebase, so it has a lot to improve.

Therefore, the VideoLAN, VLC and FFmpeg communities have started to work on a new decoder, sponsored by the Alliance of Open Media, in order to create the reference optimized decoder for AV1.

Features

We launched dav1d, exactly 2 months ago, during VDD.

We did a lot of work since. And by "we", I mean mostly the others.
There are now more than 500 commits from 29 contributors from different open source communities. This is a good result for a new open source project.

First, we've completed all the features, including Film Grain, Super-Res, Scaled References, and other more obscure features of the bitstream. This covers both 8 and 10bits, of course.
We also improved the public API.

Then, we've fuzzed the decoder a lot: we are now above 99% of functions covered, and 97% of lines covered on OSS-FUZZ; and we usually fix all the issues in a couple of days. This should assure you a secure decoding for AV1.

Finally, we've written a lot of assembly, mostly for modern desktop CPUs, but the work has been started for mobile and older desktop CPUs.
We even reduced the size of the C code!

Performance

Today, dav1d is very fast on AVX2 processors, which should cover a bit more than 50% of the CPUs used on the desktop. We wrote 95% of the code needed for AVX2, but there is still a bit more achievable.

We're readying the SSE and the ARM optimizations, to do the same. They will be very fast too, in the next weeks.

The following graphs are comparing dav1d and aomdec top-of-the-tree on master branches. (and yes, aomdec has CONFIG_LOWBITDEPTH=1).
This was done on Windows 10 64bits, using precompiled binaries.

The clips are taken from Netflix, Elecard, and Youtube, because they don't use the same parameters in the encoder, and don't have the same bitstream features.
Film Grain is not run on the CPU, so it is not visible here.

Haswell

Here, on Haswell (i7-4710, a 4 year old CPU with 4 cores), are the results:

And reported to in percentage compared to libaom:

We got in average 2.49x, and we even get 3.48x on the Youtube Summer clip!

Zen

With a more modern Zen machine (Ryzen 5 1600, 6 cores HT), here are the results:

And reported to in percentage compared to libaom:

The average is even higher at 3.49x, and we even get 5.27x on the Youtube Summer clip!

Global comparison

If we put both on the same graphs, here is what we have:

Threading

If you listened to our talks during VDD or during demuxed, we explained that dav1d threading was quite innovative, and should scale way better than libaom.

On an even less powerful machine, an i5-4590, with 4 cores/4 threads, here are our results, for the Youtube Summer clip:

You see that dav1d can scale better, in terms of threading, than libaom.

Conclusion

dav1d is very fast, dav1d is almost complete, dav1d is cool.

We're finishing the rough edges for a release soon, so that we can hope that Firefox 65 will ship with dav1d for AV1 decoding.

On the other platforms, SSE and ARM assembly will follow very quickly, and we're already as fast on ARMv8. Stay tuned for more!

I would like to thank Ewout ter Hoeven (EwoutH) from the community who did all the testing, numbers and computations.

November 21, 2018 06:33 PM

The VLC Technical Committee

Jean-Baptiste Kempf

VLC Technical Committee

I'm very proud to present to you the VLC Technical Committee, as elected during the last VDD conference: Denis Charmet, Rémi Denis-Courmont, Hugo Beauzée-Luyssen, Thomas Guillem and David Fuhrmann.

The role of the VLC Technical Committee (TC), is mostly a technical resolution committee, that will arise and decide when there are disagreements and bike-shedding in our community.

Members

The glorious members of this Technical Committee are:

• Denis Charmet, the wisest of the VLC developers, and the most-tampered, claims to have never been in conflict with anyone in the community. Let's hope that does not change. (And he has the Mon€¥, so we have to like him...)

• Rémi Denis-Courmont, the biggest contributor to VLC ever (and still the most active non-sponsored developer around VLC); without him, VLC would not exist anymore; the one that knows more about UB, threads and network than 99.999% of the developers.

• Hugo Beauzée-Luyssen, active on the VideoLAN community since the late 2000s, C++1x lover (yet I saw him write Go, once!), very active on the Medialibrary, compilers, toolchain, CI/CD, code-coverage, fuzzing and other toolings; he also knows about UB, even in C++ (some say he is secretly in love with Windows, but he will deny this). Also member of the board, since quite some time now.

• Thomas Guillem, one of the most (the most?) active on VLC development; knows wayyyyy too much about audio and video outputs, and codecs in VLC, a lot about Android (and too much about Tizen) and other weird OSes (he even has a mac on his desk). Probably the most knowledgeable about VLC, after Rémi. He loves C and will never switch to other punk langages!

• David Fuhrmann, the youngest of the TC, is the macOS/iOS touch of this TC, and knows this weird language called Objective-C. Some people claims he even understands Xcode and the macOS toolchain! But in his every day life, he knows C++ (don't tell Hugo)!

As you can see, in all fairness:

there are 2 people of the board in the TC, 2 out of 5 are VideoLabs employees, no roots are part of the TC, nor am I. They know about C, C++, obj-C, and Linux, Windows, macOS.

Details

The fine prints of this Technical Committee:

• The TC can be contacted to take action, or decide by itself to take an action, on any technical subject that did not reach consensus. If no issue there is, there is no need to call the TC.

• Votes and discussion of the TC are private.

• You can contact the TC at vlc-tc@videolaɴ.org

• The VLC TC cannot take action on community issues or CoC.

• The VLC TC can be fired by the GA or any other VideoLAN meeting with the majority of votes.

May they do good work! Good luck to them!

October 25, 2018 10:27 PM

PIC versus pic

Rémi Denis-Courmont

Have you ever wondered what was the difference between the -fpic and -fPIC compiler command line flags were?

October 10, 2018 07:48 PM

Introducing dav1d: a new AV1 decoder

Jean-Baptiste Kempf

Introducing dav1d

AV1 is a new video codec by the Alliance for Open Media, composed of most of the important Web companies (Google, Facebook, Netflix, Amazon, Microsoft, Mozilla...).

AV1 has the potential to be up to 20% better than the HEVC codec, but the patents license is totally free, while HEVC patents licenses are insanely high and very confusing.

The reference decoder for AV1 is great, but it's a research codebase, so it has a lot to improve.

Therefore, the VideoLAN, VLC and FFmpeg communities have started to work on a new decoder, sponsored by the Alliance of Open Media.

The goal of this new decoder is:

• be small,
• be as fast as possible,
• be very cross-platform,
• correctly threaded,
• libre and (actually) Open Source.

Without further ado, the code: https://code.videolan.org/videolan/dav1d

Name

dav1d is called dav1d, because Dav1d is an AV1 Decoder

(Yes, that is a recursive acronym, no need to tell us...)

Video

You can see a talk during VDD 2018 about dav1d:

VDD2018 dav1d presentation.

Technical details

Some technical details about dav1d:

• written in C99 (without VLAs),
• has asm in NASM/GAS syntax (no intrinsics),
• uses meson/ninja as buildsystem,
• currently works on x86, x64, ARMv7, ARMv8,
• runs on Windows, Linux, macOS, Android, iOS,
• licensed under BSD.

Performance

Currently the source code of dav1d is 1/10th of lines of code compared to libaom and its weight is 1/3rd of the binary size of libaom.

It currently uses 1/4th of the memory usage of libaom and uses a very limited amount of stack.

Depending on the threads conditions (see the video talk linked above), dav1d is more or less faster than libaom 1.0.0, but slower than libaom HEAD.
dav1d having almost no assembly code yet, this is not surprising, and is actually a good starting point for the future.

Of course, those metrics will evolve once we add more assembly code, and when the project evolves a bit more.

Questions

Is it production-ready?

Not yet, but you can start testing it and check how the API works for you.

Can I help?

Yes! We need C, ASM developers, but also app integrators and testers to give us feedback.

I need to ship an AV1 decoder with my OS, my hardware, my app. Can I do that?

Yes. dav1d is licensed under BSD for this very reason.

Please talk to us, if you need to get adaptations for your use-case (hybrid decoders, or specific platforms, for example).

BSD is not copyleft, why?

We want AV1 to be as popular as possible. This requires fast decoders, running everywhere. Therefore, we want to help everyone, even non-open-source software.

See RMS opinion on this subject.

October 01, 2018 04:59 PM