unlord • 3 days ago

Someone asked about benchmarks, so I ran these just now to bring data to the discussion:

  nathan@arm1:~/git/rav1d.new/target$ hyperfine --warmup 2 "release/dav1d -q -i ~/Chimera-AV1-8bit-1920x1080-6736kbps.ivf -o /dev/null --threads 8"
  Benchmark 1: release/dav1d -q -i ~/Chimera-AV1-8bit-1920x1080-6736kbps.ivf -o /dev/null --threads 8
    Time (mean ± σ):     31.532 s ±  1.971 s    [User: 244.512 s, System: 1.644 s]
    Range (min … max):   28.498 s … 34.270 s    10 runs

  nathan@arm1:~/git/dav1d.new/build$ hyperfine --warmup 2 "tools/dav1d -q -i ~/Chimera-AV1-8bit-1920x1080-6736kbps.ivf -o /dev/null --threads 8"
  Benchmark 1: tools/dav1d -q -i ~/Chimera-AV1-8bit-1920x1080-6736kbps.ivf -o /dev/null --threads 8
    Time (mean ± σ):     29.696 s ±  2.308 s    [User: 230.507 s, System: 1.479 s]
    Range (min … max):   26.618 s … 35.105 s    10 runs
It shows that as of this moment rav1d is (31.532 - 29.696)/29.696 * 100 = 6.2% slower to decode this Netflix test sequence. Note, this is an improvement from the (32.775 - 29.696)/29.696 * 100 = 10.4% when Prossimo posted a bounty for improving rav1d [1]:

  nathan@arm1:~/git/rav1d.old/target$ hyperfine --warmup 2 "release/dav1d -q -i ~/Chimera-AV1-8bit-1920x1080-6736kbps.ivf -o /dev/null --threads 8"
  Benchmark 1: release/dav1d -q -i ~/Chimera-AV1-8bit-1920x1080-6736kbps.ivf -o /dev/null --threads 8
    Time (mean ± σ):     32.775 s ±  2.694 s    [User: 254.120 s, System: 1.659 s]
    Range (min … max):   28.847 s … 37.606 s    10 runs
None of this is particularly new, I reported this gap in performance over a year ago [2]. Here are some questions for HN:

Even if rav1d closes the gap, who will ship rav1d over dav1d? More than 85% of dav1d is hand written assembly that rav1d takes as-is and calls via unsafe blocks. Is this really memory safety?

The dav1d improvements must continuously be backported to rav1d, see the 82 closed PR's here [3]. Who is going to pay for this maintenance in perpetuity?

VideoLAN fuzzes dav1d and fixes bugs extremely quickly, typically less than 24h. The rav1d developers are Rust experts, but not codec experts. Is it reasonable to expect the same support?

[1] https://www.memorysafety.org/blog/rav1d-perf-bounty/

[2] https://github.com/memorysafety/rav1d/issues/804

[3] https://github.com/memorysafety/rav1d/pulls?q=backport+is%3A...

3
StopDisinfo910 • 3 days ago

> Even if rav1d closes the gap, who will ship rav1d over dav1d? More than 85% of dav1d is hand written assembly that rav1d takes as-is and calls via unsafe blocks. Is this really memory safety?

> The dav1d improvements must continuously be backported to rav1d, see the 82 closed PR's here [3]. Who is going to pay for this maintenance in perpetuity?

No one?

Isn't the point of rav1d to show that it's possible to write a performant encoder in semi-idiomatic Rust and that it doesn't have to be very different from one written in C?

It seemed pretty clear to me that rav1d was supposed to be a fun "research" side-project from its author shared with everyone and with an unclear future which will depends of what the community wants to actually do with it but I might be completely wrong.

dataking • 3 days ago

Thanks! Is this the same hardware used here http://paste.debian.net/1374884?

Can you share what system was used (ARM CPU, OS, etc.)?

hulitu • 1 day ago

> The dav1d improvements must continuously be backported to rav1d, see the 82 closed PR's here

so basically the work is done in dav1d, but claimed in rav1d.

It really makes rust look like a parasite.