Comments on: Embedded Theora Video http://www.humboldt.co.uk/2006/02/embedded-theora-video.html Software Development and Consulting Thu, 24 Jul 2008 04:28:57 +0000 http://wordpress.org/?v=2.3.3 By: Timothy B. Terriberry http://www.humboldt.co.uk/2006/02/embedded-theora-video.html#comment-5 Timothy B. Terriberry Fri, 01 Dec 2006 18:09:00 +0000 http://www.humboldt.co.uk/2006/02/embedded-theora-video.html#comment-5 The real problem, as I saw it, was not the raster/Hilbert curve ordering (though this certainly introduces many complexities in the encoder; the decoder is not as bad), but the fact that coefficients are ordered by their frequency first, and the block they belong to second, meaning that if the 63rd coefficient in a block is non-zero, you need to decode virtually all the coefficients in the frame before you can decode that one block.<br/><br/>This structure actually helps avoid the raster order problem for DC prediction, because you can undo the DC prediction after decoding just the DC coefficients, i.e., on 1/64th of the final image data, which often fits entirely in cache (at 640x480 it's less than 16k of data).<br/><br/>The loop filter problem can similarly be handled by applying it (with a one block row delay) after each super block row is decoded. This requires slightly more cache (just under 64k of image data at 640x480, plus additional overhead for the coefficients that were decoded), but should still fit well within the 256K L2 cache on your DSP.<br/><br/>The <a HREF="http://svn.xiph.org/trunk/theora-exp/" REL="nofollow">theora-exp</a> decoder does both these things, to good effect. The spec describes everything as separate processes for conceptual simplicity, but to get good preformance they really need to be pipelined.<br/><br/>For the loop filter at larger resolutions, you could (if you're careful about it), process 3/4 of each superblock (with a one block column delay) immediately after decoding it, without waiting for the entire row to finish. Then you only need to cache one row of block data between superblock rows. The real problem, as I saw it, was not the raster/Hilbert curve ordering (though this certainly introduces many complexities in the encoder; the decoder is not as bad), but the fact that coefficients are ordered by their frequency first, and the block they belong to second, meaning that if the 63rd coefficient in a block is non-zero, you need to decode virtually all the coefficients in the frame before you can decode that one block.

This structure actually helps avoid the raster order problem for DC prediction, because you can undo the DC prediction after decoding just the DC coefficients, i.e., on 1/64th of the final image data, which often fits entirely in cache (at 640×480 it’s less than 16k of data).

The loop filter problem can similarly be handled by applying it (with a one block row delay) after each super block row is decoded. This requires slightly more cache (just under 64k of image data at 640×480, plus additional overhead for the coefficients that were decoded), but should still fit well within the 256K L2 cache on your DSP.

The theora-exp decoder does both these things, to good effect. The spec describes everything as separate processes for conceptual simplicity, but to get good preformance they really need to be pipelined.

For the loop filter at larger resolutions, you could (if you’re careful about it), process 3/4 of each superblock (with a one block column delay) immediately after decoding it, without waiting for the entire row to finish. Then you only need to cache one row of block data between superblock rows.

]]> By: Adrian Cox http://www.humboldt.co.uk/2006/02/embedded-theora-video.html#comment-3 Adrian Cox Wed, 15 Feb 2006 21:29:00 +0000 http://www.humboldt.co.uk/2006/02/embedded-theora-video.html#comment-3 Interesting question, which I'd like to look into later. Dirac has advanced to a stage where someone could do the experiment, but optimisation for DSP often involves different tactics to optimisation on a desktop platform. Interesting question, which I’d like to look into later. Dirac has advanced to a stage where someone could do the experiment, but optimisation for DSP often involves different tactics to optimisation on a desktop platform.

]]> By: Stefan de Konink http://www.humboldt.co.uk/2006/02/embedded-theora-video.html#comment-2 Stefan de Konink Sun, 12 Feb 2006 01:59:00 +0000 http://www.humboldt.co.uk/2006/02/embedded-theora-video.html#comment-2 Makes me wondering how much effort it would take to put an embedded version of <a HREF="http://dirac.sf.net/" REL="nofollow">Dirac</a> inside. After all, Theora is Open and Free, but stays VP3. So from a codec perspective it is an old codec based on older technology.<br/><br/>While the <a HREF="http://schrodinger.sourceforge.net/" REL="nofollow">new Dirac project</a> is working on performance on x86, makes you wonder how well it could work on a DSP. Makes me wondering how much effort it would take to put an embedded version of Dirac inside. After all, Theora is Open and Free, but stays VP3. So from a codec perspective it is an old codec based on older technology.

While the new Dirac project is working on performance on x86, makes you wonder how well it could work on a DSP.

]]>