dvd-hq.info

[Scott A Taylor]

I am a visual artist who among other things creates abstract digital animations inspired by clouds. The work consists of very slow moving subtle abstract shapes that are constantly morphing. I was using MainConcepts mpeg2-dvd (Adobe Premier Pro 2) but found it very poor in that it washed out my images (I think it only allows CCIR601 output) DVDHQ.info and TMPGEnc have been a god send, now I have excellent contrast and saturation. However I am still experiencing major colour banding in large areas of subtle tone (see image below, actual size)

This banding is very pronounced when the dvd is viewed on a 720i lcd tv. I don't know if I am expecting too much of mpeg2. I would be eternally grateful if anyone could offer advice as to reducing colour banding. My animations are about 8 mins in length so size is not an issue and render times don't bother me.

Here are the settings I used in TMPGEnc 2.5

-PAL 16:9
- CBR, 8 Mb
- DC Component - 10bit
- Motion search high quality
- GOP, I-1, P-4, B-2
- Detect scene change
- Quantize Matrix, CG/Animation, Output basic YCbCr


_________________
Scott A Taylor
visual artist

www.scotttaylor.info

[RMN]

Why are you using the CG matrix? The "Default" TMPGEnc matrix should be significantly better at preserving low frequencies (i.e., smooth colour transitions). The "CG" matrix is mainly for drawings and animations with hard edges (think anime).

A quantize matrix is something like this (only 8x8):

X X X X
X X X X
X X X X
X X X X

Where lower numbers represent more faithful preservation of frequencies and higher numbers represent higher quanization (less steps). The numbers near the top left corner apply to lower frequencies (smooth variations), the numbers near the bottom right corner apply to higher frequencies (hard edges, small details and noise).

There's a (simplified) explanation of how it works in my article about lossy compression.

So what you need for this kind of animation is a matrix where the numbers grow from the top left to the bottom right corner. Of the standard TMPGEnc matrices, "Default" is the best one. You can try tweaking it some more (ex., setting the topmost-leftmost 9 coefficients of both matrices to 8), but it'll probably give you good results as it is.

Also, you can probably use a longer GOP (ex., I=1, P=7, B=1). Even if there are any sudden changes, scene detection will take care of them.

And I suspect the "high quality" motion search is a bit of a waste (motion estimation should produce virtually identical results in less time).

Also, note that some decoders are better than others, and many LCD TVs use 6-bit panels (in other words, at best you'll get 64 intermediate tones, not 256, like in a proper 8-bit panel).

To judge what's really on the DVD, play it back using a good player (of the mainstream brands, Pioneer is probably the best, in my experience), connected to a CRT TV or to an LCD panel that you are sure is 8-bit or better (preferably an IPS or VA panel, not a TN panel, which is the most common - and usually cheaper - type).

RMN
~~~

[Scott A Taylor]

Great, thanks.

I'll admit to not trying the default matrix setting because of its title, 'default' The CG setting gave better results than the Mpeg standard.

Because every section of the animation is constantly morphing I thought less intermediate frames i.e. P and B frames, would result in a higher quality result. In saying that I did try I frames only but didn't see a noticeable improvement.

Render times don't bother me hence the high quality motion search.

I'll give your suggestions a go and get back to you as to the results, I might even try manually configuring the matrix.

Thanks again.

Cheers.
_________________
Scott A Taylor
visual artist

www.scotttaylor.info

[RMN]

It's odd that the "CG" matrix worked better than the "MPEG Standard". It should be significantly better at preserving hard edges, but worse at smooth colour transitions. Then again, the "MPEG Standard" matrix only has a frequency bias for the I-pictures (it preserves all frequencies equally to the P- and B-picture residuals), so for complex motion it won't be very different from the CG. The "Default" matrix should be noticeably better at dealing with the "smooth gradient" + "random motion" combination, as long as there are no small details (like thin, high-contrast lines).

Using I-pictures only will generally require a higher bitrate (over 20 Mb/s) to achieve good quality (depends on the detail in the original images and the matrices, too). At 8 Mb/s, using I-pictures only can make motion slightly more regular, but that comes at the cost of more (JPEG-style) "blockiness" per frame.

The scene change detection option should insert I-pictures if and when it determines that one would achieve better quality than a P-picture, so a longer GOP is generally the way to go. If the "morphing" is fast, then go with P=14 and B=0, instead of P=7 and B=1.

B-pictures will save some space (which will improve the P and I, indirectly) but, if there are a lot of changes, they will make the motion search less efficient, which in turn will make quality worse. By using P-pictures only (plus the starting I, of course), motion search should be able to get better results, which then requires less bits to "clean up". You'll have to experiment a bit to determine the best settings, but P=7 B=1 should be close to ideal.

But, most of all, make sure the banding isn't being caused by the player or LCD panel. LCDs have great geometry, but 99% of the models out there have far worse colour reproduction than even an average CRT.

RMN
~~~

[Scott A Taylor]

I have tried several combinations of what you suggested and have settled on increasing the bitrate to 8.9Mb, GOP 1xI, 2xP and using the CG Matrix changing the top 9 leftmost values to 8 in both fields (still found CG slightly better than the default). I do get a bit of blockiness with the low amount of P frames but I find it breaks up the colour banding a bit, of which there is still a fair amount. The smoothness of the motion is perfect. I think I am expecting too much of dvd so may rethink my animations to compensate at least until I can upgrade to HD DVD or Bluray.

Something I was wondering was the fact I use TMPGEnc to encode the bmp sequence of frames directly and each frame (all 12000) are 1.43mb in size. Would shrinking the frame size improve encoding quality since less compression is needed? If so can you recommend an image format? jpeg and png seem to degrade the quality a bit too much. I did try creating an uncompressed avi with Premier but found it decreased the quality slightly.

Thanks again Rui for your prompt professional advice.
_________________
Scott A Taylor
visual artist

www.scotttaylor.info

[RMN]

The size (in bytes) of the original files has no influence on quality. Regardless of their format, the frames always need to be decoded (in other words, converted into uncompressed bitmaps) before any program can do anything to them (such as create a video file from them or apply filters, etc.).

Make sure the original frames are the correct dimensions (in pixels), though (720x576 or 704x576, assuming you're using PAL), otherwise TMPGEnc has to resize every single frame, which takes a long time and can cause some loss of quality.

PNG is a lossless format, so it won't make any change to the quality (pixels keep exactly the same values). The uncompressed AVI should also be, pixel-for-pixel, identical to the original frames (assuming there wasn't any further processing). The same goes (for example) for an AVI in a format like HuffYUV, which is also lossless.

JPEG, M-JPEG, DV or MPEG are lossy formats (will alter the image).

But there's nothing to be gained by converting to another format, so don't bother. Just make sure the resolution is the right one. And try the result in more than one player / TV.

If you want to store a high-quality version in as little space as possible, the best compression you can get without quality loss is probably with the HuffYUV AVI codec. If you don't mind a small loss, XviD, x264 and DV are probably your best options in terms of quality and "future proofing".

When you say using shorter GOPs "breaks up" the banding, do you mean banding tends to increase through a GOP (i.e., starts smooth, then gets more quantised)? If so, I suspect that's a rounding off error in TMPGEnc. I remember seeing that problem a long time ago, then it seemed to get fixed in a newer version. Maybe it still happens with some types of images. You can try to work around it like this:

Instead of CBR, select CQ (Constant quality). In the "setting", set the "quality" to 100, set the maximum bitrate as high as possible (9800 minus the audio), set the minimum to 2000 or higher, and set both "spoilage" values to -100. Try this with a long GOP (ex., I=1, P=14) and see if the problem disappears.

RMN
~~~

[Scott A Taylor]

Yeah I am using PAL and the original frames are 720 x 576. I realise PNG is supposed to be lossless but when converting from bmp to png via Photoshop CS3 there is a slight increase in colour banding as there was with Premier's uncompressed avi format. I am not sure what is happening because as you say there shouldn't be quality loss. Maybe I have colour space conflicts. Anyway I'll stick to bmp if file size doesn't effect encoding. By the way bmp is pretty good for storage as zipping reduces bmp file size to a third of the original. I do prefer to keep the individual frames as opposed to a video for the master.

As you mentioned in a previous post shorter GOP's can create jpeg like blockiness, which it did for me. It is this subtle blockiness I find that breaks up the banding a little ie the bands don't have such a sharp edge.

I am happy with this result, so much better than my attempts before receiving your advice (most improvement is with the motion)

Thank you Rui for all your time
_________________
Scott A Taylor
visual artist

www.scotttaylor.info

[RMN]

[Scott A Taylor]

Turns out Photoshop colour space was the issue with my png’s. Thanks.
When I said zip I meant it as a generic term sorry, I do use rar. I am aware of Targa being the animator’s choice but Window’s picture and fax viewer doesn’t show them. I like to be able to quickly see an image without opening Photoshop or the like. Bmp isn’t much bigger as you say. I will stick with bmp especially considering the little colour space issue I had, I have never had an issue with bmp creating or viewing.
I have tried your suggestion and have posted frames below from the original footage, Premier’s default mpeg2-dvd codec (MainConcept) and 2 different settings using TMPGEnc.



This is the bmp output by Premier of the final edit



Mainconcept - 9.8 CBR, GOP M-3 N-12



TMPGEnc – 9.8 CBR, MP@ML, Highest Motion, I-1 P-2, Detect Scene, CG Matrix (top left 9 changed to 8 in both), Basic YCbCr



TMPGEnc – 9.8 CQ (Quality 100, Max 9800, Min 6000, Spoilage both -100) , MP@ML, Motion Estimate, I-1 P-14, Detect Scene, Default Matrix (top left 9 changed to 8 in both), Basic YCbCr

I am happiest with the first TMPGE frame. Notice the slightly better contrast and saturation which seems to be because of the CG Matrix.

I think we have tried almost everything.
_________________
Scott A Taylor
visual artist

www.scotttaylor.info

[RMN]

I've used your first frame to make a few tests, and I found that there is no change in the banding even when creating a HP@HL 11-bit MPEG-2 file, which is odd. At 11 bits per DC coefficient there should be enough margin to preserve pretty much any 8-bit RGB gradient.

Other tests with "strange" matrices suggest that there is some limit being hit, beyond wich the extra precision is not used either for encoding or (I'm starting to suspect) for decoding. In other words, MPEG-2 decoders simply don't bother to go beyond a certain number of intermediate steps (ex., maybe they always use 8 bits, because using 10 would effectively double the decoding time, due to the way most CPUs work).

If I have a chance, I'll try a few less common MPEG-2 decoders to see if there's any difference.

You can "break up" the banding by adding a bit of luminance noise to the original, but then it looks a bit dirty (and the banding is still there, just less regular).

RMN
~~~

[Scott A Taylor]

I truly appreciate all the time you are spending on my problem. Thanks.
_________________
Scott A Taylor
visual artist

www.scotttaylor.info

[RMN]

Without much in the way of useful results, though. I've tried other decoders and the same thing happens, although the exact banding pattern varies slightly (which might indicate that the issue is indeed with decoders cutting some corners - but some cut different corners from others).

And the same thing happens with MPEG-4 ASP, by the way, even with quantisation set to 1 and no limit on the bitrate. I haven't tried MPEG-4 AVC yet, but if it behaves the same way, using BluRay / HD-DVD won't make any difference.

Lossless codecs (like HuffYUV) preserve all the shades, naturally.

RMN
~~~

[Scott A Taylor]

Looks like I will have to rethink my animations, in saying that I am happier with the results we ended up with as opposed to my original solo attempts.

Technical restrictions can push the creative process into new directions so I'll wait to see where this leads me.

Thanks once again Rui and congratulations on a fantastic website.

All the best.
_________________
Scott A Taylor
visual artist

www.scotttaylor.info

[RMN]

Do you make these for sale (meaning you need a format that's easy to distribute) or for exhibitions where you have more control over the format and playback equipment?

By the way, I had a similar problem (temporal, not spatial) ages ago with some animations that included very slow cross-fades (several minutes long). Regular video editing software couldn't make them smooth enough, you always noticed little "jumps" every few seconds. In the end I had to do the cross-fades in 3D software, that renders internally with floating-point (32 bits per channel) precision and then dithers down to 8-bit (24 per pixel). The result was that different pixels effectively faded at different times, and it looked smooth.

But that's a more extreme (and rarer) case than a smooth gradient so I'm a bit surprised that MPEG encoders (or, possibly, decoders) have this problem.

RMN
~~~