nocash wrote:

And, for colored images, the YUV-to-RGB part does still lack some details about fractions/rounding, particulary in the G=(-0.3437*B)+(-0.7143*R) and R=(1.402*R) and B=(1.772*B) calculations.

The MDEC seems to be using fixed point math with a 12-bit fractional part, just as the GTE and GPU does in places.

Using these coefficients:

RCr ( 5744)
GCr (-2928)
GCb (-1408)
BCb ( 7264)

or in other words:

Red   = Y + 1.4023 * Cr
Green = Y - 0.3437 * Cb - 0.7148 * Cr
Blue  = Y + 1.7734 * Cb

I've been able to verify all 16.8M combinations of Y/Cb/Cr and the resulting R/G/B values.

However, the MDEC seems to be using too few bits, causing "interesting" overflow/rounding issues for some edge cases, which needs to be handled.