15th December 2008

I've been thinking about the problem of separating chroma information where the two fields are different, and have come up with a possible solution.

Given that each of the two fields has vertical chroma frequencies at 72 c/aph and 216 c/aph, you will have two sets of vertical signals being intermingled in the film frame.
Thus inter-field modulation frequencies will be formed at 144 c/aph, 288 c/aph and 432 c/aph, and these will strengthen when the content of the two fields starts to diverge.

If these signals can be filtered cleanly enough, they should contain enough information to derive the two original chroma fields.
E.g. The relative signal strength of the 432 c/aph signal should tell you the phase difference between the original fields at 216 c/aph. Then the signal at 216 c/aph could be filtered off, and two new signals generated with the correct phase difference.

Once you've derived the four original vertical chroma signals, it should be possible to use Richard's software to filter and decode colour information separately for each field.


16th December 2008

One thing I was forgetting about yesterday, is that the signal strength at 432 c/aph will depend not only on the relative phase between the two fields, but also the difference in amplitude (i.e. saturation). So we would need to somehow separate out the phase and amplitude information.

However it should be possible to deduce the saturation difference from the relative signal strengths of the various vertical frequency components of the chroma.
The signal at 288 c/aph will be especially useful, as it results from a mixture of the 72 c/aph waveform of one field with the 216 c/aph waveform of the other: so it will not have the same phase cancellation component as the 432 c/aph signal.

Unfortunately it will be necessary to calculate the phase and saturation differences separately in each of Richard's blocks, since they will vary over the frame depending on the colours present. The results would be used to populate a pair of mapping functions that would recreate the original chroma signals at 72 and 216 c/aph for each field.

I've no idea if this is actually practical given the amount of processsing involved?
There may be a simpler way to achieve the same result, perhaps by some clever method of mixing the various chroma signals to derive the original fields?


18th December

Here is an image showing the signal at 432 c/aph. It's extremely weak, only just poking up above the noise level. Hopefully it will grow when the two fields are very different.
432cycles.jpg
432 c/aph signal (C) BBC

432cycles(crop).jpg
Enlarged spectrum (C) BBC


Here's a couple more examples, this time from JS Lights, where we do have very different fields.
Interestingly the signal strength of the 432 c/aph peak does seem to change from one frame to the next; though it's still pretty feeble!

383_vert.jpg
(C) BBC

383_vert_432(crop).jpg
(C) BBC

384_vert_432.jpg
(C) BBC

384_vert_432(crop).jpg
(C) BBC




30th January 2009

I've noticed for some time now that the JS Lights sequence exhibits frequency-splitting in it's 2D Fourier Tranforms, such that each of the chroma signals is partially split-in-two.
I realised that this is probably caused by differences in the breathing parameters between the fields, and wondered whether it was possible to filter these split frequencies separately in order to separate out the two chroma fields.
Given that the breathing parameters are probably always going to be slightly different between the fields when you have different content, it's just possible that this method might work more widely.

I had a look at the 2D Fourier Transforms of some cropped sections of film frames, and noticed that the splitting is more exaggerated in the intermodulation beat frequencies formed between the chroma signals at (+2Fsc, 288 c/aph) and (-2Fsc, 288 c/aph).
This is not surprising, as when you mix the chroma signals together, you will double the frequency splitting in the resulting additive beat frequencies.

When you have two different fields, you have four different subcarrier signals at approximately 72 c/aph (two at circa -Fsc and two at circa +Fsc). So by mixing the U subcarrier signals at 72 c/aph with themselves (and likewise for the V), it might be possible to achieve enough separation in the split frequencies to filter and decode them separately. Alternatively you could try narrowing and re-tuning the bandwidth of the chroma filters at 72 c/aph to try and exclude one of the split frequency pair.
In the JS Lights footage the differences are so extreme that you might get away with doing this globally over the whole frame (at least for a crude test). Though to implement this properly you would need to do localised filtering, with dynamic filters capable of locking onto and tracking the split frequencies.

A major problem with this method is that you could end up filtering off a mixture of both fields, since one field will sometimes have a higher frequency than the other; and sometimes lower.

Below is an example of frequency splitting taken from a cropped version of frame 83 in the JS Lights sequence:
Image2.jpg
Fourier Transform of JS Lights frame 83 (C) BBC

You are looking at the lower half of the Fourier Plane, and it can be seen that the V carrier at (+Fsc, -72 c/aph) appears to have split vertically into three separate spots.

Ofcourse this effect will only be observed when the geometry is breathing faster than 50Hz: so you won't see it in every frame.


3rd February 2009

I have now isolated the two split chroma frequencies, which can be seen in the images below, superimposed on the luminance:
s383.jpgts383.jpg

It can be seen that the chroma patterning is now only visible in the areas of field-combing around Jimmy's eyes, mouth and chin, and the base of his nose (i.e. the areas where there is inter-field movement).
It can also be seen that the pitch of the slope angle is different in each image.

This would seem to confirm that the split frequencies are caused by differences in breathing parameters between the fields, and it is possible that I have achieved a separation of the chroma from each field.

My filtering removed the U components entirely, as well as the central V spots at 72 and 216 c/aph, preserving only the upper V spots in the lefthand image, and the lower V spots in the righthand image.

The central V spots obviously correspond to those regions where both fields are identical, and therefore this indicates that my earlier hypothesis about the breathing being highly localised is probably correct.


8th February 2009

Richard has explained that the frequencies I filtered in the above experiment, were in fact the vertical sidebands of the chroma signal; and did not amount to a separation of the fields after all.

The reason that three spots appeared on the FT is that the sidebands tend to cluster around the harmonic frequencies forming the vertical waveform. So where you have an abrupt vertical edge transition, such as around Jimmy's mouth, the sidebands will broaden out, and the harmonic frequencies will appear.

It would seem this line of research is a dead end, and I am never going to be able to separate the fields this way.
However I have made some more promising progress on undistorting the geometry, which I will write up shortly here: Geometric Correction (contd).