21st June 2008

I have attempted to utilise the vertical frequency at 288 c/aph to separate out the original fields.

Since this frequency is locked to the original scan-line structure, but at half the rate, it should be possible to make use of it to separate out the original chroma fields.

If you could extract the 288 c/aph signal cleanly across the frame, then you could enhance it's contrast and simply multiply it with the filtered chroma signal to extract one of the fields. Then use the inverse waveform to extract the other field.

Since the chroma is not affected by aliasing, it should in principle be possible to reconstruct the original chroma fields exactly.
Though this would not be the case with the luma, you could still do an approximate reconstruction with some type of interpolation method.

Recolourising the two fields separately would obviously allow significant advantages for panning shots etc...

In practice you would no doubt need to do some more sophisticated signal processing on the 288 c/aph waveform rather than simply enhancing it's contrast. It should be possible though to generate a clean vertical signal in the form of a deformed square-wave (or perhaps a deformed sinusoidal waveform would be better?)

Here is my first (very crude) attempt at extracting a field:

TOTP36_field_sep.jpg
(c) BBC

I have simply multiplied the unprocessed frame (luma and chroma) with the extracted 288 c/aph waveform several times over.

I think you can see the single field 3.75 HD line periodicity starting to come out in the vertical chroma signal (at least in some regions of the frame).


22nd June 2008

Origin of the 288 c/aph signal


The signal at 288 c/aph is not as simple as it may first appear.

It could be an inter-modulation product of the two vertical chroma signals caused by non-linearities in the film recorder.
However if this were the case, you would expect it to be weak, and to always have a corresponding subtractive inter-modulation frequency at 144 c/aph.
Instead we see a strong signal at 288 c/aph, usually with no corresponding signal at 144 c/aph (although occasionally a very weak signal appears).

Thus I have concluded that the 288 c/aph frequency has a different origin, as follows:

Since the spot wobble mechanism has the effect of squaring-off the vertical waveform, it introduces a strong Fourier component at 288 c/aph, as this is the basic building block frequency on which the square edges are constructed.
I believe it's this frequency that we're seeing in the source spectrum.

If this is the case then the phase will always be locked to the line-structure, regardless of differing chroma content between fields.
Self-evidently, the only phase change possible would be a 180 degree shift.
This could be a major snag, since I don't want to end up with a mish-mash of the two fields when I try separating them.

I'm not sure what the probability of such a phase change is intra-frame. If it's low then it could be ignored; but otherwise some compensating mechanism would need to be applied.

This is not the end of the story when it comes to the 288 c/aph frequency, since Andrew Steer discovered that it is also present in areas of the frame with no chroma.
This signal obviously has a different origin, and I would tentatively suggest it could be due to a slight difference in exposure between the two fields. This would introduce a Fourier component at 288 c/aph, whose presence would be accentuated by the HD sampling, since this filters off most of the higher Fourier components or shifts them onto different aliasing frequencies.

The presence of this luminance-derived signal is very useful, since it could potentially be used to do an approximate luma separation of the two fields. (N.B. The sub-Nyquist sampling has thrown away some of the information, so it would not be possible to do an exact separation.)


24th June 2008

Today I have generated a waveform plot of a typical column section from the TOTP footage, 30 HD pixels in height:
single_column_waveform.jpg
(C) BBC


This column has uniform colour, and is taken from the BBC1 sign behind Jimmy Saville.

The edges have become smoothed by the HD sampling; but you can see fairly clearly that the original signal would've had a series of plateau-levels with abrupt transitions inbetween. In other words a step-wave.

There is simply no way to generate such a waveform without a strong component at 288 c/aph: so I am more convinced than ever that the 288 c/aph signal we see in the source material is mostly generated from the raster, and not an intermodulation of the two chroma signals.

It's amplitude is ofcourse related to the saturation of the chroma signal, and hence the relative saturations between fields may influence it's phase. It still must remain firmly anchored to the scan-lines however: so the only rephasing possible is a 180 degree shift (at least where there is chroma).

I'm not yet sure whether this 288 c/aph signal will maintain a horizontal frequency of zero throughout the frame, and this will need further study. Visual inspection so far reveals that it appears to have this property.

It can also be seen from the waveform plot that there should be a fairly strong Fourier component at 144 c/aph, and evidence of this can be seen below:
Intermodulation.jpg

This is a 144 c/aph vertical band-pass filter of the composite signal (rotated through 90 degrees for ease of filtering).
The filtered signal appears in the upper left quadrant, and it seems to have a horizontal frequency of zero.

If it were an intermodulation product between the chroma signals, it would have a horizontal frequency of 2*Fsc; so instead it must be a remnant of the scan-line structure.
(It is "contaminated" by a slight chequerboard pattern, which could have originated from a very small chroma intermodulation component.)

N.B. It seems I was wrong about the absence of a 144 c/aph signal, it's just too small to show up above the noise bed most of the time.


30th June 2008

I have now analysed a new image from V.C. Mohan (shown below), in which he has attempted to correct the geometric distortion using slightly different parameters than before.
TOTPDEPIN2.jpg

Here are the horizontal and vertical 1D frequency spectra summed over all lines/columns:
TOTPDEPIN2_horiz_freq_2.jpg
Horizontal frequency spectrum (C) BBC

TOTPDEPIN2_(crop)_vert_freq_3.jpg
Vertical frequency spectrum (C) BBC

The horizontal correction appears very good, with the symmetry of the side-bands having almost been restored.
The vertical correction however is still way off, with severely distorted side-bands at both 72 and 216 c/aph (outlined in red).

My attempt at phase-sensitive recolourisation produced the following result:
img129_with_TOTPDEPIN2_(crop_1)_(col)_SR14161000,_Fsc_4303619_(crop).jpg

This shows a wide horizontal band of good colour across Jimmy's face, thus confirming that the correction is working well horizontally.
The curvature of the distortions tend to flatten out towards the left edge of the frame, in the same way that we have seen previously.
Thus the vertical scan-line deflection is still not properly corrected.

If a full geometric correction can be achieved, it should be possible to deinterlace the two fields at least in regards to the chroma signal.


16th July 2008

Having played around a bit with Richard's software, I have come to the conclusion that a more sophisticated pre-distortion of the reference may improve it's performance, especially if it could be made adaptive to the global frequency content in the source.

It may be more helpful to think of pre-distorting the source material rather than the reference (though it amounts to the same thing):
By setting each frame's distortion parameters to keep the horizontal and vertical chroma spikes at approximately the same position in the frequency spectrum, this should produce a rough correction for EHT breathing.
It should also produce more uniform slope angles in the chroma signal, and this would hopefully result in less cross-colour.

To test this theory I have used the two distorted frames produced by V.C. Mohan, which take the test image closer to the original geometry, and produce a more uniform subcarrier frequency over the frame.

To do a proper test I needed to move the left-side of frame away from Richard's area of pre-distorted reference: so I used the Helix YUV format instead of raw headerless YUV. The file header produced a horizontal picture phase shift, which accomplished what I wanted.

Here are the colour-restored images from the original test frame, and Mohan's two attempts at correction:
TOTP36_(col).jpg
TOTPDEPIN_(col).jpg
TOTPDEPIN2_(col).jpg

It can be seen that both Mohan's distorted frames correct the errors on the BBC1 sign very well.
His 2nd distorted frame also seems to correct the errors in the right-hand margin, even though they lie within the area of pre-distorted reference.

To understand what was going on here I had a look at the quadrant maps for each image:
TOTP36_(quad).jpg
Original Image (C) BBC

TOTPDEPIN_(quad).jpg
Mohan's 1st Attempt (C) BBC


TOTPDEPIN2_(quad).jpg
Mohan's 2nd Attempt (C) BBC


It can be seen that Mohan's frames provide a major improvement on the BBC1 sign, with a nice neat outline around the edge. This is especially true on his 2nd frame, which also has a neat outline around Jimmy Saville.

To investigate further I had a look at the horizontal displacements in all three images:
TOTP36_(H).jpg
Original Image (C) BBC

TOTPDEPIN_(H).jpg
Mohan's 1st Attempt (C) BBC

TOTPDEPIN2_(H).jpg
Mohan's 2nd Attempt (C) BBC

Here we see that Richard's software gives up entirely on the left-hand side of the BBC1 sign in the original frame; but in Mohan's 1st frame the geometry in that area has been almost completely corrected (apart from the upper left corner).

In his 2nd frame Mohan has introduced new distortions to the BBC1 sign, but also slightly increased the area (of the sign) that the software is able to track. However the lower right-hand corner of frame has worsened, which explains why the errors disappeared in this region, since the software is simply assigning the quadrant to (-U, +V).

It's difficult to draw any useful conclusions from the right-hand-side of frame anyway, since this lies within the region of pre-distorted reference.

I don't know whether Mohan's distortion will prove to be better than Richard's; but it certainly brings the frame closer to it's original geometry: so in theory it should be easier for Richard's process to track the finer errors in geometry, once the course errors have been removed by the pre-distortion process.
It would be interesting to test this further, and to that end it would be useful to have a menu option to switch off the pre-distortion of the reference. Different types of distortions could then be tested, including those based on more complex non-radial distortions.

Further software improvements could involve a mechanism to compensate for the line-bunching effect I found in the JS Lights footage, which made left-leaning diagonals look like right-leaning ones. By measuring the local density of the vertical 288 c/aph signal, you could apply a suitable chroma phase correction.

Also if I'm right about the source-derived 288 c/aph signal being firmly anchored to the line structure, then a comparison of the chroma-derived 288 c/aph waveform against the source-derived 288 c/aph waveform should allow the phase errors to be detected, and a suitable correction applied. This could solve the problem of frames where the two fields have very different chroma content, especially since this difference should increase the amplitude of the 288 c/aph signal in the source.


18th July 2008

After further study of the geometry in the source material, I've concluded that there are two principal distortions layered on top of each other, both arising from non-linear scanning on the CRT:

Distortion A arises from the effect of the CRT's scan-coils in shaping the EM field.

Distortion B is generated by the image content itself, since the strength of the cathode ray will determine how much it is deflected by the field.

Distortion A is axially symmetric (both vertically and horizontally), and takes the form of a so-called "moustache distortion". This is a combination of barrel and pin-cushion distortions, and is found in some optical lensing systems.

The centre of the frame has pin-cushion distortion, but the outer regions are closer to barrel distortion, due to the flattening of the scan-lines that I have commented on previously. This combination is what's known as moustache distortion.

The fact that it's also found in optics should not be that surprising, since an EM field is essentially a cloud of virtual photons. So the shape of the scan-coil acts like an electronic lens, in the same way that a glass lens focusses a beam of light. The main difference is that light is composed of real photons not virtual ones; but this doesn't affect the outcome. (The cloud of virtual photons are in this case coupling to a fermionic Dirac field in the form of an electron beam; but the effect is essentially the same.)

Distortion B will only have axial symmetry when the image content is symmetric: so most of the time it will skew the axial symmetry.
Modelling this distortion is harder, but far from impossible, since the deflection will be directly proportional to the luminance on the CRT (though you may have to allow for gamma distortions in the commital to film).

(If the CRT had a mechanism to modify the scan-coil driving voltages to take account of the current in the cathode ray, then this would complicate things further. Though I suspect there was no such mechanism in monitors of the era.)

Mohan has been using a so-called fish-eye deformation to re-map the geometry, i.e. a barrel-type distortion used to reverse pin-cushion deformation produced by optical lenses.
This works well in the centre of the frame; but not so well in the four corners.
To illustrate this, here is the Y-displacement map from one of Mohan's frames after being run through Richard's software (in raw headerless form this time, so no picture phase shift is present):
TOTPDEPIN_(raw)_(V).jpg
(C) BBC


Compare this with the Y-displacement map from the original test frame, and the difference is very clear:
TOTP36_(raw)_(V).jpg
(C) BBC


I've yet to find some user-friendly software for reversing moustache distortion.
Have tried Panorama Tools with the Hugin front-end GUI; but this is designed for photographic experts so I can't get my head round it!

If this could be done it should correct distortion A completely, leaving distortion B to be dealt with by Richard's process.