Experiments+in+Geometric+Correction


 * __29th May 2008__**

In these experiments I have attempted to reverse the distortions present in the film recording, using various different methods.

Firstly I tried using the mesh-warping software FantaMorph. There is no generalised pattern-recognition built-in, so the morphing pathways have to be generated by the manual insertion of control dots.

Below is an example of a test frame morphed back fairly closely to the original geometry:

Although this corrects the macro-structures; it also introduces new distortions to the micro-structures (chroma-dots and scan-lines). So it's not very useful for this problem.

An attempt at recolourising a similar morph shows up the new distortions as blotchy irregular coloration on the BBC1 sign:



Trying to lock onto the chroma dot patterning to generate the morphing pathways, would probably result in crushing out all the phase information. So you would simply end up with a delta function at Fsc.

It would be better to lock onto the local symmetry of the chroma-sidebands in the frequency domain, as this would preserve the modulating waveforms for U and V.

V.C. Mohan has kindly produced this image for me by examining the frequency distortion to the upper left region of the test frame:



He examined the source for barrel-type and pin-cushion-type distortions, and discovered that a pin-cushion reversing deformation would restore a uniform subcarrier frequency. He also discovered that different distortion parameters were needed for the x and y axes.

Below you can see 1D frequency spectra of the original frame and Mohan's corrected frame, summed over all HD lines:

In the second image we see that Mohan's frame has almost restored the original symmetry to the chroma sidebands, at least in a global analysis of the frame. He's got rid of the gross distortion to the subcarrier wavelength that was deforming the symmetry.

The shape of the cropping in Mohan's corrected frame, together with the overall form of the deformation implies that the film recorder's CRT was probably the main source of distortion.

If this is the case then we ideally need to model the EM field structure within the TV tube, to produce a model of the distortion.

The form of the EM field is determined by the shape of the CRT's field coils. These follow the surface contours of the tube, and thus have different curvatures in the vertical and horizontal axes.

The rigorous solution would be to solve Maxwell's equations in Minkowski space, with the shape of the field coils as the boundary conditions, factoring in their fluctuating driving voltages. This gives the form of the EM field tensor, which can be used to derive the classical Lagrangian for the electron beam. Extremising the Lagrangian then gives the equation of motion of the beam. (It's not necessary to use Quantum Electro-Dynamics because the quantum effects for this system are minimal at the scale we're interested in.) There may well be less rigorous methods available to model scan-line linearity and beam deflection velocity.

From memory, the field-lines cluster around the surface of the coils. This means you have a greater field-strength near the coils, so the beam will be deflected more rapidly. This would explain the marked increase in subcarrier wavelength in the left and right margins of frame. It would also produce more curvature to the scan-lines in the upper and lower regions of frame. Since the beam is scanning more rapidly in the left and right margins, it has less time to be deflected vertically: so you would expect the curvature to flatten out towards the edges of the picture.



Richard's false colour image above shows this flattening of the scan-lines, as well as the increased curvature in the upper and lower areas. In overall form it resembles the structure of an EM field: so I believe we are looking at a temporal-slice through that field. The two eye-shaped regions are probably caused by the effect of the driving voltages on the field.

There is an axial symmetry about the central horizontal bisector; but around the vertical bisector the symmetry is skewed and distorted. This is probably due in part to the film camera not being quite square-on to the CRT, which introduces a trapezoidal perspective distortion. There may be another distorting factor though. Perhaps the film flexing in the gate after pulldown? (If this is the case you would expect the distortions to vary from frame to frame.)

I have asked Mohan to look into a more comprehensive geometric correction, with the view to achieving field separation in a generalised Film Recorder.


 * __A note on separating the fields__**

As far as I recall, sub-Nyquist sampling throws away information. So I would've thought it was not possible to perfectly reconstruct the original field-lines from the 1080-line scan of the FR? If this is the case then geometric reversal would not be sufficient to separate the fields, and a higher resolution scan would be needed.

__**4th June 2008**__

Re: My previous article: http://colour-recovery.wikispaces.com/A+Brief+Frequency+Analysis

Having just read the theory behind the [|Transform Decoder] I now realise that the frequencies F and 3F I referred to in the above link were in fact just mixtures of the U and V subcarriers, at 72 and 216 c/aph. Hence their cross-hatched appearance.

The other frequency I isolated at 288 c/aph (which I referred to as 4F) is almost certainly a remnant of the original scan-line structure, after the higher frequencies have been effectively filtered off by the HD sampling.

As there are horizontal, vertical and temporal symmetries to the original chroma signal, all of these could be used to track geometric distortions and reverse them.

They could also be used for error checking Richard's process. For example if you have a phase quadrant transition, then there would have to be a fairly sharp edge transition to the modulating signal for U and/or V. Fourier theory tells us that sharp transitions are composed of many more frequencies that smooth ones: so you should see an abrupt broadening of the chroma sidebands in horizontal, vertical and temporal domains. If there is no broadening, then the quadrant transition must be an error.

Since the centre of the picture has least distortion, the quadrant detection is likely to be most accurate here: so it would be best to work outwards from the centre when looking for these transitions.


 * __8th June 2008__**

Today I tried an experiment to measure the distortions in Richard's JS Lights images showing the effect of mixing fields with differing EHT breathing.

I reasoned that the sidebands should have different asymmetries, because you are merging two fields with differing distortions, and I wanted to see if these asymmetries could be easily observed.

Below are 1D spectra of Richard's two cropped sample frames (summed over all HD lines):



The chroma sidebands are ofcourse represented by the lowest peak, with the higher frequency peaks being harmonic repeats. The first image can be seen to have a more prominent lower sideband; whilst the second image has a stronger upper sideband. The subcarrier frequency appears virtually unchanged, so there doesn't appear to have been much horizontal stretching of the image overall.

I then looked at the vertical sidebands:



The 72 c/aph and 216 c/aph peaks can be clearly seen. The resolution is too low to make out whether the sideband distortions are different; but at some point I will repeat this test with a more closely cropped image to get round this problem. Again the frequencies are virtually constant: so not much vertical stretching is evident.

As Richard has stated on his pages, the relative strengths of the peaks at 72 c/aph and 216 c/aph are affected by the amount of inter-field movement. This effect can clearly be seen in the above images, since the lower (72 c/aph) peak is much weaker in the second image. So clearly this effect is also generated by a variation in the inter-field merging.

I wondered whether the different diagonal slant to the patterning was caused by this difference in balance between the two vertical peaks, since if it were, you could compensate for it by applying similar vertical frequency attenuations to your demodulating reference waveform.

So I isolated both frequencies in each image:







The upper left quadrant in the above images shows the isolated frequencies. To my surprise I found that the patterning corresponding to the upper peak looks very similar in both images, with roughly even diagonal slanting in both directions, producing a cross-hatched pattern.

The lower peak on the other hand has left-slanting diagonals in Frame 383; but right-slanting in Frame 384. Since both peaks are formed from a mixture of U and V, the inter-field distortion is obviously affecting the balance between the two subcarriers in the lower peak.(Frame 383 has dominant V-carrier; whereas Frame 384 has dominant U.)

It occurs to me that by measuring the relative strength between the upper and lower peaks, you should be able to use this figure to set the weighting between U and V in order to apply a compensating phase rotation. So it should be possible by this method, to correct the hue errors on Jimmy's face in the JS Lights sequence, and more generally to compensate for EHT breathing faster than 25Hz.


 * __18th June 2008__**

In an attempt to explain the swing in relative U/V gain in the JS Lights footage, I have now evolved a new theory.

In the above 2D Spatial Fourier Transform, it can be seen that each of the 4 components of the subcarrier signals (together with their reflections) lie along different diagonal lines through the origin.

If you evolved this plot in time, the angles of these lines would change depending on the raster distortions present in the source (such as spatial breathing and film weave).

The HD sampling frequency and it's reflection would form a fixed diagonal line through the origin.

If you instead require the subcarrier components to remain fixed in place as you evolve in time, you would find the sampling frequency diagonal is no longer fixed, but evolves according to the source material distortions.

The alias frequencies produced by the sampling, lie along diagonals midway between the sampling frequency diagonal and the subcarrier diagonals.

So depending on the distortions, the diagonals corresponding to the alias frequencies will move around the frequency plot. Sometimes they may align closely to one of the subcarrier components and interfere with it, depressing it's amplitude; but leaving the other 3 subcarrier components untouched.

The relative phasing between the sampling frequency and the original scan-line structure, will determine whether the sampling frequency diagonal slopes upwards or downwards. So this phasing will determine whether it aligns more with the U or V signal.

In the JS Lights footage the sampling rate to scan-line structure alignments, ensure that most of the interference is focussed on the lower vertical peak (originally at 72 c/aph).

However it could align more with the upper vertical frequency if the horizontal sampling rate were reduced (e.g. in the "Paul Temple" footage).

What I've slightly glossed over here (for the sake of simplicity) is the presence of other vertical frequencies in the source material (before it's sampled). For example the peak at approximately 562 c/aph (the original scan-line rate cropped down from 576 by the film recorder), and it's harmonic repeats. In terms of aliasing it's almost certainly these frequencies that cause most of the interference (since to produce interference with the subcarrier, the aliasing diagonal must have the same length as well as the same angle). In JS Lights they align closely to the 72 c/aph diagonal, thus interfering with the lower vertical peak.

I've also glossed over the fact that the distortions produce a spread of frequencies in the source material: so instead of diagonals you will have triangular segments.

This makes it even more likely that you will get interference with the aliasing frequencies in some regions of the frame. (In the JS Lights footage I think we only see the effect in the central portion of the frame, around Jimmy's face.)

Frequency "smear" may also complicate matters with respect to choosing a suitable window function. I know very little about window functions, but I think I'm right in saying that choosing one which is too narrow in frequency-space could make the signal strength look artificially depressed.


 * __19th June 2008__**

Have plugged some numbers into my theory on aliasing, and I can't make it work. (I worked out all the angles using inverse tan functions.) So I think the following explanation of the JS Lights effect is much more probable:

The second field is being deflected upwards by around 2 scan-lines relative to the first, such that the frame interleaving is reversed in the region of Jimmy's face.

This would make left-leaning diagonals look like right-leaning ones (and vice versa): hence the apparent swing in U/V gain.

The patterning at 216 c/aph would not be affected in the same way, since it has triple the density.

It's a much simpler theory, but it seems to fit.


 * __21st June 2008__**

Richard Russell has pointed out that my theory is flawed because he can see no such effect in the luminance.

At first I thought the answer to this is that one of the fields could be masking the luminance signal of the other. This could happen if it was brighter overall, and the two fields were sitting almost on top of each other, with the interleaving only just being reversed.

The chroma signal overall is weaker in one frame than the other, which suggests the scan-lines could be bunching in pairs like this, and partially cancelling out the chroma.

Having thought about it further I realised that simply bunching all the scan-lines closer together (without actually reversing the interleaving order) can also make left-leaning diagonals look like right-leaning ones.

Looking closely at the periodicities, you can see that frame 383 exhibits exactly this type of bunching. The periodicity of the strong chroma lines is much weaker, and has changed from 7.5 HD lines to around 3.75. Frame 384 on the other hand has the more usual periodicity of c.7.5 HD lines. (N.B. I'm not claiming there's been a 7.5:3.75 contraction; just that bunching the scan-lines together brings out different periodicities when they are sampled to 1080 HD lines. The fact that the periodicity has almost disappeared suggests the scan-line density has changed.)

If the lines are bunching together in an irregular way, such that they are paired, you would expect the 288 c/aph frequency to split in two. It's hard to be sure, but if you look closely at the frequency spectrum for frame 383 you can see this splitting appears to have occurred: