4th January 2009

I thought this would be a good time to summarise what I have deduced about the geometric distortions in film recordings, and how I plan to investigate them further.

First I want to correct an error in my previous work:
It seems I got my reference frames backwards when plotting the surface distortions.
I initially got it right on 28th July 2008; but then erroneously corrected myself, so everything from 30th July 2008 is backwards.

This means that my plots actually represent the distorted surface relative to an undistorted reference frame.
So we're back to the idea that brighter areas of the frame have the greatest distortion/dilation.

I believe this is caused by poor power supply regulation, such that an increase in power to the electron beam also leads to an increase in power to the scan-coils. So the beam scans more rapidly, and is deflected slightly more in the vertical direction. (Increased attraction from the increased current in the beam also contributes.)

After that correction lets get back to my deductions about the geometry:

Firstly the vertical displacements appear to be more stable than the horizontal.
Thus the fixed distortion seems to dominate over the transient distortion in the y direction.

I think this is because the downwards "momentum" of the electron beam is much smaller than it's horizontal "momentum", therefore the rate of change of flux pulling the beam downwards is small: so any luminance-driven distortions will be small.
They may even be small enough to be disregarded, except in extreme examples such as the JS Lights footage, where rapid changes in luma appear to have squeezed the scan-lines together.

The vertical displacement measurements always seem to have the appearance of a wavy pattern, sometimes forming a double-eye shape (see below).
TOTP36_(raw)_(V).jpg
(C) BBC

pt450pal_(V).jpg
(C) BBC

I suspect this distortion arises from a small temporal asymmetry in the flux density inside the CRT as the beam scans down the field. Therefore it will probably always be present in all film recordings.
The asymmetry is too small to affect the rapid horizontal scanning; but large enough to affect the slow vertical scanning.

Interestingly when you compare the vertical distortion between the TOTP and Paul Temple frames (above), the direction of the wavy pattern reverses like a mirror image. Thus it may be affected by the way the CRT's scan-linearity has been adjusted.

This wavy formation can result in a half-moustache distortion in the scan-lines.
This is a cubic function in x, with it's origin displaced to the left in the top half of frame, and to the right in the bottom half (or vice-versa).
Thus you half a left-hand moustache in the upper frame and a right-hand moustache in the lower frame.
I was completely wrong about this being a quartic function in x, and Richard was right (as usual :)). (That's why my correction of the y-distortion in the Paul Temple frame failed, because I was using the wrong function.)

Thus the vertical fixed distortions are anti-symmetric in nature.


I've yet to determine the form of the fixed horizontal distortion, however I believe the transient horizontal distortions are driven mostly by luminance.
The transient and fixed distortions are more evenly matched than in the y direction.

To check for a correlation between luma and transient distortion, we first need to remove the fixed distortion by deforming the reference to it.
Then we need to measure the distortions again, and look for a correlation.

The distortion measurements form a vector field (delta_x, delta_y) over the (x,y) manifold (where (x,y) is an undistorted Cartesian coordinate system).
This field generates a bijective automorphism from (x,y) to (u,v) where (u,v) is the distorted manifold.

Since delta_x and delta_y are likely to respond differently to luma, it's simpler to look at them independently, as two scalar fields.
Assuming the transients in delta_y are small, I plan to start my investigations with the scalar field delta_x(x,y).

I was incorrect in my previous assumption that the correlation should be between the scalar field of delta_x measurements, and the scalar field of luma values L(u,v).
Instead the correlation would be between the luma values, and the curvature of the distorted manifold.
So we need to use the delta_x values to determine this curvature, in order to produce a field of curvature scalars C(u,v).

Plotting C(u,v) should readily reveal whether there is indeed any correlation with the luma, as it should look like a low-pass filter of the frame's luma content.

Normally one would use the Ricci Scalar to measure curvature. This equals the trace of the Ricci Curvature Tensor with respect to the Metric Tensor, as follows:
3363f878078880860675119e79f1cb2f.jpg
Or in terms of Christoffel Symbols:
6e5aa8cc685dfdbe39c727637aeda134.jpg

However this is a complicated formula to calculate, so I need to find a simpler way of determining curvature.

At first I considered taking the first derivative of the delta_x field; but what form of the derivative should I use?
The total derivative is useless because x and y are independent.
The partial derivative with respect to x gives only the rate of change in the x direction; whereas I need the rate of change in all directions.
The partial with respect to both x and y gives the rate of change along a diagonal axis: so that's no good.
The grad derivative produces a vector field: so I can't match that up with my scalar field.
The Laplacian does give a scalar field; but that's a second derivative, so it's of no use here.

Anyway assuming I find a suitable scalar field C(u,v). If my model is correct we should then find a simple function connecting C(u,v) to the luminance field L(u,v). (Hopefully it would be C(u,v) = k*L(u,v), where k is a constant.)

Then we would have a transient distortion model based entirely on luma.
The background fixed distortion model would be based on Richard's software's measurements derived from the chroma.
So we have a hybrid model.

Of the other transient distortions introduced by the film (such as weave and bounce), these are likely to be mostly positional.
Since the luminance would follow the change in position, it shouldn't affect my model.
If you wanted to do a direct demodulation of the undistorted frames, then obviously it would be a problem; but apparently weave and bounce are fairly easy to remove.
Also the centre of the swirling pattern in the delta_y will correspond to the centre of the CRT screen: so that may provide a useful method for tracking these positional instabilities.

It's best to use continuous equations in the surface model, such as the cubic equations Richard has fitted to the data.
The equations can then be used to generate the convolution that corrects the distortion. This avoids the discontinuities that can be produced by using a mesh.

If there is any capacitance or damping effect in the CRT that slows down the response time of the EHT breathing, then obviously my model is not going to work. However the speed of the response in the JS Lights footage leads me to believe this is not the case.

As a first step I've written a short bit of code to turn Excel and csv files into black and white images.
Here is an example showing frame 36 from TOTP: the brighter areas correspond to the areas of greatest delta_x displacement.
TOTP36_delta_x.jpg
(C) BBC


I also wrote some code to produce an image of Richard's surface fitting equations for that frame, plotted as luma:

TOTP36_surface.jpg
(C) BBC



9th February 2009

My latest experiments have proved quite successful in reversing the geometric distortion.

I have now generated a set of reference files with no pre-distortion. These can be used to assess the results of any geometric correction applied.

Let's take a look back at some of the previous attempts at undistorts, along with representations of their remaining vertical and horizontal distortions.
First V.C. Mohan's filter designed to remove pin-cushion distortion:
TOTPDEPIN.jpg
TOTPDEPIN_V.jpg
Vertical displacement map (C) BBC

TOTPDEPIN_H.jpg
Horizontal displacement map (C) BBC


Now let's look at Richard Russell's attempt using his csv vector field, with the small-displacement method applied to generate an inverse vector field:
totp36_customdistort.jpg
(C) BBC

totp36_customdistort_V.jpg
Vertical displacement map (C) BBC

totp36_customdistort_H.jpg
Horizontal displacement map (C) BBC


This was made with respect to his pre-distorted reference, hence the errors in the right and left margins of the frame.
It was also done with respect to his original vertical reference, which had a certian amount of diagonal tilt.

When measured against these references the results are as follows:
totp36_customdistort_V_(ver97).jpg
Vertical displacement map (C) BBC

totp36_customdistort_H_(ver97).jpg
Horizontal displacement map (C) BBC



Now here is my latest attempt, using an iterative process to compare the FR with the VT reference image, and home in on the correct solution:
TOTP36_(with_copyright_logo_removed)_undistort_3_(very_fine).jpg
(C) BBC

TOTP36_(with_copyright_logo_removed)_undistort_3_V_(uref1).jpg
Vertical displacement map (C) BBC


TOTP36_(with_copyright_logo_removed)_undistort_3_H_(uref1).jpg
Horizontal displacement map (C) BBC


In my opinion I have achieved the best result yet on the vertical distortion.
The remaining horizontal distortion is for the most part neatly regularised into vertical bands. These can be tracked and decoded by Richard's process.

My method doesn't "know" about the cropping on the FR, hence the problem in the upper left corner.
However this could be corrected using Richard's Custom Reference utility, since this tracks the amount of cropping in the frame.
By comparing one of Richard's custom-distorted reference files to an undistorted reference, a very precise undistort could be generated, which should give an even better result on both the vertical and horizontal distortions.

I have applied a (primitive) spatial mask to my undistorted frame to separate the fields.
The results are shown below:
TOTP36_undistort_3,_field_1.jpg
Field 1 (C) BBC

TOTP36_undistort_3,_field_2.jpg
Field 2 (C) BBC


It can be seen that the strong chroma lines are aligned to my spatial mask, which indicates that I am correctly tracking the original scan-lines.

Running the two fields through Richard's process produces the following results:
field_1_(contrast_enhanced,_low_pass).jpg
Field 1 coloured (C) BBC

field_2_(contrast_enhanced,_low_pass).jpg
Field 2 coloured (C) BBC


There's very little movement in this frame; but if you flick between the images you can see that the microphone is moving upwards, along with Jimmy's left arm and sleeve. Also the nose of one of the cuddly toys is twitching sideways.

Here is the 2D Fourier Transform of one of the individual fields (lower plane only):
FT_field_1.jpg
(C) BBC

Since a single field can only support up to 144 c/aph you would expect the 216 c/aph signal to be substantially weakened, and indeed it is.
The luma and chroma repeat at intervals of 144 c/aph, reflecting the field sampling rate.

Here is the vertical displacement map for one of the fields:
field_2_V.jpg
(C) BBC

It appears to show perfect geometric correction, though this may be because Richard's software is not designed to work with deinterlaced fields? (And also because my spatial filter is aligned horizontally.)

The undistort vector field I have created is on an ultra-fine grid, and can be applied quickly to other frames using a macro acting on an avi file or image stack.
Since the vertical distortion is relatively stable, it may be possible to apply the same undistort across an entire scene.
(Obviously you would need to remove the vertical film jitter first; but this is easily accomplished with AviSynth, or other professional software.)

The spatial filtering I'm using is very primitive, and my results so far should not at all be taken as representative of the likely image quality that could be achieved.
The filters could be substantially improved, which should give a much better outcome.

Also digitising the source at 2k may help to improve field separation.
My undistort field can easily be scaled up to any resolution using a simple parametric adjustment.

I'm now awaiting some source material where the fields are substantially different. This should provide a more convincing acid test of my undistort method.
However I'm encouraged by the results so far, and by the fact that Richard's software seems quite capable of coping with deinterlaced fields.
This is not really surprising, since his software's horizontal distortion tracking (other than in the latest version) relies only on the 72 c/aph subcarrier signals, and therefore removing the 216 c/aph signals has no bearing on the result. It is only used for tracking the vertical distortion, which I have now (virtually) removed.

Later:

I have now found a much improved way of spatial filtering to extract the two fields.
The new versions are shown below:
Image7.jpg
Field 1 (C) BBC

Image12.jpg
Field 2 (C) BBC


Here are the colourised fields:
field_12.jpg
Field 1 colourised (C) BBC

field_22.jpg
Field 2 colourised (C) BBC


Here is the vertical displacement map for field 2:
Image12_V.jpg
(C) BBC