Extrapolating+A+More+Complete+Field+Separation

__**7th February 2012**__

Today I performed an experiment based on an idea I had ages ago (probably back in 2009); but never got round to testing. To my surprise it worked perfectly first time!

The idea was to take my partially separated fields from TOTP, and extrapolate a more complete separation of the luma component only. This is done by calculating a 2D scalar field containing the difference in pixel values between the two partially separated TV fields, amplifying this difference, and adding it back in to the TV field data. I wrote the following piece of simple QBASIC code to perform this task: The code assumes 573 active lines are preserved in the film frame (though this figure will vary depending on what type of film recorder was used). It also assumes planar headerless Y8 image data with a width of 720 horizontal pixels, since this processing is best done at standard definition.

Here are the results using an amplification factor of 3:

As you can see there's a lot more movement visible in the second animated GIF: so the experiment has definitely achieved the desired result. There's also a vast increase in noise; but this is most likely because I've not properly isolated the active field line data. Instead I'm working on the whole image, including all the blank lines. My next task is to modify the code to work only on the active line data.

In principal this method ought to solve the problem of out-of-phase film inserts; but only if the film recording is suitable for partial field separation. I've only tried my partial field separation technique on two film recordings: the TOTP was successful, but the other source failed to separate (by coincidence it was an out-of-phase film insert!) Since the TOTP film recording is unusually pristine, I suspect the threshold for partial field separation is quiet a bit higher than that required for colour recovery. So unfortunately it may well be that only a small number of film recordings would be suitable candidates (most likely those with the widest vertical bandwidth, and perhaps lower than usual spot-wobble settings).

Also the geometric distortion must remain stable enough for the same undistort vector field to be applied across each camera-shot; otherwise you get disturbing changes in geometry. This does appear to be the case in the TOTP footage; but it's unlikely to be true in general, especially on panning or zooming shots. If you have to change the vector field with each frame in order to achieve partial field-separation, then some way would have to be found to constrain the vector fields so that picture disturbances are minimised. (In a previous experiment I managed to reduce these disturbances by constraining the curl and div derivatives of the vector field; but although this produced an improvement, it still wasn't acceptable for viewing.)

So it's quite possible my result today was nothing more than an interesting experiment, with no practical applications!

__**8th February 2012**__

Have modified my code to work only on the active lines. It produces this result:



Still very noisy, so I tried filtering out the chroma dot crawl patterning before processing. That produced the result below:



I then tried reducing the amplification from 3 to 2:



This reduces the noise; but at the cost of a slightly reduced level of field separation.

Here is a sequence of two frames back-to-back with separated fields (amplification 2):

I'm working on a longer sequence, which I'll publish shortly.

__**9th February**__

As promised here is a longer sequence of frames, first the original film, then the field separated version:

There's obviously a problem in the middle of the sequence, where my field filters are drifting out of alignment with the original scan-lines. Some way would have to be found of tuning in the filters to the undistorted frames. It may have to be manual tuning; though automated would obviously be better.

There also seems to be an error with the field sort-order at the very end of the sequence. This may not in fact be an error, since Jimmy is just about to move the microphone in the opposite direction at this point; but I've corrected it anyway below:

Finally here are the original, and field separated versions, at 25Hz and 50Hz respectively (at least they're supposed to be; though the animated GIF software seems to have got the timings a bit wrong somehow!):

__**11th February 2012**__

Here are de-noised versions of the field-separated sequence:

This is the latest version of my code:

These are my field filters (used for the partial field-separation):



__**19th February 2012**__

I've started an AviSynth development thread here: []

With help I wrote the following AviSynth script: It requires the following plugins: FQuiver, MaskToolsv2, RawSource, and fft3dfilter. Also requires Y8 versions of my field filters: It takes an undistorted HD Y8 video stream, and attempts to separate the fields. The output stream is monochrome 576i.

Have tested it on a 361 frame section of TOTP chunk 3, and it continues to produce separation over the whole clip. Please see development thread for a list of ongoing problems.

Have also written this test script: This treats the fields as if they were frames and outputs 576p.

Processing times range from 3-10 fps on my system. (The two denoising filters at the ends of the scripts can be removed to achieve maximum speed.)

Many thanks to pbristow, Gavino and everyone else at Doom 9 for their invaluable help!

Example of output in test mode, with manually re-sorted field-order (frame-rate slowed to 1 fps with Virtual Dub):

Another example (no re-sorting necessary on this one): Note how Jimmy's expression changes with each field.

__**A note regarding 405-line material:**__

I wonder whether anyone has looked for energy at around 0 Hz, 377 c/aph? Unlike 625-line material, this lies well below the Nyquist Limit of an HD 1080-line scan.

If there is energy at 377 c/aph, it could in theory be used to track the FR distortion, apply a vertical undistort, and perhaps separate the fields. The main purpose would be to repair out-of-phase film inserts.