The distortion of geometry in a film recording causes the harmonic frequencies in the active line signal to spread out in the frequency domain, in what could be called "frequency blurring".
This is due to the wavelengths being either stretched or compressed towards the edges of the frame, where the distortions are most evident: so you have a shift in frequency compared to the central portion of the frame.

An iterative series of deformations could be performed on a test frame, to home in on the point where this "frequency blurring" reaches a minimum.
At this point you should have recovered the correct line-structure and geometry.

A suitable reference to measure the amount of "blurring" might be the embedded chrominance, since you could probably detect the amount of overlap between the upper and lower sidebands. The degree of overlap would correspond to the amount of "blurring".
To do this you would ideally need a frame with as much colour as possible covering all areas of the picture, so that the sidebands show up more clearly. Failing this, you could simultaneously deform several test frames from different scenes with differing colour content (i.e. superimposing them all together).

Alternatively you could simply try to detect the "sharpness" in all of the luminance harmonics, by comparing each of them to a delta function.

The best approach would probably be to deform the horizontal axis first, followed by the vertical axis.

Correcting the vertical deformations relies on being able to scan along enough of the curved active lines to be able to detect the frequency spreading within the line fragments.
It might be simpler to take the frequency spectrum of each column, and compare this to a normal 625-line raster. If you take a test frame with as few dark areas as possible, the regular spacing of the scan-lines should produce a characteristic fundamental frequency when you scan down each column of the image, which could be compared to the "blurred" frequency produced by the distorted film-recorded image. Thus you can use the comparison to home in on the correct raster. (This relies on the line structure being clearly visible on the HD scan of the film recording.)
In this case it would be better to do the vertical correction first, then the horizontal.

You could either apply the iterations on a column-by-column, followed by a line-by-line basis; or over the whole frame simultaneously. Applying a variable amount of stretch along each line/column.

(If you apply a low pass filter to the image, then scan down each column, you could perhaps deform the image in the spatial domain, rather than the frequency domain. The filter should reveal an irregular sine wave down each column indicating the scan-line spacing. If you deform the column around it's central point to regularise this sine wave, then apply the same deformation to the unfiltered image, you should remove the scan-line curvature. N.B. It may be simpler to rotate the image through 90 degrees and work on the rows instead of the columns, then rotate it back after processing.)

Another possibility would be to use something similar to a heterodyne frequency shifter, using an advance feedback loop to try to continually correct the frequency error in each scan-line, before demodulating the chroma (or at least the equivalent of this in software).
Obviously you don't have a colour burst: so you would have to use the symmetry of the sidebands within the active line itself as a reference. Perhaps by detecting frequency duplets of identical amplitude around the approximate position of the subcarrier frequency, and using their symmetry as a guide reference.
This may be like looking for a needle in a haystack; but you do at least have two sets of duplets (one set for U and one for V).
Obviously they would be easier to find within areas of low luminance detail, where there are no high frequency luminance harmonics overlapping with the chroma signal.
You would have to look at the frequency spectrum within a small segment of each scan-line, and then advance the segment by one or two pixels at a time to try and track the frequency variations along each line.

Where the scan-lines have been curved vertically, this method would struggle, although it should still work on partial scan-lines.
It would be better to correct the curvature first before applying the method.

The variable line cropping down the image could also cause errors; but this could perhaps be compensated for by the PAL-delay averaging.

Alex Weidmann