# Bringing an 8mm Home Movie Back to Life: A Restoration Story

*How a shoebox reel of flickering, scratched, washed-out 8mm film becomes a clean,
smooth, color-graded video, told step by step, for people who don't do computer
vision for a living.*

This is the story of the **`yossi_final`** recipe: the exact pipeline we tuned and
ran on *Yossi's Bar Mitzvah* reel (`YossiBmFull.mp4`, ~16.5 minutes, 17,908 frames).
Every stage below is a real piece of software doing a real, specific job. I'll tell
you **what each one fixes**, **the idea behind it**, **the actual technique in code**,
and **what it's really doing to the picture**, plus the exact dial settings we chose
for this film.

---

## What's wrong with old 8mm film

Home-movie film from the 1960s-80s has a very consistent set of ailments, and it
helps to name them before we start curing them:

- **It wobbles.** The film never sat perfectly still in the projector's gate, so the
  whole frame jitters a few pixels every frame. On top of that: handheld camera shake.
- **It breathes.** The exposure pulses frame-to-frame: the image gets subtly
  brighter and darker several times a second. This is *flicker*.
- **It's dirty.** Specks of dust and dirt flash on for a single frame. Long vertical
  **scratches** run down the film where it dragged against something.
- **It's grainy.** Film grain is real silver crystals; blown up to HD it looks like
  crawling noise.
- **It's soft.** 8mm is a tiny format. Detail is genuinely limited and the scan is fuzzy.
- **It's faded and color-cast.** Dyes decay unevenly with age, so the whole image
  drifts toward a muddy orange/green, blacks turn milky gray, and everything looks flat.
- **It's choppy.** Filmed at ~18 frames per second, motion looks slightly stuttery to
  a modern eye used to 30-60fps.

The restoration is essentially a **sequence of specialised stages**, each one curing
exactly one of these ailments and handing a slightly cleaner image to the next.
The *order* matters enormously: fixing them in the wrong sequence makes some cures
undo others. More on that as we go.

---

## The big idea #1: work shot by shot, not frame by frame

The single most important architectural decision: we don't process the movie as one
long strip. We first cut it into **shots**, continuous runs of frames between scene
changes (a cut from the ceremony to the crowd, say).

For *Yossi's* reel, the detector found **261 shots**.

Why bother?

1. **Honest color and light.** Many of our corrections need to *measure* the footage
   first ("how gray is the average color here? how milky are the blacks?") and then
   apply one consistent fix. If you measured across a scene cut, you'd average a bright
   outdoor shot with a dark indoor one and get a fix that's wrong for both. Measuring
   *per shot* gives each scene its own tailored correction, and, crucially, keeps that
   correction **constant within the shot** so nothing flickers.
2. **The frame interpolator can't cross cuts.** When we later invent smooth in-between
   frames, we do it *inside* a shot. If it tried to blend the last frame of one scene
   into the first frame of the next, it would morph a face into a wall.

We detect shots with **PySceneDetect's `AdaptiveDetector`**, which flags a cut when the
frame-to-frame difference spikes relative to its rolling neighborhood (a moving average
of change), robust against the flicker and grain that would fool a fixed threshold.

```
raw reel  ─▶  [ shot 0 ][ shot 1 ][ shot 2 ] … [ shot 260 ]
                  │         │         │              │
                  ▼         ▼         ▼              ▼
             each shot goes through the FULL relay independently
```

---

## The big idea #2: lossless intermediates and streaming frames

Between every stage we save the video as **FFV1** (a mathematically lossless codec)
inside an MKV file. Nothing is ever thrown away or re-compressed mid-pipeline, so
stage 8 sees exactly what stage 7 produced, with no accumulating JPEG-style mush.

But an HD film is huge, and we can't hold 18,000 uncompressed frames in memory. So the
Python code **streams**: it asks FFmpeg to decode one frame, hands that single frame
(as a NumPy array) to the algorithm, writes the result, and moves on. Memory stays flat
whether the shot is 10 frames or 10,000.

The mechanism is a lesson learned the hard way. In [`ffio.py`](../orchestrator/ffio.py),
frames flow through an OS **pipe**:

```python
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=log)   # decode
while True:
    buf = proc.stdout.read(frame_bytes)          # one frame's worth of bytes
    if not buf or len(buf) < frame_bytes:
        break
    yield np.frombuffer(buf, np.uint8).reshape(h, w, 3)   # -> a NumPy image
```

> **The deadlock trap:** if you give FFmpeg a pipe for *both* its output and its error
> messages and only drain one of them, the other fills its buffer and the whole thing
> freezes forever. The rule the code enforces: **pipe only one stream, send the other
> to a logfile.** Reading frames? Pipe stdout, log stderr. Writing frames? Pipe stdin,
> log stderr. Never both.

A recurring pattern you'll see below is **two-pass streaming**: stream once to *measure*
something (a few bytes per frame, a brightness value, a histogram), then stream again
to *apply* the fix. This keeps memory tiny while still letting a stage "see the whole
shot" before deciding what to do.

Now, the pipeline itself. Here's the running order for `yossi_final`:

```
stabilize → photometric → scratch → denoise → light → sharpen → color → interpolate → grain
```

---

## Chapter 1, Stabilize: hold the frame still

**Ailment:** gate weave + handheld shake.
**Tool:** FFmpeg's `vid.stab`, two passes.
**File:** [`stabilize.py`](../orchestrator/stages/stabilize.py)

Stabilization runs in two passes:

- **Pass 1, `vidstabdetect`:** watch the whole shot and record how the image moves
  from each frame to the next. It writes these measurements to a little `transforms.trf`
  file. Nothing is changed yet; it's pure analysis.
- **Pass 2, `vidstabtransform`:** read that motion track, *smooth* it, and shift/rotate
  each frame to cancel the jitter.

The subtle part is the **smoothing window** (`smoothing=5`, meaning ±5 frames). We keep
it deliberately **small**. A big window tries to lock the entire scene rigidly in place,
which fights against the camera's *intended* pans and tilts and forces a big zoom-in to
hide the black edges that swing into view. A small window does the opposite: it *follows*
the real camera movement and only cancels the high-frequency wobble riding on top of it.

```
camera's true slow pan:   ╲╲╲╲╲╲╲   (we want to KEEP this)
gate weave on top of it:  ∿∿∿∿∿∿∿   (we want to REMOVE only this)
small smoothing window → subtract the ∿, keep the ╲
```

Because the corrections are tiny, we set `optzoom=0`, **no corrective zoom**, so the
original framing is preserved exactly.

> **Yossi's dials:** `smoothing=5`, `optzoom=0`. Local stabilization, true framing.

---

## Chapter 2, Photometric: stop the breathing (deflicker)

**Ailment:** exposure pulsing frame-to-frame.
**Technique:** temporal luminance normalization in NumPy.
**File:** [`photometric.py`](../orchestrator/stages/photometric.py)

This is the first stage written as pure Python image math, and it's a clean example of
the two-pass idea.

- **Pass 1, measure.** Stream every frame and record just its average brightness
  (luminance). That's one number per frame:

  ```python
  lum = [float(frame.mean(axis=(0, 1)) @ _BGR_LUMA) for frame in read_frames(src)]
  ```

  `_BGR_LUMA` is the standard perceptual weighting `(0.114, 0.587, 0.299)`, human eyes
  weigh green far more than blue, so brightness isn't a naive channel average.

- **Compute the target.** Take a **centered rolling mean** of that brightness curve over
  a 31-frame window. This smoothed curve represents where the exposure *should* be, the
  slow, intended changes (a lamp switched on, a pan into sunlight) survive; the fast
  frame-to-frame flicker gets averaged away.

- **Pass 2, apply.** For each frame, compute a gain = `target / actual` brightness and
  multiply every pixel by it, nudging that frame back onto the smooth curve. The gain is
  **clamped between 0.5× and 2.0×** so a single freak dark or blown frame can't explode.
  The same gain hits all three channels, so **color is untouched**, we only adjust
  brightness.

```
frame brightness   ▁▇▁▇▁▇▁▇   (flickering)
smoothed target    ▄▄▄▄▄▄▄▄   (what it should be)
after correction   ▄▄▄▄▄▄▄▄   (pulled onto the line, flicker gone)
```

Why a 31-frame window? Wide enough (~1.7 seconds) to treat the pulsing as noise, narrow
enough to still follow a genuine lighting change.

> **Yossi's dials:** `window=31`.

---

## Chapter 3, Scratch: remove dust and vertical lines

**Ailment:** single-frame dust specks + long vertical scratches.
**Tools:** AviSynth+ (`RemoveDirt`, `DeScratch`), driven from Python.
**Files:** [`run_avs.py`](../runners/avisynth/run_avs.py) via the runner subprocess.

Here we step outside Python's own image loop and borrow the **AviSynth+** ecosystem,
decades of specialized film-restoration plugins that are the gold standard for this job.
The trick we use: generate a tiny AviSynth *script* on the fly and let FFmpeg (built with
AviSynth support) execute it. No heavyweight frame server, no manual install into system
folders, the plugins live right next to the code and are loaded by path.

Two operations, in order:

1. **`Clense` (temporal median).** For each pixel, look at it in the previous, current,
   and next frame and take the **median**. A dust speck exists in only *one* frame, so
   the median throws it out automatically. But this alone would smear anything that's
   genuinely moving.
2. **`RestoreMotionBlocks`.** So we then detect the blocks that are actually *moving* and
   copy the **original** (un-Clensed) pixels back into them. Result: static dust is gone,
   but moving subjects stay crisp.
3. **`DeScratch`.** Finally, hunt for the long vertical scratches specifically. We keep it
   conservative so it targets only thin, long, near-vertical lines and leaves real vertical
   structures (a doorframe, a pole) alone: `maxwidth=5` (thin), `minlen=100` (long),
   `maxangle=3` (near-vertical), `mindif=6` (clear scratches only).

```
dust:      • flashes on for ONE frame  →  temporal median deletes it
motion:    a waving hand                →  RestoreMotionBlocks protects it
scratch:   │ long thin vertical line    →  DeScratch paints it out
```

> **Order rule:** dust removal must run **before** denoise. `RemoveDirt` finds dirt by
> looking at how much a pixel *differs* between frames, and denoising erases exactly
> those differences. Denoise first and the dust becomes invisible to the dust remover.

> **Yossi's dials:** conservative defaults (no overrides).

---

## Chapter 4, Denoise: remove the grain

**Ailment:** crawling film grain.
**Tool:** AviSynth+ `MDegrain3` (motion-compensated temporal denoise).
**File:** [`run_avs.py`](../runners/avisynth/run_avs.py)

This is the heaviest lifter in the chain and the reason 8mm can look so clean afterward.

Naive denoising blurs everything and destroys detail. `MDegrain3` is smart about it. It
works on the insight that **grain is random from frame to frame, but the real image is
not**. A person's cheek looks almost identical in three consecutive frames; the grain
speckling it is different each time. So if you can *average* the same patch of cheek
across several frames, the grain cancels out and the true cheek remains.

The catch: the cheek *moves* between frames. So before averaging, `MDegrain3` does
**motion estimation**: it builds a multi-resolution pyramid of each frame (`MSuper`) and
searches for where each little block went in the neighboring frames, both backward and
forward, up to ±3 frames away (`MAnalyse`). Then it averages each block along its true
motion path.

```
grain (random each frame):   ▓░▒  ░▒▓  ▒▓░   →  average along motion → ░░░  (cancels)
real detail (consistent):    ▉▉▉  ▉▉▉  ▉▉▉   →  average along motion → ▉▉▉  (survives)
```

The `thSAD` parameter is the aggressiveness knob: how different two blocks can be and
still be treated as "the same thing, just noisier." We run **`aggressive` (thSAD=1000)**
because 8mm grain is coarse. Importantly, we denoise **chroma as hard as luma**
(`thSADC=thSAD`), because a later stage boosts color saturation, and any leftover color
grain would get amplified into ugly speckle if we didn't kill it here.

> **Yossi's dials:** `strength="aggressive"` (thSAD 1000).

---

## Chapter 5, Light: lift the veil (tone / exposure)

**Ailment:** milky, low-contrast blacks; flat exposure.
**Technique:** measured per-shot black-point lift + gentle contrast, in FFmpeg.
**File:** [`light.py`](../orchestrator/stages/light.py)

Old film blacks never reach true black: there is a hazy gray "veil" over the shadows
that makes everything look washed out. This stage lifts that veil, and it does so
**intelligently per shot** rather than with a blanket setting.

The important part is how it measures the veil. It streams the shot and builds a brightness
**histogram**, then reads off the value at the 0.5th percentile (`black_pct=0.5`), i.e.
"how dark are the darkest 0.5% of pixels really?" That value *is* this shot's black floor.
A hazy shot reads high (lots of veil to remove); a genuinely dark shot reads near zero
(nothing to lift, so we don't crush it).

There are two guardrails, both discovered while tuning this exact film:

- **Measure only the center.** Old scans have pure-black borders around the picture. If
  you include them, the "black floor" reads as 0 and nothing ever lifts. So we **crop
  the outer 15%** before measuring, ignoring the scan frame.
- **Don't over-lift.** We apply only a **fraction** of the measured floor
  (`black_strength=0.55`) and **cap** it (`black_max=0.15`). During tuning, the full
  auto-measured lift (~0.11) looked too aggressive; 55% of it (~0.06) was the sweet spot
  that opened the shadows without going harsh.

Then a whisper of contrast (`contrast=0.08`) for a bit of punch. The black lift uses
`colorlevels` (applied identically to all channels, so **color balance is preserved**,
this stage only touches tone), and contrast uses `eq`.

```
before:  gray veil over shadows      ▁▂▃▄▅▆▇   histogram floats off zero
after:   true blacks restored        ▁▂▃▄▅▆▇   histogram anchored at black
         (measured per shot, gently, center-only)
```

> **Why a separate stage from color?** Light and color are deliberately split so we can
> tune exposure without disturbing hue, and vice-versa. Light runs *after* denoise (so
> lifting the shadows doesn't amplify grain) and *before* color (so white balance lands
> on already-corrected tone).

> **Yossi's dials:** `auto=true`, `black_pct=0.5`, `black_strength=0.55`, `black_max=0.15`,
> `contrast=0.08`. (Net effect ≈ a 0.06 black lift, the gentle setting we picked over the
> stronger auto value.)

---

## Chapter 6, Sharpen: restore bite (carefully)

**Ailment:** softness.
**Tools:** FFmpeg CAS + unsharp mask.
**File:** [`sharpen.py`](../orchestrator/stages/sharpen.py)

Now that the image is clean, we can safely sharpen. Sharpening *before* denoise would
just amplify grain, which is why it waits until here. Two complementary sharpeners stack:

1. **CAS (Contrast-Adaptive Sharpening)**, an AMD FidelityFX algorithm that crispens
   detailed areas while leaving flat areas (skin, sky) alone, so it adds almost no halos
   or grain. This is the gentle, safe workhorse. `strength=0.95`.
2. **Unsharp mask**, the classic technique: blur a copy, subtract it from the original
   to isolate the edges, then add those edges back amplified. This gives extra "bite"
   beyond CAS. We run it **luma-only** (`chroma_amount=0`) so it sharpens detail without
   introducing colored fringes on edges. `luma_amount=1.3`.

**The ceiling we found.** During tuning we compared `0.85/1.1` (too soft), `0.95/1.3`
(crisp and clean), and `1.0/1.5` (**halos**, dark/blue rings drawn around every
high-contrast edge, like a window frame behind someone's head). Classical sharpening
*always* has this ceiling: past a point, the unsharp mask stops adding detail and starts
tracing outlines. `0.95/1.3` is the practical maximum for this footage: visibly sharper,
no ringing.

```
under-sharp   0.85/1.1   → soft, muddy
JUST RIGHT    0.95/1.3   → crisp hairline & eyes, no rings   ✓ chosen
over-sharp    1.0/1.5    → blue/black HALOS around edges     ✗
```

(There's a separate AI super-resolution stage available, Real-ESRGAN, that goes further
without halos, but at ~13 seconds *per frame* it would take ~60 hours for this reel, so we
deliberately left it out.)

> **Yossi's dials:** `strength=0.95`, `unsharp=1.3`.

---

## Chapter 7, Color: undo fading, warm it up (the grade)

**Ailment:** color casts, faded/washed color, flat grade.
**Technique:** per-channel histogram stretch + gray-world white balance + film LUT + vibrance.
**File:** [`color.py`](../orchestrator/stages/color.py)

This is where a muddy orange memory becomes a film with real color. Four layers, all
**measured once over the whole shot** and applied identically to every frame (per-frame
color estimation would flicker, a hard rule).

1. **Cast removal (per-channel percentile stretch).** Build a separate histogram for Red,
   Green, and Blue. For each, find the value at the 0.5th and 99.5th percentile
   (`clip_percent=0.5`) and stretch that range to fill 0-255. Because each channel is
   stretched independently, a uniform color cast (say, everything too orange) gets pulled
   apart and neutralized.
2. **Gray-world white balance.** The assumption: averaged over a whole scene, the world is
   roughly neutral gray. So we scale the channels until their averages match, removing
   whatever global tint remains.
3. **Blue-shadow attenuation** (`blue_shadow=0.15`). Old film grows a blue cast
   specifically in the *dark* areas. So we pull blue down, but only in the shadows: the
   effect ramps from full at black to zero by mid-gray.
4. **Film LUT + vibrance (the "cinematic" grade).** Blend in a warm film **look-up table**
   (`film_warm.cube` at `lut_blend=0.4`) for a filmic tone, then a modest saturation boost
   (`saturation=1.2`).

**The key optimisation here** is how layers 1-3 are applied. Instead of doing float math
on millions of pixels, we fold the entire correction into three **256-entry lookup
tables** (one per channel), computed once from the histogram. Then the heavy per-pixel
pass is just an array index:

```python
for c in range(3):
    out[:, :, c] = luts[c][frame[:, :, c]]   # every pixel: one table lookup, no math
```

This is a classic optimisation: precompute the answer for all 256 possible input
values, then the actual work is instant lookups. Memory stays flat, speed stays high.

```
faded orange, milky   ▓▓▓▓  →  stretch each channel  →  neutral, full-range
                                + gray-world balance
                                + warm film LUT (40%)  →  cinematic, alive
                                + vibrance ×1.2
```

> **Yossi's dials:** `clip_percent=0.5`, `white_balance=true`, `strength=0.7`,
> `blue_shadow=0.15`, `lut="film_warm"`, `lut_blend=0.4`, `saturation=1.2`.
> (Note: no contrast boost here, the *light* stage owns tone, so the two never fight.)

---

## Chapter 8, Interpolate: smooth the motion (RIFE)

**Ailment:** choppy ~18fps motion.
**Tool:** Practical-RIFE, a neural frame interpolator, on the GPU.
**File:** [`run_rife.py`](../runners/rife/run_rife.py)

Film shot at 18fps looks stuttery on modern screens. Rather than just showing each frame
twice, we **invent** brand-new in-between frames so motion genuinely flows.

RIFE (Real-time Intermediate Flow Estimation) is a neural network that, given two
consecutive frames, predicts the **optical flow**, where every pixel *moved* between
them, and synthesizes the frame that would sit exactly halfway along that motion. For
each original pair we emit: original frame, synthesized middle, next original, doubling
18fps to **36fps**.

```
originals:   A ──────── B ──────── C
RIFE adds:   A   (A½)   B   (B½)   C      (½ = invented middle frame)
result:      smooth 36fps motion
```

This is why we split into shots earlier: RIFE only ever interpolates *within* a shot, so
it never tries to morph across a scene cut. It streams frames through the GPU two at a
time, so VRAM stays modest even on HD.

> **Yossi's dials:** defaults (2× → 36fps).

---

## Chapter 9, Grain: put the film texture back

**Ailment:** the "plastic," over-clean look denoising can leave.
**Tool:** FFmpeg `noise` filter (temporal).
**File:** [`grain.py`](../orchestrator/stages/grain.py)

Counterintuitively, after working so hard to *remove* grain, the last step **adds
a little back**. Fully denoised footage can look plastic and digital, which is wrong for old film.
A fine, ever-changing grain restores an organic, filmic texture and hides subtle
compression artifacts.

The key is that the grain is **temporal** (`c0f=t+u`): a fresh, independent grain pattern
every frame, exactly like real film, not a static overlay frozen in place (which would
look like dirt on your screen). It's applied mostly to luma with only a third as much on
chroma (`c1s`/`c2s = strength//3`), so it reads as neutral film grain rather than colored
speckle.

Grain runs **dead last**, *after* interpolation, so it's crisp on every final frame and
doesn't get smeared by the interpolator.

> **Yossi's dials:** `strength=7` (subtle).

---

## The finish line: encode to shareable video

After all 261 shots finish the relay, they're stitched together and encoded. Because a
lossless reassembly of a 16-minute HD reel would need *hundreds of gigabytes*, the
`--clean` mode we ran stitches the per-shot finals **directly** into the compressed
outputs (via FFmpeg's concat demuxer), no giant intermediate master ever touches disk.

Two deliverables come out ([`encode.py`](../orchestrator/stages/encode.py)):

- **Viewing tier**, **AV1** (SVT-AV1), 10-bit, with a touch of film-grain synthesis
  baked into the codec. Smallest file, best quality, modern.
- **Sharing tier**, **H.264** (x264), 10-bit, `+faststart`. Universally playable
  anywhere, ideal for uploading.

Both are **10-bit** (`yuv420p10le`) even though the source is humble 8mm, 10-bit gives
1024 brightness levels instead of 256, which prevents *banding* (visible stair-steps) in
the smooth gradients that denoising and grading create.

```
output_videos/YossiBmFull/
  YossiBmFull_yossi_final_<timestamp>.viewing.mp4   ← AV1, small & pristine
  YossiBmFull_yossi_final_<timestamp>.sharing.mp4   ← H.264, upload-anywhere
  YossiBmFull_yossi_final_<timestamp>.recipe.json   ← exact recipe used (provenance)
```

---

## The engineering ideas worth stealing

Beyond the image science, a few software patterns made this robust and are worth calling
out for any pipeline of this shape:

- **Recipe = flow + dials in one JSON.** The *order* of stages and every parameter live
  together in a single recipe file. The code owns a fixed menu of stage *implementations*;
  a recipe just composes them by name. You can't invent a stage that doesn't exist, but
  you can reorder, retune, and omit freely, powerful and safe. `yossi_final.json` *is*
  the movie's restoration, fully reproducible.
- **Two-pass streaming.** Measure in one cheap pass (bytes per frame), apply in a second.
  Memory stays flat no matter the length.
- **Precompute into lookup tables.** The color stage does zero per-pixel math in its hot
  loop, just 256-entry table lookups. Fast and exact.
- **Isolated subprocess runners.** The GPU stages (AI upscaler, RIFE) and the AviSynth
  stages each run in their **own Python environment** with their own CUDA/torch, launched
  as subprocesses. They can't conflict with each other's dependencies, and parameters are
  passed in through an environment variable, no fragile command-line contracts.
- **Graceful fallback everywhere.** If any heavy stage errors out on a bad shot, it
  **losslessly copies its input to its output and reports success**. One ugly shot degrades
  to "unfiltered" instead of killing a multi-hour run.
- **Never trust order-of-operations to luck.** Dust before denoise. Denoise before
  sharpen. Sharpen before grain. Grain dead last. Light before color. Each ordering has a
  concrete reason, and the recipe validator warns if you break one.

---

## The complete recipe

Here is the whole `yossi_final` restoration, exactly as run, the flow *and* the dials, in
one file:

```json
{
  "name": "yossi_final",
  "pipeline": [
    { "stage": "stabilize",   "params": { "smoothing": 5, "optzoom": 0 } },
    { "stage": "photometric", "params": { "window": 31 } },
    { "stage": "scratch",     "params": {} },
    { "stage": "denoise",     "params": { "strength": "aggressive" } },
    { "stage": "light",       "params": { "auto": true, "black_pct": 0.5, "black_strength": 0.55, "black_max": 0.15, "contrast": 0.08 } },
    { "stage": "sharpen",     "params": { "strength": 0.95, "unsharp": 1.3 } },
    { "stage": "color",       "params": { "clip_percent": 0.5, "white_balance": true, "strength": 0.7, "blue_shadow": 0.15, "lut": "film_warm", "lut_blend": 0.4, "saturation": 1.2 } },
    { "stage": "interpolate", "params": {} },
    { "stage": "grain",       "params": { "strength": 7 } }
  ],
  "encode": { "av1_preset": 6, "av1_crf": 22, "av1_film_grain": 8, "x264_crf": 18 }
}
```

Nine specialists, in a carefully chosen order, each curing one ailment of a 40-year-old
home movie, turning a flickering, scratched, faded shoebox reel back into something you'd
actually want to sit down and watch.

*Run it yourself:*

```bash
python -m orchestrator.pipeline YossiBmFull.mp4 --recipe yossi_final --clean
```

---
---

## Part Two, The sequel: how `yossi_final` became `yossi_v2`

`yossi_final` was a good restoration. But "good" invites a second opinion, so we
handed the whole pipeline to an outside reviewer who does this for a living. The
notes came back sharp and specific, and a few of them stung in the productive way.
This chapter is what we changed, and, just as important, what we *didn't*.

We treated every note as a hypothesis, not an order: check it against the actual
code, build the fix behind an **opt-in switch** (so `yossi_final` kept rendering
byte-for-byte identically as the control), then prove it with pictures before
believing it. Three changes earned their place.

## Upgrade 1, Deflicker that thinks locally

The reviewer's sharpest catch: our deflicker (Chapter 2) applies **one** brightness
gain to the whole frame. But 8mm flicker is often *local*: a projector lamp with a
hot spot, or one edge of the film fading differently than the other, so the left
side of the frame breathes while the right sits still. A single global gain can't
see that.

The fix keeps the exact two-pass structure but swaps the single number for a **4×4
grid** of them: measure each tile's brightness, smooth each tile's gain over time
independently, then **bilinearly interpolate** the grid back up into a smooth
full-frame gain map. One corner can now be calmed without touching the opposite one.

```
global gain:   one number for the whole frame  → misses local pumping
tiled gain:    ┌─┬─┬─┬─┐  each cell its own gain,
               ├─┼─┼─┼─┤  smoothed in time, then
               ├─┼─┼─┼─┤  blended into a smooth map
               └─┴─┴─┴─┘
```

There is a risk here: if the tiles drift too far apart,
you trade a flickering image for a **breathing vignette**, a low-frequency
brightness gradient sloshing around the frame, subtler but more annoying. So each
tile's gain is **clamped to within ±15%** of the frame's global gain
(`tile_spread=0.15`). Calm the local flicker; never invent a spotlight.

Measured on a test shot: **local flicker down ~24%** versus the global method, with
no visible vignette. Opt-in dial: `tiles: 4`, off by default.

## Upgrade 2, White balance that isn't fooled by a red shirt

Gray-world white balance (Chapter 7, layer 2) rests on one assumption: *averaged over
a scene, the world is neutral gray.* Usually fine. It fails, badly, when a shot is
**dominated by one strong color**, a grass field, a blue-tableclothed hall, or, in
Yossi's reel, **a boy in a bright red shirt filling the frame.**

Here's what gray-world did to that shot. It saw all that red, decided the average was
"too warm," and "corrected" it by shoving the entire image toward the opposite
color, **green**, until the average went neutral. The shirt got tamed... and the
boy's **skin turned a sickly teal.** A clear over-correction.

```
red shirt fills frame → gray-world: "too much red!" → shoves everything green
                                                     → SKIN goes teal   ✗
```

The fix is a smarter measurement: estimate the balance from the **near-neutral pixels
only**, the genuinely low-saturation parts of the image (skin highlights, teeth, a
white collar, the tablecloth), which is where a real color cast actually shows, and
leave the saturated content alone. On that shot, near-neutral balance kept the skin
warm and natural and let the shirt stay red, exactly as it should. The numbers backed
the eyes: gray-world had crushed the shot's red-over-blue balance from **+43 to +15**
(neutralizing a genuinely red scene); near-neutral held it at **+43**, faithful to
the film.

Opt-in dial: `neutral_wb: true`. It falls back to plain gray-world automatically if a
shot has too few neutral pixels to trust.

## Upgrade 3, Stop adding grain twice

A quieter bug, but real. `yossi_final` **baked** film grain in Chapter 9 *and* told
the AV1 encoder to **synthesize** grain of its own. Those two fight: SVT-AV1's grain
synthesis works by *denoising* the input, modeling the grain it finds, and re-adding
it at playback, so it was busy trying to strip out and re-guess the grain we'd
carefully baked in. For a measured **zero** file-size benefit.

So grain became an **encode-tier decision**, not a doubled effort: bake the grain
once (we control it), and turn the encoder's synthesis **off** (`av1_film_grain: 0`).
One grain, cleanly.

A knock-on tweak followed: with near-neutral balance now *preserving* each shot's
real warmth, the warm film LUT, which had partly been compensating for gray-world's
habit of cooling things down, was suddenly pushing too far. So we dialed it back from
`0.4` to `0.22`. Fixing one stage let us relax another; the grade got more honest.

## The discipline underneath it all: golden frames

None of these calls could be made by gut, because they're *subtle and shot-dependent*.
So we built a small harness, [`golden_frames.py`](../tools/golden_frames.py), that
exports a fixed set of telling moments (a face close-up, the red shirt, a flickery
interior) as **labeled side-by-side stills**, across every recipe *and* the untouched
original. Change one dial, regenerate the sheet, look. It's the difference between "I
think this is better" and "look at panel three." Every decision above was made on a
golden sheet, one variable at a time.

> **Validate with pictures, not vibes.** It's the whole reason we caught the green
> skin, and, later, the reason we caught something looking *worse*.

## `yossi_v2`, the new keeper

Fold the three upgrades into `yossi_final` and you get **`yossi_v2`**, the honest new
best. We froze it, tagged it `v3-bkm2`, and rendered the full 16-minute reel overnight
(~9 hours, all 261 shots):

```json
{
  "name": "yossi_v2",
  "pipeline": [
    { "stage": "stabilize",   "params": { "smoothing": 5, "optzoom": 0 } },
    { "stage": "photometric", "params": { "window": 31, "tiles": 4, "tile_spread": 0.15 } },
    { "stage": "scratch",     "params": {} },
    { "stage": "denoise",     "params": { "strength": "aggressive" } },
    { "stage": "light",       "params": { "auto": true, "black_pct": 0.5, "black_strength": 0.55, "black_max": 0.15, "contrast": 0.08 } },
    { "stage": "sharpen",     "params": { "strength": 0.95, "unsharp": 1.3 } },
    { "stage": "color",       "params": { "clip_percent": 0.5, "white_balance": true, "neutral_wb": true, "neutral_sat": 0.15, "strength": 0.7, "blue_shadow": 0.15, "lut": "film_warm", "lut_blend": 0.22, "saturation": 1.2 } },
    { "stage": "interpolate", "params": {} },
    { "stage": "grain",       "params": { "strength": 7 } }
  ],
  "encode": { "av1_preset": 6, "av1_crf": 22, "av1_film_grain": 0, "x264_crf": 18 }
}
```

The same nine stages, two of them now noticeably better.

---

## Part Three, The road not taken: AI super-resolution

The reviewer's most tempting note was the one we chased hardest and, in the end,
walked away from: *replace the classical sharpener with real AI super-resolution.*
8mm is genuinely soft; a good neural model can hallucinate plausible detail that was
never on the film. This is the account of why we let it
go, because a negative result, clearly established, is worth as much as a win.

## Three candidates, three ways to lose

**KEEP (a face-restoration model).** On a *frozen frame* it was striking: it
took a mushy, barely-a-face blur and rebuilt sharp eyes, teeth, and skin, recognizably
the same boy. Then we watched it *move*. Frame to frame the reconstructed face
**shimmered and subtly morphed**, because the model rebuilds each frame independently
and never quite agrees with itself. Sharp as a still, unsettling in motion, not a
restoration. Rejected.

**Real-ESRGAN x4plus (general super-res).** Genuinely good, crisper hair, cleaner
edges, natural, and *stable* in motion (it enhances the whole frame, not just faces).
The problem was time: **~13 seconds per frame**, about **78 hours** for this reel. We
even chased a faster runtime, a prebuilt Vulkan backend, and proved the wall was the
*model*, not the framework: the same heavy model ran at the same 13 seconds
regardless. Crisp, but immovable.

**Compact models (animevideov3, general-x4v3).** The reviewer's actual suggestion,
small, fast networks, and they *were* fast: **0.5-1.5 seconds per frame**, 10-25×
quicker. But on a real face they **over-smoothed**: they cleaned the image without
adding x4plus's crisp detail, landing softer than the look we wanted. A hard trade
revealed itself: **fast or crisp, not both.** The crispness we liked was welded to
the heavy, slow model.

## The compromise, and why it *still* lost

If the good look is slow, spend it only where it counts. So we built a **hybrid**: a
"big-face gate" added to the super-res stage that runs the slow but detailed x4plus
**only on close-ups**, where a face earns the cost, and lets every wide or scenery
shot pass through fast. One automatic render, no manual splicing. We called it
`yossi_v3`.

It ran (~20 hours, and it even survived a full-disk crash halfway, see below),
correctly applying x4plus to **59 close-up shots** and skipping the other **202**. On
paper, a triumph. Then we watched it.

## The verdict, and the lesson

On clean single frames, x4plus had looked like an obvious win. In the *finished reel*,
with film grain re-added over it and AV1 compression on top, that fine detail was
largely **masked**, and in motion, on grainy 8mm, the whole thing read as
**processed**. Artificial. The eye that had loved the still frame flinched at the
moving one. Watched properly, `yossi_v3` was *worse* than `yossi_v2`, not better.

So we pulled the plug, deleted the reel, the recipe, the models, the code, all of it,
and kept `yossi_v2`.

> **The lesson, banked:** a clean-still A/B **oversells** AI super-resolution. Grain,
> compression, and motion are the real jury, and they're unforgiving. Trust the moving
> image over the frozen frame. For this footage the classical pipeline won and the AI
> didn't, and knowing that for certain is its own kind of progress.

---

## War stories from the render room

Two operational lessons worth recording, because a pipeline is only as good as its
behavior on a bad night:

- **The disk that filled at shot 71.** The 20-hour hybrid render died three-quarters
  of the way in when the drive hit **100% full**. Lossless intermediates pile up fast,
  and a single full-reel job's scratch runs to ~300 GB. What saved it: the **resumable
  manifest**. Every finished shot is recorded, so freeing space and re-running the exact
  same command reused all 71 completed shots and picked up precisely where it fell over,
  no work lost. Crash-resistance isn't a feature you notice until the night you need it.

- **Lossless is expensive; tidy up after yourself.** `--clean` bounds *one* job's disk
  footprint, but a *finished* job's scratch lingers until you sweep it. Across all the
  experiments, cleanup reclaimed **~355 GB**. The intermediate files that make the
  pipeline safe and resumable are the same ones that eat your drive; respect both.

---

## Where it all landed

Nine classical stages, refined by an outside review, are `yossi_v2`, and it's the one
we kept. We took the AI approach all the way to a finished 20-hour render and chose,
deliberately, to walk back. Sometimes the best thing a restoration can do is know when
it's done.

*Run the keeper yourself:*

```bash
python -m orchestrator.pipeline YossiBmFull.mp4 --recipe yossi_v2 --clean
```