The linesolid node = trunkdashed node = fork

  1. 00The line
  3. 01Choreograph
  5. 02Despill
  7. 03Slice & register
  9. 04 forkComposite-back
  11. 05 forkPlanes & plates
  13. 06Judgment & pack
assetfarmer Stage 03 · slice & register slice.py — 146 lines · align.py — 170 lines

input: sheet-transparent.png · output: canvas-locked RGBA frames + canvas.json

Recovering frames, and refusing to guess

Two stages, one shared conviction. Slicing declines to invent frames it cannot find; registration declines to let a gesturing arm move the body. Both are small pieces of code whose value lies almost entirely in what they refuse to do.

  1. 03a

    Components, not grid lines

    farmer/slice.py

    The obvious way to cut a 2×3 sheet is to divide by three and by two. assetfarmer does not do that by default. It labels 4-connected components of alpha > 16, ranks them by area, and takes the declared largest as frame seeds. The reason is diagnostic: grid cutting always “succeeds”, even when two frames have fused into one blob or a subject has drifted across a cell edge. Component labelling notices.

    Connected-component slicing, seed ranking and the no-silent-fallback test On the left, a despilled two by three sheet holds six subject blobs plus two small noise specks. In the middle, all eight components are listed in descending area order; the six largest become seeds and the two smallest are noise. Two guard tests are drawn: the count test, which requires at least as many components as declared frames, and the seed ratio test, which requires the smallest seed to be at least ten percent of the largest. Below them, a failing case shows two frames fused into a single component, which trips the count test and raises SliceError with every component's area and bounding box in the message. On the right, noise components are attached to the nearest seed by centroid distance, the seeds are ordered row major by row band then x, and each is cropped to its own bounding box with everything outside its mask zeroed. 1 — label components of alpha > 16 c1c2c3 c4c5c6 n1 n2 8 components found, declared = 6 2 — rank by area, apply two guards seed 1 — largestseed 2 seed 3seed 4 seed 5seed 6 — smallest seed noise n1 noise n2 guard 1 — count len(comps) >= declared 8 ≥ 6  →  passes guard 2 — seed ratio smallest >= largest × 0.10 a 6th seed under a tenth of the 1st means fused frames + noise the failing case — two frames fused frames 2+3 touch 7 components found, declared = 6 → guard 1 still passes, but the fused blob is one seed and a filler cell is missing → guard 1 fails once the count drops below the declaration The error, verbatim (slice.py:84–87) f"declared {declared} frames but only {len(comps)} components found (frames touching/fused?). Components: [{listing}]. Regenerate the sheet or fix containment — no silent fallback." The listing carries every component's area and bbox, so the agent can tell “two frames merged” from “one frame vanished”. 3 — attach, order, crop attach noise each leftover component joins the seed with the nearest centroid; the seed's mask and bbox both grow to include it order row-major cell_h = sheet_h / rows key = (int(cy // cell_h), cx) bucket into row bands first, then sort left to right inside the band crop to the mask each frame is cut to its own bbox and every pixel outside its component mask is zeroed — a neighbouring frame's overlapping bbox cannot leak in the opt-in fallback -‌-slice-mode grid cuts on cell boundaries and keeps each cell's largest component. Never automatic.
    Figure 7 — slicing with two guards. The second guard is the subtle one: even when enough components exist, a smallest seed under a tenth of the largest usually means two frames fused and a speck of noise was promoted into the gap. Both guards raise rather than repair.

    “If the declared frame count cannot be recovered from the components found, this is a HARD ERROR listing what was found — never a silent guess.”

    farmer/slice.py:3–5
    Grid mode is not a repair

    slice_by_grid() exists and is reachable, but only because the caller asked for it. It keeps each cell's largest component plus any detached part at least 2% of that area — a lamp's separated glow, a held prop — and raises if a cell is empty. The distinction the codebase insists on is that a fallback the user chose is a decision, and a fallback the code chose is a lie.

  2. list[Image] → align_frames(frames, cell_w, cell_h)
  3. 03b

    Registration that ignores the arms

    farmer/align.py

    Sliced frames arrive as independent crops with no shared origin. Registration puts them back on a common canvas by translation only — no rotation, no scale, because scale drift is a generation failure, not something to paper over. The choice that makes it work is which pixels get to vote.

    Torso-weighted registration on the working canvas, and the canvas lock Left: a 576 by 576 working canvas, derived from a 512 pixel cell plus 32 pixels of shift margin on each side, with the baseline at y equals 540. A frame is pasted with its alpha centroid at the horizontal centre and its bounding box bottom on the baseline. The upper 65 percent of the subject's rows is shaded as the torso mask; the lower 35 percent, including a raised gesturing arm, is excluded from the vote. Middle: an FFT cross-correlation of the two torso masks produces a correlation surface, and the argmax is searched only inside a plus or minus 32 pixel window, yielding an integer shift. Right: the union of all aligned frames' alpha bounding boxes, padded by 12 pixels, becomes the canvas box stored in canvas.json. A second panel shows what happens when a later action needs a larger box: the boxes are merged and every sibling action is re-cropped from its stored full working canvases, with its manifest frame size rewritten. working canvas — cell_w + 2×MAX_SHIFT 512 cell 576 TORSO_FRACTION = 0.65 — cut here only these rows vote legs excluded arm excluded baseline = canvas_h − 32 − 4 = 540 initial paste: alpha-centroid x at canvas centre, bbox bottom on the baseline (align.py:38–58) shift search ±32 px window corr = ifft2(fft2(ref) · conj(fft2(mask))) dx, dy = argmax inside the window integer pixels only — no resampling, so no frame is ever softened by the alignment then one global x-centre frame 0's full alpha centroid is centred and the same offset is applied to every frame, so relative registration is preserved exactly canvas lock union bbox + pad 12 canvas.json — guide canvas_box: [110, 163, 366, 557] cell_w: 512, cell_h: 512 → every frame of every guide action is 256 × 394, confirmed in both runs' manifests when a later action needs a bigger box 1. merge_canvas_box() takes the union of stored and new 2. _recrop_sibling_actions() reopens every other action's work/work_i.png full canvases 3. old frames/*.png are deleted and rewritten at the new crop 4. each manifest's frame_size is rewritten to match pipeline.py:141–169 why work/ exists The cropped frames are lossy with respect to a future larger box, so the full 576 × 576 aligned canvases are kept on disk per run. Without them the lock could only ever shrink, never grow. pipeline.py:119–122
    Figure 8 — registration and the lock. The upper-65% mask is what lets a presenting gesture animate without the body sliding: the arm is outside the vote. The canvas lock is what lets two different actions of the same character be composited at the same on-screen scale without any runtime bookkeeping.

    Two different numbers that both look like “jitter”

    The validation report carries both, and only one of them gates the run. Confusing them would make every gesture animation look broken.

    Measurements from the repository's committed validation.json files
    Run max anchor jitter (px) max torso motion (px) Gated?
    receptionist/idle-breathing0.250.86jitter only
    guide/idle-breathing0.720.90jitter only
    guide/gesture-presenting1.129.02jitter only
    janitor-idle-breathing1.171.12jitter only
    lamp/flicker0.4123.04jitter only

    Anchor jitter is measured by qa.anchor_jitter() on the bottom 15% of frame 0's subject bounding box, in fixed canvas coordinates. It answers one question: did the standing base slide? That is the artefact the eye reads as broken, and it is the number checked against JITTER_LIMIT_PX = 3.0.

    Torso motion is the residual torso-centroid distance left after registration. It is supposed to be non-zero — it contains the choreography. The two extremes in the table make the point: a presenting gesture registers 9 px of torso movement because the character leans, and the lamp's flicker loop registers 23 px because a growing glow changes what the upper-65% mask even covers. Both runs pass, correctly, because their bases are planted to within half a pixel.

    A limitation the code states about itself

    Jitter <3px is measured on the torso, not on gesturing limbs — the README's wording, matching the comment on max_torso_motion_px in qa.py:138, marked informational: includes choreographed motion. No check in the codebase gates gesture quality; that judgment stays with the agent looking at the contact sheet.