Fallback Capture — Design Spec

Date: 2026-08-18 Status: Approved by Nicolay — ready for implementation planning Issue: AF#1930Branch: add-fallback-capture (Application-Frontend; CE branch only if bridge extensions are needed) Feasibility & spike results: todos/FallbackCapture/feasibility.md — all three risky primitives verified 2026-08-18 (pixel-perfect Region Capture of the preview iframe, flow stepping with branch choices + reset, video seek).

Summary

Let users generate the single static fallback image for a creative directly in the builder. A modal tool shows a live preview navigated to the creative's end state by default; a timeline lets the user scrub flow steps (choosing branches) and video time; one click captures the preview iframe pixel-perfectly, downscales it to the ad size, and stores it on the creative. The Delivery tab shows the current fallback and is the place to create, replace, download, or remove it.

Core UX rules:

• One fallback per creative. A new capture overwrites the old one.
• Publish never blocks and never nags. The Delivery section carries the status.
• The capture permission dialog is native and unavoidable (getDisplayMedia requires a fresh user gesture per capture by spec — no persistent grant exists in any browser). The UX embraces it: capture is a deliberate action, the stream is auto-stopped right after each capture, and no "stop sharing" UI is needed.

UX flow

Delivery tab — "Fallback image" section

A new section alongside tag export:

• Has fallback: thumbnail, dimensions, captured-at timestamp. Actions: Replace (opens tool), Download PNG, Remove.
• No fallback: empty-state with short explanation ("Static backup image for ad servers and no-JS environments") and Create fallback button.

All labels via $t() (new keys under a fallback section in en.js).

Capture tool — modal

FallbackCaptureDialog opens from the Delivery section (works regardless of active builder tab because it mounts its own preview):

1. Own LocalBuildPreview instance at 100% zoom, sized to the creative format.
2. On open, auto-navigate to the end state: walk the flow to the last step, taking the first answer at each branch as default.
3. Timeline below the preview (see next section) for adjusting the state.
4. Capture button → native tab-share dialog → grab one frame → stop stream immediately → show captured result in the modal.
5. Use as fallback → downscale to exact ad dimensions → upload as asset → save reference on the creative → close modal → Delivery section shows the new image. Retake discards and returns to step 3.

Timeline

Driven by the existing CE devtools flow bridge (window.__cavaiDevtools: getFlowInfo, getStepComponents, stepTo):

• Horizontal strip of clickable step nodes; current step highlighted.
• At branch points (a step whose components offer multiple choices), the choices render as selectable chips; picking one advances along that path.
• Go to end button repeats the default end-state navigation.
• Backwards navigation = reset (iframe reload; LocalBuildPreview re-sends data on preview-iframe-ready) + replay of the recorded choice sequence up to the target step. Known gotcha: the browser keeps the same contentWindow proxy across iframe reloads, so step-UI must be remounted via a :key bumped on reload.
• When the current step contains a video block: a time scrubber sets videoEl.currentTime directly (same-origin) and waits for seeked before capture is allowed. Video is paused while scrubbing.

Determinism note: the rendered state at a step is deterministic given the choice path (verified in feasibility). Time-based operators (Delay) may need the same stabilization polling FlowControls.executeStep() uses.

Capture pipeline

All capture logic lives in a pure util module src/utils/fallbackCapture.ts (no Vue reactivity, no store access — per conventions), shared by the product dialog and the existing DevTools spike panel (which stays, as a thin consumer, for debugging):

1. getDisplayMedia({ video: true, audio: false, preferCurrentTab: true }) — one native prompt per capture.
2. Region Capture (CropTarget.fromElement(iframe) + track.cropTo) when available; otherwise manual crop of the full-tab frame using the iframe's bounding rect scaled by videoWidth / window.innerWidth (handles devicePixelRatio and browser zoom in one factor).
3. Hide overlapping UI (the modal chrome around the preview) during the frame grab; wait for fresh compositor frames (requestVideoFrameCallback ×3) so crop and hiding have settled.
4. Stop the stream immediately after the frame is grabbed.
5. Downscale to exact ad dimensions (e.g. a 1050×2100 capture of a 300×600 creative → 300×600 PNG) via canvas drawImage with high-quality scaling. Spike measured 3.5× capture scale on a retina display.

The crop/scale math is pure and unit-testable (Vitest).

Data & storage

• Upload: reuse the existing asset upload service/endpoint. Hide the image from the normal Asset Library views if the asset model supports a type/flag for it (verify during planning); otherwise use a clear naming convention (fallback_<creativeName>_<w>x<h>.png).
• Reference: stored in the creative's existing data blob:

  creativeProperties.fallbackImage = {
    assetId: number,
    url: string,
    width: number,
    height: number,
    capturedAt: string, // ISO timestamp
  }

  Saved through normal creative save — no new backend endpoints expected.
• Replace: new capture overwrites the reference; delete the previous asset if the API allows.
• Remove: clears the reference (and deletes the asset if possible).

Type definition goes in the blocks/creative types companion file (type, not interface, per conventions), not inside .vue files.

Creative-Engine changes

Expected: none required — the flow bridge already exposes everything the timeline needs. Candidates only if AF-side proves clunky during implementation (decide in planning):

• resetAndReplay(choices) helper in flowBridge.ts for atomic backwards navigation.
• Per-step video element lookup.

Any CE change follows CE conventions (JSS styling only, DataStore state, #include annotations preserved, manual lint-save before commit).

Component architecture (Application-Frontend)

Component	Location	Responsibility
FallbackSection.vue	src/pages/Chatbots/components/Delivery/	Status card + actions in Delivery tab
FallbackCaptureDialog.vue	src/pages/Chatbots/components/Delivery/	Modal: preview instance, timeline, capture orchestration, save flow
FlowTimeline.vue	src/pages/Chatbots/components/Delivery/ (promote to common/ only if a second consumer appears)	Step strip + branch chips + go-to-end; wraps flow bridge
fallbackCapture.ts	src/utils/	Pure capture/crop/downscale logic

Conventions checklist (applies to all of the above):

• All user-visible strings via $t(), keys added to en.js.
• Reuse existing components: Card, InputSelect (no native <select>), existing dialog patterns from components/dialogs/, LocalBuildPreview, existing asset upload service, toastSuccess/toastError utils, cloneDeep from lodash.
• Options API (mixin ecosystem); these components don't use sectionLogic, so no sectionSettings entries are needed.
• inputLocked/permission gating: the Delivery section's mutating actions (create/replace/remove) respect the same locked/read-only state as other Delivery mutations.
• Arrow functions for utils, early returns, breathing room, type over interface, section comments in longer files.
• Scoped SCSS with CSS variables (var(--surface-base) etc.) and SCSS size variables — the tool must look at home in the builder: this is product UI, not devtools chrome. Timeline styled to match the builder's design language (dots/segments consistent with existing preview controls).

Out of scope (v1)

• <noscript> image in the generated tag (easy v1.1 once the URL exists on the creative).
• Multi-size batch capture; server-side headless rendering (the only path to zero-dialog/bulk generation).
• In-builder video crop/trim tooling and flow play/pause video operators (separate future issues — ideas logged in feasibility doc).

Testing

• Unit (Vitest): crop rect math, scale-factor math, downscale dimensions, fallbackImage reference shape.
• Manual (per repo conventions): create/replace/remove flow in builder; capture correctness with video + branches; Delivery download; verify saved creative round-trips the fallbackImage blob; no console errors; standalone delivery preview unaffected; Chrome primary, one non-Chromium browser for the manual-crop path.
• Spike panel in DevTools remains available for low-level debugging.