Cavai Docs

Appearance

Specsavers "Design ditt par" -- Data Correction Plan ​

Creative: Specsavers // Design ditt par // Double Fullscreen // 2026 Creative ID: 90991 Campaign period: 2026-04-14 to ongoing Date for analysis: Monday 2026-04-28

Root Cause: Script-Driven Actions ​

The creative uses 4 custom scripts + 1 library script (responsive-video-swap). The 4th script is the root cause of inflated actions and conversation metrics.

How it works ​

Users select glasses and colors via radio buttons (a custom HTML form, not flow choices). On every radio change, script 4 programmatically navigates the flow:

User changes radio button
  -> script clicks ".c15" (Back choice node)
  -> script waits for target choice to appear
  -> script clicks target choice (e.g. ".c18" = Hele_Sun_RX_Blue)

Each glasses/color change = 2 programmatic .click() calls on flow choice nodes. These count as actions and flow events. A user trying multiple combinations generates many actions -- this is correct behavior, each action represents a real user choice of frame + color.

The ready flag (original auto-click fix) ​

javascript
setTimeout(() => { ready = true }, 500);

This 500ms delay prevents the initial default radio values from triggering clicks on load. Before this fix (Apr 14-21), every single impression auto-fired clicks.

Actions rate timeline ​

  • Apr 14-19: Test period, low volume, auto-click bug active but few impressions
  • Apr 20-21: Auto-click bug + high volume = massively inflated (38-69%)
  • Apr 22-23: Bug fixed, ~1.5% action rate (but interacted was broken, so low engagement visible)
  • Apr 24: New publish (goal change) -- actions jumped to ~829%. The publish got actions working properly, so now it reflects real user choices at scale.
  • Apr 25: 1,119.75% actions -- confirms this is the real rate post-fix. Each interacting user tries ~11 combinations on average.

Metrics Status (validated Apr 25) ​

Working correctly now ​

  • Interacted: 0.86% (Apr 25, full day) -- use this as the real baseline, not 0.31% from Apr 24 partial day
  • Started: 0.18% -- consistent
  • Continued: 92.59% of started -- consistent
  • Actions: ~1,119% -- correct, represents programmatic flow clicks from user choices
  • Clickthroughs: 0.42% -- link clicks out to Specsavers
  • Video metrics: W25/50/75/100 all reliable throughout
  • Viewable / Seconds in View: Reliable throughout

Still broken ​

  • Finished Conversation: 0 in report UI. Goal/reachedEnd mechanism does not fire. This is a platform bug to investigate separately.

How to derive Finished Conversation ​

Multiple approaches, all valid:

  1. Go_Back? flow node -- users who completed the full flow (chose frame + color and saw the result)
  2. Any choice click at the end -- all color choice nodes (c18-c35) represent a completed interaction
  3. Clickthroughs -- users who clicked through to Specsavers have definitely "finished" the experience

For client reporting, recommend using Go_Back? flow data as primary "Finished" metric, with clickthroughs as a separate "Converted" metric.

Data Periods and What to Correct ​

Period 1: Apr 14-19 (test/ramp-up, 4,879 impressions) ​

  • Very low volume, no flow data was being recorded
  • Auto-click bug was active but few impressions
  • Approach: These days have impressions and video metrics (plays, watch quartiles, viewable, seconds in view) but NO meaningful conversation/action data
  • For client: Show only video/viewability metrics for these days. Note as "test period" with limited data. Don't estimate conversation metrics -- just show what we have (impressions, viewable, seconds in view, video completion)
  • Key question for Monday: Do these days have viewable/plays data? If yes, show those. If no, they're truly empty test days.

Period 2: Apr 20-21 (auto-click bug, 244,813 impressions) ​

  • High volume but actions/started/continued massively inflated
  • Approach: Estimate corrected values using post-fix baselines
  • Actions baseline: use Apr 25 rate (~11x per interacting user, with ~0.86% interacted)
  • Started/Continued: use post-fix rates (0.18% / 92.6% of started)
  • Video/viewability metrics are correct as-is
  • Finished: estimate from Go_Back? flow data if available, otherwise use post-fix rate

Period 3: Apr 22-23 (post-fix but interacted broken, 238,459 impressions) ​

  • Actions and conversation metrics are correct
  • Interacted = 0 (broken) -- estimate using Apr 25 rate (0.86%)
  • Everything else is reliable

Period 4: Apr 24+ (all working except Finished Conversation) ​

  • All metrics reliable
  • Interacted confirmed working (0.86% on Apr 25)
  • Finished = 0 is a platform bug -- derive from flow data (Go_Back?)
  • Actions rate is high but correct (reflects real user choices)

Monday Checklist ​

Data ​

  • [ ] Pull fresh report Apr 14-27 (will have 3 full post-fix days: Apr 25-27)
  • [ ] Pull daily flow data for Apr 14-27
  • [ ] Verify Apr 14-19 has viewable/plays data (determines if we show those days or mark as test-only)
  • [ ] Extract Go_Back? node impressions per day as "Finished" metric
  • [ ] Extract clickthroughs per day as "Converted" metric

Corrections ​

  • [ ] Use Apr 25-27 as baseline for interacted rate (expect ~0.86%)
  • [ ] Estimate interacted for Apr 14-24 using that baseline
  • [ ] For Apr 20-21: estimate corrected actions using post-fix interacted rate * avg actions per interacting user
  • [ ] For Apr 14-19: only show video/viewability metrics, note as test period
  • [ ] Add Finished (Go_Back?) column from flow data
  • [ ] Add Clickthroughs column

Presentation ​

  • [ ] Decide on final metrics to show client:
    • Reach: Impressions, Viewable Impressions
    • Attention: Seconds/Minutes/Hours in View, avg seconds per viewable impression
    • Video: Plays, W25%, W50%, W75%, W100% (completion rate)
    • Interaction: Interacted (unique users who engaged)
    • Completed experience: Go_Back? (explored product) + Clickthroughs (clicked to Specsavers). Both count as "finished" -- Go_Back means they completed the glasses/color flow, clickthroughs means they converted to the landing page. Some users click through without completing the full flow, which is equally valuable.
    • NOT raw actions (confusing for client, inflated by programmatic flow clicks)
  • [ ] Build final corrected Excel with clean numbers
  • [ ] Write short summary explaining the test period and estimation methodology
  • [ ] Be transparent: "new setup, some metrics needed calibration in the first week"

Platform bug ​

  • [ ] Investigate why goal/reachedEnd doesn't fire (separate from client report)
  • [ ] File as internal bug if confirmed

Internal documentation

Cavai