Midshift has a serious, well-engineered automated test effort for a product of its size: 687 fast unit tests that all pass in about 16 seconds, a browser-driven "journey" suite of roughly 780 tests that exercises every screen against a simulated back end (467 of the 483 tests that completed in this audit passed; the remaining failures are an expired-date test and one layout check, not product defects), database policy tests, and a scripted acceptance walk that five test accounts run against a live site. The business-logic library is well covered (67% of its lines, and 85–100% for the most important parsing, matching and link-building rules). The weakness is where the tests do not reach: the server's API routes, where permissions are checked and data is written, have under 1% unit coverage, and the browser tests run against an in-memory imitation of those routes rather than the real ones, so they cannot catch a permission bug in the real code. The two most serious security findings in this report (Module 9) sit in exactly that gap. The browser suite and the database tests are also not part of the automated pipeline; they run only when someone runs them by hand.
🎯 Rating
🟡 ●●●○○ 3 / 5 — Amber. Strong unit testing of domain logic and broad UI journey coverage, but the authorisation and write path — the riskiest code — is effectively untested against the real implementation, and the slower suites are not enforced in CI.
24 of 25 files completed: 467 passed, 7 failed, 9 skipped._ui-metrics (293 page-walk tests) did not finish within two time-boxed runs (86 min and ~78 min, single worker as the runner enforces)
~45 min for the 24 files; > 2.5 h in total
[Observed]
Live end-to-end (e2e/*.spec.ts, 8 specs)
playwright test with BASE_URL
Not run
—
[Unavailable — requires a reachable deployment and test credentials; some specs write data]
Acceptance walk (e2e/walk/, 15 files)
.github/workflows/walk.yml (manual)
Not run
—
[Unavailable — requires a deployment and five persona accounts]
[Unavailable — needs a local PostgreSQL server; only the psql client is installed in the audit environment]
Typecheck / lint (static gates)
tsc --noEmit, eslint
Clean (exit 0, 0 warnings)
—
[Observed]
Flaky/skipped tests: none in the unit suite. The live e2e and walk specs contain 34 conditional test.skip(...) guards that skip teardown or dependent steps when an earlier step did not run — legitimate, not suppressed failures. [Observed] grep -rn "test.skip". No .only committed. [Observed]
Journey suite — failures and skips explained
File
Result
Cause
Product defect?
chain-dates-timeline
6 failed, 2 skipped, 30 passed
Three tests (each at two viewport widths) expect a fixture job starting 5 Oct 2026 to read "In production" (start date in the future). The status derivation in src/lib/campaign-derive.ts:56 uses the real system clock instead of the harness's pinned date (NEXT_PUBLIC_QA_TODAY=2026-09-04, read through todayIso()), so since 5 Oct 2026 the job reads "Active" and the dependent steps cascade. A time-bomb that breaks the project's own "one calendar" rule (CLAUDE.md, Verification). Skips are the chain's a11y step after the failures.
❌ No — test/clock inconsistency; but it shows status derivation and the harness disagree about "today" [Observed]
chain-help
1 failed, 11 passed
Layout-spacing assertion on Help sub-headings ("a sub-heading sits nearer its own lines than the block above it"); reports e.g. "32 above, 8 to its lines". Cause not diagnosed — possibly the assertion's own logic or font rendering in this container.
Cosmetic at most [Observed failure; cause Inferred-unknown]
chain-home-tracking
7 skipped, 26 passed
Tests skip themselves when the Ask button is absent: "Ask is parked until the API exists". The README advertises Ask as a feature; it is not rendered in the QA build.
Possible documentation drift — production exposure [Unavailable]
_ui-metrics
Not completed
293 page-walk tests on one worker; exceeded 86 min and ~78 min budgets
[Unavailable — needs a longer run or a sharded CI job]
That a date-bound test began failing two days before this audit, and nobody noticed, is itself evidence that the journey suite is not run routinely (it is not in CI). [Inferred]
Business-critical modules (selected from Module 4):
Module
Line %
Branch %
Business role
copy-deck-parser.ts
85.0
82.8
Turns copywriters' decks into ad units
asset-match.ts
98.2
96.0
Filename → unit join
utm-template.ts
100
100
Every tracking link
approval-events.ts
100
100
When an approval is reopened
copy-import.ts
100
100
What a deck may overwrite
retention.ts
100
100
When files may be purged
custody.ts
100
100
Handover custody
final-grid.ts
98.9
85.9
Go-live checks
plan-change.ts
96.0
88.9
Ad-name construction
discipline.ts
90.5
97.1
Who edits which area
branding.ts
76.3
94.7
White-label identity
api-auth.ts
3.6
—
All route authorisation
activity.ts
5.6
—
Audit trail
Note: the 11.7% headline understates protection of UI code, which the journey suite exercises in a browser; V8 coverage from that suite was not collected. [Observed / Inferred]
supabase/tests/040_rls.sql (187 lines) + data-migration fixture/assert sets for 062–064
Throwaway Postgres built from baseline.sql
❌ No (manual)
UI journeys
~780 tests in 25 files (483 executed to completion here)
Real built Next.js app + in-memory mock store (src/app/qa/store.ts) instead of Supabase
❌ No (local; documented as ~30 min, measured > 2.5 h in this container)
End-to-end, live
8 specs (~20 tests) + 15-file acceptance walk (28 tests)
Real deployment
❌ No (manual dispatch)
The shape is a reasonable pyramid at the bottom and top, with a hollow middle: there are no automated tests that execute the real route handlers against a real (even local) Supabase. [Observed] The journey suite substitutes a hand-written emulation of all 125 handlers; scripts/harness-contract.js checks that every route has a twin, not that the twin behaves like the route. [Observed] The QA store re-implements the authorisation helpers (src/app/qa/store.ts:761–822, Module 4), so a bug in the real api-auth.ts is invisible to the browser tests. [Observed]
4. Test quality
Behavioural, specification-style assertions. The parser baseline test asserts the agreed parse of an invented 22-unit deck — unit count, channel histogram, card labels, Journey states — with comments explaining why each assertion exists. [Observed] src/lib/copy-deck-parser.baseline.test.ts:1–40.
Tests as guard rails over the codebase itself.write-results.test.ts fails if any awaited write's result is unread or a DELETE does not count rows; schema-columns.test.ts fails if code names a column absent from the schema. These encode past production bugs as permanent checks — a mature practice. [Observed]
"Plant once" discipline — HANDOVER.md requires each fixed rule to get a test that is proven to fail when the rule is broken. [Observed] docs/HANDOVER.md "How a change lands".
Fixed calendar for the harness (NEXT_PUBLIC_QA_TODAY=2026-09-04) avoids date-rot flakiness. [Observed] e2e/journeys/qa-server.env.
Accessibility is checked in the journey suite with axe (_a11y.spec.ts). [Observed]
grep -n "users" supabase/tests/040_rls.sql → users rows created as fixtures only; no self-update case.
which psql postgres pg_ctl → only psql present.
⚠️ Risks
Risk
Severity
Effort
Authorisation and write routes untested against the real implementation; the harness re-implements the rules, so it cannot detect real permission bugs (Module 9's Critical and High findings went undetected)
🟠 High
M
Journey, RLS and live suites not enforced in CI — they protect only when someone remembers to run them
🟠 High
M
No integration environment running real routes against a real Supabase; parity between harness, baseline and each production database unverified
🟡 Medium
M
No coverage thresholds — coverage can fall silently
⚪ Low
S
Harness store (3,349 lines) is itself untested logic that can drift from the routes it imitates
🟡 Medium
M
✅ Recommendations
Recommendation
Priority
Effort
Add route-level integration tests that run real handlers against a local Supabase (Supabase CLI in CI), starting with api-auth.ts, members, ad-unit PATCH, brief/campaign PATCH, share-link and cron routes
P1
M
Add RLS test cases for every table a user client can write — users first
P0
S
Run the journey suite in CI (sharded, nightly at minimum) and the RLS check on every migration change
P1
M
Stand up a shared staging deployment with its own Supabase project for the live e2e and walk
P2
M
Set coverage floors for src/lib (≥ 65%) and a rising floor for src/app/api