Required Testing Behavior

Mandatory behaviors for AI agents when testing frontend code.

The Five Core Rules

Every AI agent working on frontend code must follow these five rules when testing:

Rule 1: Declare Your Test Plan First

Before writing any test or running any verification, state:

• What you are testing

• Which level you are using

• Why that level is appropriate

Test plan:
- Testing: notification banner visibility after CSS fix
- Level: 5 (verify-ui + screenshot)
- Reason: This is a visual bug — the element exists in DOM but
  user reports it's not visible. Need to check computed styles.

Rule 2: Match Test Level to Goal

The test level must match what you are actually verifying:

Do not use Level 1 for a Level 5 problem. The test will pass, but the bug will remain. Do not jump to Level 6 just because L4 is hard or L5 is awkward — L6 requires that both lower levels (L4 and L5) are genuinely intractable.

Rule 3: Escalate When Lower Levels Pass But Problem Persists

When a test passes but the user says the problem is not fixed:

1. Do not re-run the same test

2. Do not suggest the user clear their cache

3. Do escalate to the next testing level

4. If already at Level 4, escalate to Level 5

5. If at Level 5, investigate deeper (check ancestor elements, stacking context, etc.)

6. Only escalate to Level 6 (AI-based) when both L4 is intractable to write and L5 cannot reach the assertion. L6 is not the next step after L5 by default — it is reserved for canvas / photo-editor surfaces where no deterministic check is feasible.

Rule 4: Default to Level 5 for UI/CSS

Any task involving:

• CSS changes

• Layout modifications

• Visibility issues

• Visual appearance

• Responsive design

• Spacing, colors, fonts

should default to Level 5 verification. Lower levels are structurally unable to verify CSS correctness.

Rule 5: Report What Was NOT Tested

After testing, explicitly state the blind spots:

Verification complete:
- Tested: computed styles confirm banner has display:block,
  opacity:1, and parent has overflow:visible
- Screenshot: banner is visible at top of page
- NOT tested: responsive behavior at mobile breakpoints
- NOT tested: animation transition timing

This transparency helps the user decide if additional testing is needed.

The Four Execution Rules

Agents operating inside a tiered test suite must also follow these four rules. For background on verification artifacts, tier tags, and the graduation lifecycle, see Execution Tiers.

Rule 6 — Verification Tests Do Not Graduate Themselves

Agent-authored verification specs are tagged @verification and excluded from all gates. Promotion to a tier is an explicit, deliberate step:

1. Remove the @verification tag

2. Assign the appropriate tier tag (e.g., @heavy, @gpu)

3. Confirm the spec fits within the target suite's time budget

An agent never promotes its own verification test. It proposes promotion in the PR description and leaves the decision to the reviewer.

# PR description excerpt
## Verification spec — graduation proposal
- spec: tests/banner-render.spec.ts
- currently: @verification (excluded from gates)
- proposed tier: T1 (untagged CI-safe e2e)
- rationale: stable for 5 runs, sub-2s, no platform dependency
- action needed: reviewer removes @verification tag and assigns tier

Rule 7 — Red Checks Block the Author

Any red check on a PR you authored is blocking for you, regardless of whether the failing check is marked required in the repository settings. The only exception: the failing test carries @flaky with a linked open issue.

# Blocked — do not merge
PR #42 authored by agent-session-A
  ✗  unit: auth-service.test.ts — 1 failed
     (not a required check, but authored by this session)
  → investigate and fix before declaring done

# Not blocked — exception applies
  ✗  e2e: flaky-animation.spec.ts @flaky — see issue #38
     (quarantined; allowed-to-fail in scheduled tier)
  → merge may proceed; issue #38 tracks the fix

Rule 7 is the agent/convention layer; the branch ruleset is the machine/enforcement layer -- neither substitutes for the other (see Make the gate enforced, not just declared).

Rule 7 binds you as the author of the red PR. The inverse role — you are operating a merge gate, and the red is on a test you don't own — has its own procedure: the gate operator's playbook.

Rule 8 — Never Game the Gate

Never introduce any of the following without a linked open issue reference:

• test.skip on an existing test

• @flaky tag on a test that was previously passing

• Loosened screenshot tolerance (e.g., raised pixel threshold)

• Deleted assertions

Making a gate pass by editing existing assertions requires a fresh-context review of the test diff — do not apply such edits in the same session that authored the original change.

This requirement extends to gate-defining files, not just assertions. A gate can be neutralized without touching a single assertion: narrowing a testMatch pattern so failing specs stop running, adding a --grep-invert filter, raising a retry count, dropping or de-requiring a CI job, or editing a fixture/setup script so the failing scenario is never built. Because retry budgets, grep filters, testMatch patterns, and job wiring can neutralize a gate this way, any edit to a gate-defining file — test-runner config, CI workflow definitions, fixture/setup scripts, the pre-push gate script — requires the same fresh-context review: read the gate file from scratch rather than trust the author's diff summary, and do not apply such edits in the same session that authored the original change.

Projects should enumerate their concrete gate-file list in their testing strategy doc (for example: the runner config, the fixture setup script, both CI workflow files, and the pre-push script).

# Wrong — no issue reference
test.skip('banner renders correctly', () => { ... })

# Wrong — no issue reference
expect(page).toHaveScreenshot({ maxDiffPixels: 500 }) // was 50

# Wrong — gate-defining file edited to make a gate pass, no fresh-context review
testMatch: ['**/*.spec.ts', '!**/banner-*.spec.ts'] // silently excludes failing spec

# Right (a) — legitimate skip: a real, deliberate skip, not a flake
// skipped: https://github.com/your-org/your-app/issues/55 — pending backend flag rollout
test.skip('banner renders correctly', () => { ... })

# Right (b) — quarantine: NOT skipped; the test still runs, allowed-to-fail,
# with @flaky in the title, exactly matching the quarantine pipeline's convention
// quarantined: https://github.com/your-org/your-app/issues/55
test('banner renders correctly @flaky', () => { ... })

Rule 9 — Scoped Heavy Verification

When a change touches code that is covered only by heavy-lane tests (tagged @heavy, @gpu, @interactive, or @macos-only), run the related heavy specs on a capable host before declaring the work done. Alternatively, dispatch the exam workflow on the branch.

See Heavy Test Decision Rule for how to identify which tests count as heavy-lane and which host is required.

# Before declaring done — change touches GPU rendering path
$ gh workflow run exam.yml --ref my-feature-branch
# or, if running locally on a capable host:
$ pnpm exec playwright test --grep "@gpu" tests/gpu-render.spec.ts

Summary Checklist

Before declaring any fix complete:

• Test plan was declared before testing

• Test level matches the nature of the change

• If test passes but user reports failure, escalated to next level

• For CSS/visual changes, used Level 5

• Reported what was and was not tested

• New verification specs are tagged @verification and not self-promoted

• All red checks on own PR are resolved or carry @flaky with a linked issue

• No test.skip, @flaky, or loosened tolerances introduced without a linked issue

• If change touches heavy-lane-only code, related heavy specs were run on a capable host