Elicitation: The Analyst and the grill-me Skill

Part 5 of 8 — the Analyst's loop made concrete, the flagship grill-me skill doing real work, and the path from a one-line ask to a story you can build.

The loop, made concrete

Chapter 2 sketched the Analyst's five-step loop in the abstract: ground → grill → resolve → draft → scan impact. This chapter makes it real on PromoDesk. Each step maps to a skill the Analyst actually runs, and each skill reads from the catalog rather than from the model's imagination.

The Analyst's loop is a read/ask cycle: grill-me keeps asking until the human has resolved the open questions, then draft-story and impact-scan turn the resolved need into an artifact.

grill-me: the flagship

Of all the skills, grill-me is the one that earns the system its keep. Its job is to do, mechanically and exhaustively, what a great human analyst does on a good day and a tired one skips: interrogate a half-formed idea against everything already known to be true about the system before a single line of a story is written.

When invoked on a draft requirement, grill-me:

• Reads the relevant catalog slice — the entities the ask plausibly touches: domain rules, screens, batch jobs, integrations, the authz matrix, the glossary.
• Surfaces ambiguities — every place the request could mean two things.
• Finds conflicts with current behavior — where the ask contradicts a documented rule or threshold.
• Names missing edge cases — the unmatched claim, the late EDI file, the back-dated promotion.
• Flags overlap with existing features — "the Fund Dashboard already shows committed accruals; is this the same view or a new one?"
• Lists the impacted entities — a first cut at blast radius, refined later by impact-scan.

Its output is not prose. It is a structured set of questions, each one carrying its own justification: why the question matters, the trade-offs of each plausible answer, and the catalog fact that prompted it. A question without a cited fact behind it is the model guessing; grill-me is built to never do that.

A worked transcript

A key-account director drops a one-liner into the Orchestrator. Watch grill-me work.

Human:  We need better visibility into promotion spend.

Analyst (grill-me):
  Reading catalog slice:
    - docs/rules/fund-balance.md        (accrual vs. committed vs. settled)
    - docs/rules/claim-matching.md      (deduction → promotion tolerance)
    - docs/security/authz.md            (who sees which accounts)
    - docs/ui/fund-dashboard.md         (existing spend view)
    - docs/batch/claims-matcher.md      (nightly match job & freshness)

  Before I draft a story, six things are underspecified. Each
  changes what we build. Please decide:

The six questions it produces — the same hidden questions chapter 1 warned that "promotion spend" silently smuggles in — each grounded in a fact it just read:

The director answers. Note that the human decides; the agent only framed the choices.

Human:
  1. Committed spend (accrued less released), not settled.
  2. Per key account, rolled up to fund. No company-wide tier yet.
  3. Daily snapshot is fine. No live query.
  4. Show unmatched claims as a SEPARATE "at-risk" line, not folded in.
  5. As-of last night's matcher run; show the as-of timestamp.
  6. Extend the Fund Dashboard. New "Spend & Exposure" panel.

Six answers, six decisions captured with provenance. The ambiguity that would have caused two sprints of rework is gone before the story exists.

draft-story: from resolved need to Jira

With the questions resolved, the Analyst runs draft-story. It turns the decisions into a Jira user story: a title, an "as a… I want… so that…" narrative, an INVEST self-check, Gherkin-style acceptance criteria, and links to every catalog entity it touches.

PROMO-412  —  Fund Dashboard: Spend & Exposure panel

Narrative
  As a key-account director
  I want a daily Spend & Exposure panel on the Fund Dashboard,
    scoped to my accounts and rolled up to fund,
  so that I can see committed promotion spend and at-risk
    unmatched claims without exporting to a spreadsheet.

INVEST check
  Independent  — new panel, no dependency on other open stories
  Negotiable   — grain and freshness fixed by elicitation (see links)
  Valuable     — removes manual spreadsheet rollup for directors
  Estimable    — reuses Fund Dashboard auth + daily snapshot table
  Small        — one panel, one snapshot query, no new batch job
  Testable     — criteria below

Links (impacted entities)
  rules/fund-balance.md · ui/fund-dashboard.md · rules/claim-matching.md
  security/authz.md · batch/claims-matcher.md

The acceptance criteria are written as Given/When/Then so they are mechanically testable — and they encode the exact decisions from the grill, including the ones a looser process would have lost:

Acceptance criteria

Scenario: Committed spend, scoped to my accounts
  Given I am a key-account director
    And I own accounts A and B
  When I open the Fund Dashboard Spend & Exposure panel
  Then I see committed spend (accrued less released) per account
    And a fund-level rollup of those accounts
    And no accounts I do not own.

Scenario: Unmatched claims shown separately
  Given there are deduction claims unmatched by the nightly matcher
  When I view the panel
  Then those claims appear on a distinct "At-risk" line
    And are NOT folded into committed spend.

Scenario: Freshness is explicit
  Given the claims-matcher last ran at 02:00 today
  When I view the panel
  Then it displays "As of 02:00" alongside the figures.

impact-scan: walk the graph

Before the story leaves the Analyst, impact-scan walks the catalog as a graph — the linked-entity model from chapter 3 — and lists every screen, batch job, integration, and rule the change touches. It is the difference between "build the panel" and "build the panel, and here is everything downstream that will feel it."

impact-scan PROMO-412

Screens
  ui/fund-dashboard.md          MODIFIED  new Spend & Exposure panel
Rules
  rules/fund-balance.md         READ      "committed" definition reused
  rules/claim-matching.md       READ      "unmatched" pool surfaced in UI
Batch
  batch/claims-matcher.md       DEPENDS   panel freshness bound to its run
Integrations
  (none)                        —        no ERP/EDI contract change
Security
  security/authz.md             REUSED    account-scoping unchanged
NFR
  nfr.md                        CHECK     daily snapshot keeps query under cap

Verdict: 1 screen modified, 1 rule newly surfaced in UI, freshness
coupled to claims-matcher. No integration or schema-of-record change.

trace: where does X live?

The smallest of the elicitation skills, and one of the most used. trace answers a single question with provenance: where is X implemented, documented, or decided? "Where is the $50k approval threshold set?" returns security/authz.md plus the ADR that chose it; "where is committed spend defined?" returns rules/fund-balance.md. It is how the Analyst (and the human) avoid re-litigating settled facts, and how every claim in a grill stays traceable back to a real entity.

What a skill actually is

A Claude Code skill is a small Markdown file declaring a name, a description, and — for these — what catalog it is allowed to read. Here is the shape of grill-me. Chapter 7 covers the full file layout, subagent wiring, and config; here the point is the behavior, not the plumbing.

--- SKILL.md ---
name: grill-me
description: >
  Interrogate a draft requirement against the existing system
  catalog BEFORE a story is written. Surface ambiguities,
  conflicts with current behavior, missing edge cases, overlap
  with existing features, and impacted entities.

reads:
  - docs/glossary.md
  - docs/rules/**
  - docs/ui/**
  - docs/batch/**
  - docs/integrations/**
  - docs/security/authz.md
  - docs/nfr.md

output:
  A structured list of questions. For each: the question, why it
  matters, the trade-off of each plausible answer, and the cited
  catalog fact that raised it. Never invent a fact; if the catalog
  is silent, say so and flag it as an open question.

Where this leaves us

The Analyst is now concrete: a loop of four skills — grill-me, draft-story, impact-scan, and trace — that turns "better visibility into promotion spend" into PROMO-412, a story with testable criteria and a known blast radius, every claim traceable to the catalog. None of it works without the catalog being good, and none of it stays good unless someone keeps the catalog current after the story ships. That round-trip is the backend — Jira on one side, GitHub docs-as-code on the other — which is next.