Fund the AI work that pays back, not the best demo.

workflow-radar turns discovery-interview estimates into a ranked AI roadmap: explainable friction and AI-suitability scores, a delivery pattern with guardrails for each step, and ROI as a P10/P50/P90 range from a seeded Monte Carlo simulation.

Synthetic data. Every figure on this site is computed at build time by running the tool on the 4 bundled synthetic example workflows. The numbers illustrate the method; they are not measurements of any real organization.

Quickstart

Requires Node.js 20 or later. No API keys, no network access after npm ci.

git clone https://github.com/seanmcrae/workflow-radar.git
cd workflow-radar
npm ci
npm run demo   # build, then audit examples/*.yaml into report/

Or run the commands directly after npm run build:

node dist/cli.js score examples/invoice.yaml --explain
node dist/cli.js report examples/*.yaml --out report
node dist/cli.js parse examples/notes/invoice-interview.txt --out draft.yaml

Features

• Validated workflow modelYAML or JSON, zod-validated, three-point ranges for every uncertain input, field-qualified errors.
• Explainable scoresFriction and AI suitability are weighted sums; every factor's weight, points, and reason are in the report.
• Pattern rules with guardrailsCopilot, automation with review, or agent, gated by judgment, regulation, and task type before any threshold.
• Monte Carlo ROISeeded triangular sampling gives P10/P50/P90 hours, net savings, payback, and P(first-year net > 0).
• SensitivityOne-at-a-time tornado analysis shows which input to go and measure before funding.
• RoadmapValue-versus-effort quadrants plus a payback gate produce a ranked Now / Next / Later / Park plan.
• Interview notes parserOffline heuristic parser by default; Anthropic and OpenAI adapters behind one interface, opt-in by key.
• Portable reportsMarkdown, self-contained HTML, SVG, and JSON. Same inputs and seed give byte-identical output.

Results on the bundled synthetic workflows

Default config, 5,000 Monte Carlo iterations per opportunity, seed 42. Regenerated on every site build from examples/*.yaml.

Assessed 23 human steps across 4 workflows (4 synthetic); 21 recommended, 2 not recommended.

Phase   #  Opportunity                                                                 Hours/mo P50  1st-yr net P50  Payback P50
-----  --  --------------------------------------------------------------------------  ------------  --------------  -----------
Now     1  Customer support triage: Draft first response                                        837        $333,756       1.2 mo
Now     2  Customer support triage: Read new ticket and tag product area and intent             359        $114,925       3.0 mo
Now     3  Sales proposal drafting: Draft proposal narrative and scope                          105         $72,012       4.2 mo
Now     4  Invoice processing: Key invoice header and line items into the ERP                   176         $66,135       4.5 mo
Now     5  Customer support triage: Escalate to Tier 2 with a case summary                      208         $46,037       5.4 mo
Next    6  Customer support triage: Look up customer account and order history                  283         $80,893       3.9 mo
Later   7  Sales proposal drafting: Answer customer security questionnaire                       31         $18,327       5.1 mo
Later   8  Customer support triage: Set priority and route to the right queue                   131         $10,437       9.4 mo
Later   9  Sales proposal drafting: Map customer requirements to product capabilities            50          $8,549       9.8 mo
Later  10  Invoice processing: Resolve three-way match exceptions                                37          $7,480       7.7 mo
Later  11  Sales proposal drafting: Build pricing and approve discounts                          14            -$31      12.0 mo
Later  12  Customer support triage: Decide on refund or goodwill credit                          32         -$2,138      14.2 mo
Later  13  Invoice processing: Assign GL codes and cost centers                                  64         -$6,822      14.6 mo
Park   14  Invoice processing: Approve invoices above the $5,000 threshold                       15         -$7,202      26.0 mo
Park   15  Employee onboarding: Collect and verify new-hire documents                             9        -$11,339      76.2 mo
Park   16  Employee onboarding: Enter new hire into payroll                                       4        -$14,436   no payback
Park   17  Invoice processing: Download invoice attachments from the AP inbox                    37        -$24,875      33.9 mo
Park   18  Employee onboarding: Answer policy and benefits questions                             30        -$30,957      58.5 mo
Park   19  Sales proposal drafting: Summarize discovery call notes                               12        -$35,001     116.7 mo
Park   20  Employee onboarding: Draft a personalized first-week plan                             19        -$37,344     > 120 mo
Park   21  Employee onboarding: Request accounts and access for each system                      22       -$114,258   no payback

Wrote site/report/report.md, site/report/report.html, site/report/quadrant.svg, site/report/audit.json

Invoice processing (Accounts Payable) [synthetic data]
Volume 1,800 (1,500–2,200) / month · workflow friction 16.8

#  Step                                            Task type       Friction  Suitability  Pattern                 Hours/mo P50  Payback P50
-  ----------------------------------------------  --------------  --------  -----------  ----------------------  ------------  -----------
1  Download invoice attachments from the AP inbox  extraction             3           87  Automation with review            37      33.9 mo
2  Key invoice header and line items into the ERP  extraction            17           84  Automation with review           176       4.5 mo
3  Assign GL codes and cost centers                classification        10           84  Automation with review            64      14.6 mo
4  Resolve three-way match exceptions              decision              35           55  Copilot                           37       7.7 mo
5  Approve invoices above the $5,000 threshold     decision              30           50  Copilot                           15      26.0 mo
6  Scheduled payment run                           -                      -            -  system step (skipped)              -            -

Sample report

The full self-contained HTML report the report command writes for the synthetic examples, including per-factor score breakdowns, ROI tables, and tornado sensitivity for every opportunity. Open it on its own page, or get the Markdown and machine-readable audit.json.

How evaluation works

Each human step is scored independently, then ranked:

1. Friction (0-100) is a weighted average of hands-on minutes, rework rate, handoffs, and wait time, each normalized against a saturation point (60 minutes, 25% rework, 4 handoffs, 1,440 minutes by default).
2. AI suitability (0-100) is a weighted average of task-type fit (extraction and classification high, decisions low), data readiness, judgment (inverted), and regulatory sensitivity (inverted).
3. Pattern comes from ordered hard rules first (no AI-shaped task, or suitability below the minimum, means not recommended; high regulation, high judgment, or a decision task caps the step at copilot), then suitability thresholds.
4. ROI samples every three-point input from a triangular distribution: hours saved = volume x share of runs x minutes x (1 + rework rate) / 60 x automation fraction, and first-year net = 12 x (hours saved x loaded hourly cost - monthly run cost) - implementation cost. Each opportunity gets its own seeded stream, so adding a workflow never changes another's numbers.
5. Prioritization maps P50 annual net savings to a value score, combines pattern, data readiness, regulation, and system count into an effort score, places each step in a quadrant, and parks anything whose median payback exceeds the horizon.

How the tool checks itself, in this repository:

• Scoring math is pinned by unit tests with hand-computed expected values (friction, suitability, effort, quadrant edges, value cap).
• Monte Carlo tests cover determinism per seed, percentile ordering, degenerate ranges collapsing to the point estimate, and the triangular sampler's mean.
• The heuristic parser is evaluated field by field against synthetic interview notes and must produce a schema-valid workflow.
• Hosted LLM adapters are tested against recorded-shape responses through an injected fetch, including HTTP errors and malformed content.
• CLI integration tests run score, report, and parse in-process and require byte-identical reports for a fixed seed; the site build test requires the embedded report to equal the committed sample.
• tests/readme-numbers.test.ts recomputes the Numbers card, including the friction-only baseline, and the figures in Where it fails, so the README cannot drift from the code.

How a team would measure it in use (estimate accuracy, interval calibration, draft acceptance rate) is defined in docs/PRODUCT.md.

Architecture

flowchart LR
    notes["Interview notes (.txt)"] --> parse["parse: NotesProvider<br/>mock | anthropic | openai"]
    parse --> draft["Draft workflow YAML"]
    yaml["Workflow YAML / JSON"] --> load["zod schema validation"]
    draft --> load
    cfg["config/default.yaml<br/>(weights, thresholds, cost bands)"] --> score
    load --> score["Scoring<br/>friction + AI suitability"]
    score --> pattern["Pattern rules + guardrails"]
    pattern --> roi["ROI model<br/>Monte Carlo P10/P50/P90<br/>tornado sensitivity"]
    roi --> prio["Value vs effort quadrant<br/>payback gate, ranking"]
    prio --> out["Report<br/>Markdown, HTML, SVG, JSON"]

Design decisions

• Transparent weighted sums over a learned model. There is no labelled dataset of "AI projects that paid back" to train on, and the people who act on an audit need to argue with it. Every score is sum(weight x normalized factor), the weights are in YAML, and each factor's points and a one-line reason are in the report.
• Hard rules before thresholds. Pattern selection checks task type, regulatory sensitivity, judgment, and decision tasks before it looks at the suitability score, so a high score cannot buy autonomy on a regulated or high-judgment step. Agents additionally require low risk, low judgment, at least two systems, and at least one handoff.
• Ranges, not point estimates. Volume, duration, rework, hourly cost, automation fraction, build cost, and run cost are all three-point ranges sampled as triangular distributions. The report shows P10/P50/P90 and the probability that first-year net is positive, so a fragile opportunity looks fragile.
• Reproducible randomness. Simulations use a seeded mulberry32 PRNG. Each opportunity's seed is the run seed XOR a hash of its id, so results do not change when workflows are added or reordered. Same inputs and seed produce byte-identical reports.
• Rework is labor. Hours saved use minutes x (1 + rework rate), because a step redone 12% of the time costs 1.12x its nominal time.
• Wait time is friction, not cost. Queue time drives the friction score (it is what people complain about) but not labor savings, because nobody is paid to wait in a queue.
• Value is net of run cost; build cost is effort. The value axis uses P50 annual savings after model and maintenance costs; one-time build cost shows up in effort and payback. A payback gate then parks anything whose median payback exceeds the configured horizon, whatever its grid position.
• LLM output is untrusted input. The parse providers return a raw object that goes through the same zod schema as hand-written YAML, the prompt embeds the schema generated by z.toJSONSchema, and the offline heuristic parser is the default so nothing requires an API key.

Data

All bundled data is synthetic and labelled as such in each file (synthetic: true plus a header comment):

• examples/invoice.yaml, support-triage.yaml, sales-proposal.yaml, employee-onboarding.yaml: invented workflows with plausible but made-up volumes, durations, and costs.
• examples/notes/invoice-interview.txt: invented interview notes for the parser.

No external datasets are downloaded or required. The data and code are MIT-licensed.

Limitations

• Inputs are estimates from interviews. The model is only as good as the minutes and volumes people report; the ranges help, but systematic bias (everyone underestimating rework) is not corrected.
• Inputs are sampled independently. Real volume and duration often correlate, which would widen the tails. Portfolio totals in the report are sums of per-opportunity medians, not a percentile of the portfolio.
• Opportunities are per step. Automating two adjacent steps together usually shares build cost; the model charges each step its own band.
• Automation fractions and cost bands are defaults, not benchmarks. They are deliberately round numbers meant to be replaced with an organization's own figures.
• The task-type classifier is keyword-based. It explains its matches and can be overridden per step with taskType, but it will misread unusual phrasing.
• Labor savings are not cash savings. Hours freed only become money if the capacity is redeployed or hiring is avoided; the report labels figures as savings, not budget cuts.