prototype/README.md

# Slow Triage

A private, local-first issue-export triage prototype. It uses a real Mastra workflow and a fixed open embedding model to suggest issue labels, surface near-duplicates, and make uncertainty visible. Nothing is posted to GitHub. No account, paid API, live repository integration, hosting, or external inference service is required.

**Status:** private preparation only. No repository has been published, no DEV account created, and no competition entry submitted. All included examples and evaluation inputs are synthetic. This prototype was built with AI assistance; any future submission must disclose that accurately.

## What works

1. Load the clearly labeled synthetic demo or import GitHub-shaped JSON.
2. Mastra executes four real stages: validate-export → embed-locally → compare-and-abstain → build-review-queue.
3. Transformers.js executes the pinned quantized MiniLM model locally on CPU. Similarity to labeled reference examples produces suggestions; low scores or small margins enter the review queue.
4. Inspect the matching reference and the cosine similarity. Similarity is **not** a probability of correctness.
5. Explicitly select and optionally override labels. Apply them only to an in-memory copy, undo with a revision/hash check, and export a new file. Originals are never changed.
6. Near-duplicates remain suggestions. Issues are never closed, merged, or sent to GitHub.

## Reproduce

Requires Node.js ≥22.13 and Python 3 with its standard library. Tested on Node 24.19.0 and Python 3.12.14, Linux x64 CPU.

```sh
npm ci --ignore-scripts --no-audit --no-fund --cache .npm-cache
python3 scripts/download_model.py
npm test
npm run test:setup
npm run evaluate
npm start
```

Open http://127.0.0.1:4317 in a browser. The server binds only to loopback and rejects other Host/Origin values. Its page uses no external scripts, fonts, analytics, or requests. Each inference call is limited to 100 issues; the total normalized JSON is limited to 256,000 characters. Server state is volatile and disappears on restart.

**Initial setup needs internet** to obtain ordinary pinned packages from npm and fixed, non-executable model artifacts from Hugging Face. Package install lifecycle scripts are disabled. The model download is bounded to 48 MiB total (the current files total about 23 MiB). Model and dependency downloads have no API charges. Existing local compute is assumed. There is no paid backend.

After setup, model loading explicitly disallows remote models. The CLI and evaluation additionally replace JavaScript fetch with a throwing function, so attempted fetch calls fail. This is a reproducible application-level network check, not an OS network sandbox or a blanket security certification.

```sh
node scripts/triage.mjs data/sample.json evidence/demo-triage.json
npm run demo:cli
```

A socket-free, repeatable demonstration is available with `npm run demo:cli`. It invokes the actual triage CLI twice, checks successful execution of all four Mastra stages, checks identical decisions/scores, and confirms the original synthetic file is unchanged. The two result files and a factual report are saved in `evidence/cli-demo/`. A separate `npm run demo` exercises the real loopback HTTP sample/triage/apply/undo API and writes `evidence/end-to-end-demo.json`; it does not record a browser or video.

A clean machine may require platform-specific optional dependencies. Do not omit optional packages: Transformers.js imports Sharp even for this text-only task. No GPU is required.

The model setup script verifies fixed SHA-256 hashes and byte counts for both newly downloaded and existing files. A corrupt cache fails closed; it cannot redefine the expected hashes. `npm run test:setup` runs four network-free setup-integrity checks using synthetic response bytes. `npm run verify:model` independently checks the recorded model manifest.

## Input format

Either an array of issues or an object containing `issues`. Fields:

- `id`: unique positive safe integer. `number` is accepted when `id` is absent.
- `title`: nonempty string up to 300 characters.
- `body`: string up to 12,000 characters; null or missing becomes empty text.
- `labels`: optional array of strings or GitHub-shaped `{name: "label"}` objects.

Only these fields are retained. Existing labels never drive inference. Empty arrays, duplicate IDs, oversized content and malformed types are rejected. Issue text is treated as data, never as executable instructions. This prototype intentionally rejects string IDs and does not import comments, attachments, repository permissions, or personal data.

## Model and decision policy

- `Xenova/all-MiniLM-L6-v2`, pinned revision `751bff37182d3f1213fa05d7196b954e230abad9`
- Quantized q8 ONNX, 384-dimensional normalized embeddings, CPU
- Mean pooling; maximum 256 wordpieces, so later body text can be truncated
- 24 original synthetic references: six each for bug, feature, documentation and question
- Each label is scored by the mean similarity of its two closest references
- Review if the best score is below 0.40 or the best/second-best margin is below 0.04
- Suggest an earlier issue as a duplicate candidate if similarity is at least 0.83; always expose the nearest other item for inspection

These thresholds and references were frozen before the first successful inference/evaluation. They are conservative heuristics, not calibrated confidence. See `evidence/pre-evaluation-freeze.sha256`. There was no model fine-tuning or label training. The existing pre-trained open model is central to semantic comparison; Mastra actually orchestrates the stages.

## Measured developer-fixture results

The included evaluation is a small, fixed, developer-authored synthetic fixture. It is not a representative benchmark and does not establish real-world performance or time saved. A separately authored reviewer fixture was frozen and tested independently; its results are recorded below.

- Classification: 9/20 exact label-or-abstention results correct (45%); keyword baseline 18/20 (90%)
- Model answered 5/20; all five answered items were correct. It sent 15/20 to review, including all four intentionally unknown examples
- Duplicate suggestions: 2 true positives, 0 false positives, 1 missed true pair across six pairs (precision 2/2; recall 2/3)
- First evaluation run: 215 ms including model/reference load; repeated same-process batch: 46 ms; process RSS approximately 246 MiB
- Repeated labels, review decisions and similarity scores matched within 1e-7; fetch disabled throughout evaluation
- 10-issue demo: 6 above-threshold suggestions, 4 review cases, 1 duplicate candidate in the first recorded 236 ms run

These are one observed run on this environment, not guaranteed speed or memory requirements. The conservative model policy underperforms the simple keyword baseline on overall accuracy. Do not market it as a better classifier. The working demonstration is explainable semantic matching with review-first, reversible output and no external inference.

`evidence/evaluation.json` contains individual predictions, confusion matrices, pair scores, settings and limitations. `npm run evaluate` regenerates actual results, so timings will vary. The freeze/checksum record helps disclose if future implementations change.

## Independent reviewer fixture

A second worker wrote and froze separate synthetic inputs without seeing predictions or modifying the model policy. Its original fixture and per-row results are preserved under the adjacent `review/` directory. Results are still synthetic and are not real-world efficacy evidence.

- 40 classification cases: 15/40 exact label-or-abstention outcomes correct (37.5%), against 24/40 (60%) for its independently fixed keyword baseline
- 8/8 answered cases correct at 20% coverage; 31 abstentions and one rejected blank-title input
- Duplicate detection: 7 true positive pairs, 1 false positive, 1 missed pair; precision and recall both 7/8 (87.5%)
- Nearest-candidate retrieval: 15/16 positive queries matched the correct issue
- Actual 384-dimensional model output, repeatability verified, zero attempted network calls in the reviewer’s blocked-network process

**Interpretation:** a conservative semantic review and duplicate-discovery aid, not a proven classifier improvement. Both evaluations favor the keyword baseline for broad classification accuracy. Never omit this comparison from a public write-up.

## Safety and limitations

- All recommendations require review before export. No automatic GitHub actions exist.
- Semantic similarity does not prove duplication or severity. No priority/severity or sensitive traits are inferred.
- English short-text model; long, multilingual, ambiguous or domain-specific issues can be wrong.
- Synthetic references and test examples are small and developer-created; any benefit claims need real-world independent evaluation.
- The app is a single-user local prototype, not a multi-user production service. Do not expose the port publicly.
- Current working copies and undo records are memory-only. Reloading the page loses UI revision controls; restarting the server loses all working state.
- Import only data you are authorized to process. This demonstration uses synthetic data exclusively.

## Verification status

- Passed: 14 shipped Node tests; four model-setup integrity tests; verification of all seven pinned model files; real CLI inference and loopback HTTP apply/undo.
- Additional retained checks: 39 independent HTTP/adversarial cases; 11 matching UI/backend schema cases; nine source/simulated-DOM flows; 27 CSV cell checks.
- The original UI import-schema mismatch was fixed and the shipped validator retested. No model references or thresholds were tuned after viewing the held-out results.
- Real rendered-browser E2E, layout, keyboard behavior, browser payload execution and actual browser downloads remain unverified. Standalone Chromium could not create a socket in this sandbox, and the separate cloud browser could not reach this executor's loopback service. No screenshot or video is claimed. The service has not been exposed publicly.
- Preserved developer/reviewer metrics are original observations. Regenerated evaluation timings may differ; a fresh process can still benefit from filesystem caches.

## License and release gate

See THIRD_PARTY_NOTICES.md, package-lock.json and models/manifest.json. The local original code/fixtures are prepared for an Apache-2.0 release, pending owner approval. The model and runtime components are open source under their recorded licenses.

Before any contest entry: obtain the owner's public-release/account/legal approvals, confirm eligibility and the official template/category requirements, replace every placeholder with actual public code/demo links, disclose AI assistance, and submit before the verified deadline. Prize eligibility and awards are not guaranteed by this prototype.