Model quality and grounding

This page answers the question a compliance engineer should ask before pointing an agent at any model API: why would this one be good at the job? It documents what serves the aliases, how framework knowledge reaches the model, the evaluations behind our model choices, and the limits of every claim made here.

The lineup

The API serves a small set of aliases. Behind them:

• The standard tier (isms-fast, isms-thinking, and their -eu variants) runs GLM 5.2, a strong open-weights model, at a 1M-token context window. The global path and the EU path run the same model class; the difference is processing region, not capability.
• The bulk lane (isms-mini) runs Mistral Small, a smaller model, for high-volume work that does not need deep analysis: formatting, extraction, classification, JSON normalization.

Exact upstream providers can change. The aliases are the stable contract for your integration, and a change to the model behind an alias follows the upgrade terms.

How knowledge reaches the model

The API is not a fine-tuned model, and the compliance knowledge is not in the weights. The mechanism is injection at inference time:

1. Your request arrives as a normal OpenAI-compatible chat completion.
2. The server detects named frameworks in your messages (auto mode), or takes the exact modules you pin with ismscopilot: { "frameworks": [...] }.
3. The curated reference module for each selected framework is assembled into the prompt, along with the published server prompt and the Reference Integrity policy.
4. The response tells you what ran: the x-isms-frameworks header and the ismscopilot response object list every injected module with a labelled token estimate.

This is deliberate. Knowledge in weights cannot be audited, dated, or corrected per response. Knowledge in the prompt can be: every module carries a verification stamp, the catalog is queryable, and the injected bytes are billed to you as ordinary input tokens you can see in usage.prompt_tokens.

The evidence on file

Every quality claim we make internally is backed by a dated, method-recorded evaluation. These are our own internal evals, with the sample sizes we actually ran; read the caveats as part of the results.

Model selection: GLM 5.2 vs Mistral Small (2026-07-03)

The dual-mode evaluation that chose GLM 5.2 for the standard tier. Three compliance tasks (ISO 27001 gap analysis, SOC 2 vendor-incident mapping, a GDPR DPIA), four arms, three samples each: 36 generations, blind-judged by a frontier model against a five-criterion 0-to-2 rubric. Both model arms ran with knowledge injection.

Caveats as recorded in the report: single-day run, N=9 per arm; do not read per-task differences as significant; latency was measured under eval conditions, not production, and we do not quote it publicly.

Citation traps (2026-08-17)

Five prompt traps designed to catch confident invention of compliance facts: exclusion scoping, research-and-development scope, key-person dependency, log retention, and a benign rename that should not trigger anything. GLM 5.2 (thinking) passed 5 of 5. Mistral Small passed 4 of 5, failing the log-retention trap. This is the faithful rerun; an earlier scoring attempt with a cruder scorer was marked non-decisive in our design docs and does not count as evidence.

Hostile-prompt gates (2026-08-17)

The same GLM 5.2 weights serve our heyGRC review product, whose launch gate includes hostile and injection-resistance suites. GLM passed the full gate twice consecutively (core, hostile, baseline, benign, grounding: all green). Mistral Small had previously failed the hostile suite. Corroborating, not independent: same weights, different product harness.

Server prompt discipline (2026-08-02)

The published server prompt shipped only after a pre-registered gate: rules frozen before the run, then 848 calls across the aliases. Final run: 20/20 accuracy on baseline-correct fixtures (no regression from injection), 16/16 correct identity attribution, 24/24 refusal of intellectual-property elicitation. The first run failed one rule and the prompt was iterated, not the rules; the failure is recorded in the results file.

Detection determinism (2026-06-05)

Framework detection in auto mode is a deterministic, tested function, not a model call. Its eval gate is 98%+ precision/recall on the fixture set, and it runs in CI on every change.

Verification you can check

You do not have to trust this page. Everything it describes is observable from a single API call:

• GET /v1/frameworks lists the live catalog (101 modules observed on 2026-08-26; the live endpoint is authoritative, not this number).
• GET /v1/frameworks/changelog records registry additions, corrections, and verification refreshes with dates.
• The system prompt is published verbatim, and x-isms-policy-version names the version that served each request.
• x-isms-frameworks tells you what was injected into your own request, every time.
• The zero-data-retention posture is documented in Zero Data Retention.

What we do not claim

• We do not claim quality superiority over Claude or any frontier model. No head-to-head evaluation against raw frontier APIs exists on file yet. A mode-aware benchmark (our aliases versus raw frontier models, publishing losses alongside wins) is in design; until it runs, no such claim is made anywhere in our copy.
• We do not claim the knowledge is the text of the standard. Modules are curated references assembled by us; the copyrighted standards themselves are licensed by you if you need the source text.
• We do not claim detection is exhaustive. In auto mode detection is name-level: a bare control number with no framework name can inject nothing. Pin the framework when you know it.
• We do not quote latency numbers publicly. Eval-condition measurements are not production measurements.
• Nothing here is an audit opinion or legal advice.

Where this leaves the choice

For compliance work, the case is: a strong open-weights model, grounded on request in a maintained and verifiable reference, with per-response disclosure of what was injected, at a price point built for agent-scale throughput. For frontier-hard reasoning, multimodal input, or tool-calling agents, a frontier API may still be the right tool, and nothing here forbids a hybrid architecture that uses both. See Models and regions for the alias table and Framework knowledge for the injection mechanics.