ZynoviqAn assessment is a photograph. This is the film.
A capability assessment shows how your AI performed on one day. Two things then start to erode that result — and neither announces itself.
The rules move
Rates change, thresholds change, notifications and circulars are issued. An answer that was correct last quarter becomes wrong without anything in your system changing.
For rule changes specifically, the Regulatory Regression Service watches the changes, re-runs the affected cases on your system and sends an impact note. It can run alongside this programme or on its own.
The system moves
You update a model, change a prompt, add a data source or switch provider. Each change can improve one area and quietly break another, and nothing in a normal release process tells you which.
Both are found the same way: by re-testing.
How the programme works
Your first assessment becomes the baseline. Then, on an agreed cycle:
- Every quarter we re-run your AI against a fresh selection from our certified case pool — including new cases for any rule changes.
- After a major rule change (for example a GST rate revision) we run a targeted re-test on the affected topics.
- After a major change on your side — a new model or a big prompt change — you can trigger an extra run.
- Each cycle you receive the difference: what improved, what got worse, what is newly wrong, and which failures are regressions.
Why the test stays fair every quarter
We draw each run from a much larger hidden pool and rotate the cases, so your AI is never simply re-sitting a paper it has already seen. Cases shown to you with answers are retired for your next run. That is why the shared pool stays with us. Data rights and confidential packs.
What you receive each cycle
| Item | What it tells you |
|---|---|
| Score trend by topic | Is the system getting better or worse, and where |
| Regression list | Things that used to be right and are now wrong |
| Rule-change impact | How your AI handles the latest notifications and circulars |
| New failures explained | Correct answer and law for each |
| Short briefing note | For your risk committee or leadership |
Who this is for
Organisations where the AI does work that matters — customer-facing advice, regulatory output, decisions with money attached — and where being quietly wrong for a quarter is not acceptable. It is not for a pilot. Assess first; move to a programme when the system is load-bearing.
What it is not
This is periodic re-testing against our case pool. It is not a team reviewing your live cases as they arrive — for high-risk live decisions, see the expert review queue.
Where continuous evaluation sits among the packages: packages and how to start.
Continuous AI evaluation — FAQ
Why re-test AI that has already been assessed?+
Because both the rules and the AI change. A GST notification or a model update can make last quarter's correct answer wrong without anyone noticing.
How often are re-tests run?+
Every quarter, after major rule changes, and on request after a major change to your AI.
Will the AI see the same questions each time?+
No. Each run draws fresh cases from a larger private pool, and cases shown to you with answers are retired, so the AI cannot simply learn the test.
What is a regression in AI testing?+
Something the AI used to get right and now gets wrong — usually after an update. Each cycle's report lists them.
Is continuous evaluation a subscription?+
Yes, usually an annual subscription per AI system, quoted, covering quarterly re-tests and re-tests after notified releases; extra releases are priced per run. How we price.
How does this differ from the Regulatory Regression Service?+
Continuous evaluation re-tests on a schedule and after your releases. The Regulatory Regression Service is triggered by the law: when a rule changes, we re-run the affected cases and send an impact note. Many teams combine the two.