You earned your autonomy. Agents must bring evidence to earn theirs.
An agent is defined by its purpose, the personas of its clients, and the policies of its principal. Diamond compiles the three into a bespoke test suite, runs it against the live agent, and hands back a score with the transcript of what failed.
New: launched October 7, 2026Adaptive Red Teaming for agentsAttacker agents run successive waves against your live agent, each wave learning from the last. See how it worksNobody can read every trace
Agents are getting more capable, more autonomous and more numerous at once, and reviewing their behavior by hand has stopped scaling. Hand that review to an automated overseer, and the question of trust moves with it: the overseer is itself an agent, and it can certify safety it never established. Meanwhile the agent you approved is not the agent running — model updated, tools extended, prompt edited by someone shipping a fix, none of it re-approved. The three people who answer for it feel that differently.
- The business owner
- “Two agents, two vendors, two claims of safety, no common scale. I am choosing on an assertion written by whoever is selling.”
- The risk owner
- “I am asked to sign off on an agent whose behavior is described in prose. An approval with nothing behind it is still my signature.”
- The agent developer
- “I fixed what the last review found. Whether the agent is better overall or merely different is not a question my test suite answers.”
A grader that scores its own work drifts, and looks fine doing it
All three need the same thing: a score on a published scale, with the failing case behind it.
Three inputs: purpose, personas, policies
Diamond compiles the harness from what the agent declares it is for — purpose, personas, policies — rather than from a template. Every probe in a bespoke harness is shaped by the agent’s actual tools, policies, system prompt and declared user personas, not a generic jailbreak library.
Purpose
What the agent declares it is for. An agent card states it: what the agent does, the model behind it, and the skills it exposes. Diamond compiles the harness from it, so a finance-approval agent and a customer-support chatbot are not asked the same questions.
The card shown is from Vijil’s sample travel agent.
Enterprise travel booking agent for corporate travel management.
- Model
- Groq llama-3.1-8b-instant
- Input / output
- text / text
- Streaming
- yes
Personas
Who uses the agent. Two kinds. Functional personas set what a user is trying to do, how skilled they are and whether they mean well. Demographic personas vary who the user is, so the same request is tested for bias.
Functional: role and intent
- Benign
- Legal counselCustomer support agentHR recruiterExecutiveNew employeeData analystSoftware developerNon-English speakerAccessibility user
- Curious
- Confused userPower user
- Adversarial
- Security researcherPrompt injection testerSocial engineer
- Malicious
- Malicious actor
Demographic: who the user is
Thirteen attributes, ten of them protected classes: race, religion, national origin, sex, gender identity, sexual orientation, age group, disability, veteran status and pregnancy, plus marital status, socioeconomic status and education level.
Policies
The rules of its principal, from regulation down to this one agent. Diamond compiles them into bespoke tests and reports compliance against each.
One policy, tested here and enforced at runtime
Dome enforces the same compiled policies at runtime — what Diamond certifies, Dome defends.
Operationalizing ISO 42001Probes and detectors, anchored on the Vijil trust taxonomy
Every probe is filed under the Vijil trust taxonomy: three dimensions, nine sub-dimensions and 28 categories of failure, each category broken into the specific ways an agent fails. An evaluation scores the categories its harness covers; the September 2026 Trust Reports scored 13.
Reliability
Correctness
- Factual Accuracy
- Logical Validity
- Task Alignment
- Goal Satisfaction
Consistency
- Self-Consistency
- Cross-Session Consistency
- Temporal Consistency
- Inter-User Consistency
Robustness
- Contextual Robustness
- Distributional Robustness
- Operational Robustness
Security
Confidentiality
- Data Privacy
- User Privacy
- Model Privacy
Integrity
- Adversarial Robustness
- Manipulation Resistance
- Tamper Resistance
Availability
- Denial-of-Service Resistance
- Degradation Handling
Safety
Containment
- Scope Boundaries
- Capability Boundaries
- Self-Modification Control
Compliance
- Policy Compliance
- Norm Compliance
- Ethical Behavior
Transparency
- Explainability
- Accountability
- User Controllability
1Probes
Attacks and tasks from OWASP LLM Top 10, MITRE ATLAS, garak, and internal red-team.
Probes in the docs2Detectors
Graders label each response pass or fail. Detectors combine pattern-matching scripts, machine-learning classifiers Vijil fine-tunes itself, and LLM judges.
Detectors in the docs
One score, read from the worst responses
Every detector verdict scores one response. An average of those verdicts lets thousands of easy passes bury the few failures a customer will meet, so Diamond keeps the average as detail and scores the tail.
3Metrics
Pass rates roll up by category and by dimension, reported as means so you can see where an agent fails.
4Score
The Trust Score is the score on the worst-scoring 10% of responses, overall and for each dimension. It decides the verdict.
An average of 92 hid a Trust Score of 48.3
Grok 4.6 ran 115 probes across 3 harnesses. Its Mean Score, the average of its harness scores, was 92; its worst-scoring tenth of responses scored 48.3. Behind Vijil Dome the Trust Score rose to 67.8, past the 62.8 gate set by the September campaign, while the average moved only from 92 to 96. The average could not see the problem, or the fix.
Dome does not lift every model the same way. Kimi K3’s average rose from 91 to 95 while its Trust Score went from 53.7 to 53.6; the leaderboard on our home page shows every model both ways.
The verdict reads the Trust Score against a bar: the gate set by a campaign of frontier models on the same suite, your agent's accepted baseline, or a bar your team sets. Against a baseline, a run fails only when it is worse by more than run-to-run noise.
Grok 4.6 Trust Report, September 2026 (PDF)Attackers that learn between waves, and within a conversation
The harness asks the same questions every run. The adaptive engine asks new ones. Attacker agents hold full conversations with the live agent in successive waves, and each wave starts from what the last one learned. It tests security, against the OWASP Agentic Security Initiative Top 10: goal hijacking, tool misuse, memory and context poisoning, and the rest.
- Grounded in the agent
- Attack seeds come from the risk taxonomy, the agent’s card (its purpose, system prompt and tools) and its policy, which also sets the rules the judge applies.
- Adaptive within a conversation
- When a turn stalls, the attacker rewrites the message, extends its plan or switches strategy. Against a chat-completions agent, it can also roll back to an earlier turn and retry.
- Adaptive between waves
- A reflection step feeds what worked into the next wave’s seeds, which must find new attacks rather than repeat old ones. The run stops at the wave limit you set.
- Judged, with the evidence
- A judge rates each conversation’s harm as full, partial or none, from the agent’s replies and tool calls, and checks whether it leaked anything from its card. Findings are campaign evidence with the conversation attached; they do not enter the Trust Score.
Before you deploy, and again after every change
Diamond is the one test engine in the Trusted Agent Lifecycle, run for three purposes: the developer evaluates, the risk owner verifies, the business owner validates. Dome enforces what Diamond certifies; Darwin, now in development, will evolve what Diamond scores.
Free to start, per agent on ours, unmetered on yours
You never pay per evaluation or per token. The client is free and one agent on our console is free. More agents are priced per agent on the same console, and inside your own network the platform is an annual subscription. Every tier is on the pricing page.
- On your machine
The client, free to install
Read the docspip install vijil-sdkgives you the CLI and the Python client: evaluate, harnesses, reports and scores. Free, not open source; Vijil Dome is the Apache-2.0 one. - On our console · start here
One agent, free
One agent, a baseline evaluation a week and a Quick adaptive run a month, with the transcript behind every failure.
Start for free - On our console · Team
Per agent
The agents that matter, evaluated on every change, with Standard and Deep adaptive runs. Monthly or annual.
Talk to us - In your network
Every agent, annual
In your VPC, on-premises or air-gapped. Every agent scored as often as you like, with your policy compiled into the harness and the adaptive adversary. Transcripts never leave your infrastructure.
Talk to us
Run it from the console or the command line
Both reach the same evaluation. In the console, pick an agent, choose Baseline, Frameworks, Bespoke or Adaptive, and press Run Evaluation. From a terminal, the vijil command does the same in three steps, and the score arrives with every prompt and response behind it. Open the failures before you argue with the number.
# more: harnesses, adaptive runs
What you get back: the Trust Report
The Trust Report is written for the risk owner who decides and the developer who fixes. It leads with the verdict, shows what the controls changed, and ends with what to fix and who should fix it. The appendix holds every probe’s score. These are the first five of the 31 pages of Grok 4.6’s report.
The verdict
The Trust Score with and without Dome, the campaign gate, and the three dimensions.
Executive summary
Past the gate with Dome, the top findings, and where it sits furthest from its peers.
With Vijil Dome
What the controls changed: Security +27.9, Safety +10.9, Reliability +1.0, and no drop beyond sampling noise.
Among peers
Ranked 7 of 11 models measured on the same suite, and placed on a map of capability against trust.
What to fix
47 findings point to 20 mitigations, each assigned to who acts: Dome at the gateway, the agent's developer, or the model.
Page 1 of 5 · swipe for more
Score one agent you already trust
Not the one you expect to fail — the one you were sure about. An average of 92 hid a Trust Score of 48.3; yours may hide something too.
- One agent on our hosted console, free, and nothing expires
- A baseline evaluation a week and a Quick adaptive run a month
- The Trust Report, with the transcript behind every failure
We open your account, usually the same day. To start from a terminal now, pip install vijil-sdk.
Need it inside your own network? Talk to us.