Version: 1.0
Published: July 2026
Issued by: AIOBES Governance Group
Status: Final Specification
Identifier: AIOBES CAS 1.0
This document may be freely referenced, cited and integrated. Modification of AIOBES content is not permitted.
Most corporate AI governance operates without behavioural evidence. Any governance framework, tool or process that does not include a defined methodology for evaluating, assessing and recording operational AI behaviour is non-compliant and exposes the organisation to legal, financial and reputational risk. Governance that cannot demonstrate how an AI system behaves under real operational conditions cannot satisfy regulatory, audit or assurance requirements.
The purpose of AIOBES CAS 1.0 is to define the mandatory behavioural evaluation layer required for corporate adoption of AIOBES. This specification establishes how organisations must integrate behavioural evaluation into existing governance, assurance and development workflows. It sets out the requirements for adopting the AIOBES behavioural evaluation methodologies, including LLM Inquisitor for long form workflow behavioural evaluation and Vectored Conversational AI Testing for the evaluation of conversational interaction. CAS 1.0 provides the operational foundation needed to produce behavioural evidence that aligns with regulatory frameworks such as the EU AI Act, ISO 42001 and NIST.
This specification covers behavioural evaluation requirements, evidence production obligations, integration with existing governance frameworks, operational adoption processes, corporate responsibilities and conformance demonstration. CAS 1.0 applies to all AI systems used in real tasks, workflows and interactions where behavioural assurance is required for governance, compliance or risk management purposes. It defines how behavioural evidence must be generated, documented and maintained to support organisational accountability.
This specification is intended for governance leads, compliance officers, risk and assurance functions, product owners, engineering teams, software developers, data scientists, quality assurance teams and any other corporate stakeholders responsible for the safe, compliant and accountable operation of AI systems.
Many corporate AI governance programmes rely on policy frameworks, dashboards and compliance tools that do not incorporate any behavioural evaluation methodology. These systems track documentation, metadata and policy alignment, but they do not assess how an AI system behaves when performing real tasks or interacting with users. As a result, organisations operate governance structures that cannot produce behavioural evidence and therefore cannot demonstrate operational accountability.
Governance tools that lack behavioural evaluation produce non evidentiary governance. They record policies, risk categories and control mappings, but they do not generate evidence of actual system behaviour. Without behavioural evidence, organisations cannot verify whether an AI system behaves consistently, safely or in accordance with regulatory expectations. This creates a structural compliance failure, as regulators require demonstrable behavioural assurance rather than policy declarations.
Regulatory frameworks such as the EU AI Act, ISO 42001 and NIST require organisations to understand and document how AI systems behave under operational conditions. behaviour-blind governance cannot meet these requirements because it does not evaluate or record behavioural outputs. Organisations that rely solely on dashboards, policy templates or governance portals cannot satisfy regulatory obligations without a behavioural evaluation methodology.
Regulatory frameworks such as the EU AI Act, ISO 42001 and NIST require organisations to understand and document how AI systems behave under operational conditions. These frameworks expect evidence of behavioural stability, alignment, continuity, coherence and integrity. Governance systems that do not evaluate operational behaviour cannot meet these requirements because they do not observe or record how an AI system performs during real tasks, workflows or interactions. Without behavioural evidence, organisations cannot demonstrate compliance with regulatory obligations relating to safety, transparency, misuse prevention or post market monitoring.
When behavioural evaluation is absent, organisations are exposed to regulatory, legal, operational and reputational risk. The absence of behavioural evidence creates the following categories of exposure:
Regulatory exposure Organisations cannot demonstrate compliance with frameworks such as the EU AI Act, ISO 42001 and NIST because they cannot provide evidence of behavioural stability, alignment, continuity, coherence or integrity during real operational tasks.
Legal exposure Without behavioural evidence, organisations cannot defend system behaviour during investigations, disputes or litigation. They cannot demonstrate due diligence or provide documented behavioural records to support legal positions.
Operational exposure Behavioural failures such as instability, misalignment, hallucination, contamination, workflow collapse and incorrect guardrail behaviour cannot be detected or documented by governance tools that do not evaluate behaviour. This results in uncontrolled operational failure modes.
Assurance exposure Internal and external assurance functions cannot verify that AI systems behave consistently or safely. Without behavioural evidence, assurance processes cannot validate system performance or confirm compliance with organisational controls.
Reputational exposure Organisations cannot provide stakeholders with credible behavioural assurance. Behavioural failures that occur without evaluation or documentation undermine trust in the organisation’s governance, risk management and AI deployment practices.
These exposures persist regardless of the apparent maturity of governance tooling, because governance maturity without behavioural evidence does not constitute compliance.
Regulators, auditors and legal frameworks require organisations to demonstrate how AI systems behave under operational conditions. Policy declarations, documentation and dashboard outputs do not satisfy this requirement because they do not provide evidence of actual system behaviour. Regulatory frameworks such as the EU AI Act, ISO 42001 and NIST expect organisations to produce behavioural evidence that shows stability, alignment, continuity, coherence and integrity during real tasks, workflows and interactions.
Policy based governance establishes intent but does not verify operational behaviour. Dashboards and governance portals can record controls, risk categories and compliance mappings, but they cannot demonstrate how an AI system behaves when performing real work. Without behavioural evaluation, governance systems cannot detect behavioural failures, cannot validate operational performance and cannot provide assurance that deployed AI systems meet regulatory expectations.
Behavioural evidence is the foundation of defensible governance because it provides verifiable records of how an AI system behaves in practice. Organisations require behavioural evidence to:
• demonstrate compliance with regulatory obligations
• validate operational behaviour against organisational controls
• support assurance functions with verifiable behavioural records
• defend system behaviour during audits, investigations or legal proceedings
• provide stakeholders with credible assurance of safe and consistent operation
Behavioural evaluation enables organisations to move from declarative governance to evidence based governance, ensuring that compliance claims are supported by documented behavioural performance.
Behavioural evidence must be incorporated into governance, assurance and risk management processes. This includes:
• evaluating AI systems during real tasks
• recording behavioural outputs
• identifying behavioural failure modes
• maintaining behavioural evidence as part of compliance documentation
Integrating behavioural evaluation ensures that governance frameworks are aligned with regulatory expectations and capable of demonstrating operational accountability.
AIOBES defines formal behavioural evaluation methodologies that enable organisations to assess how AI systems perform real work under realistic operational conditions. These methodologies provide structured, reproducible evaluation frameworks that expose behavioural failure modes, document operational behaviour and produce behavioural evidence suitable for regulatory, assurance and governance requirements. They replace informal prompt testing, ad hoc evaluation and dashboard based governance with formal behavioural assessment aligned to regulatory expectations.
LLM Inquisitor is a formal behavioural evaluation methodology designed to assess how AI systems perform real work under realistic operational conditions. It provides a structured, scenario based framework for evaluating operational tasks, multi step reasoning, workflow execution and complex decision sequences. The methodology exposes behavioural failure modes, identifies behavioural drift, reveals misunderstanding, hallucination, data leakage and workflow instability, and documents how AI systems behave when placed under authentic operational stress. LLM Inquisitor produces behavioural evidence suitable for regulatory, assurance and governance requirements, enabling organisations to understand system behaviour, map risks and validate operational stability across real workloads.
Vectored Conversational AI Testing is a formal behavioural evaluation methodology designed specifically for real conversational interactions. It provides a structured, repeatable and auditable framework for assessing how AI systems behave across full conversational arcs rather than isolated prompts. The methodology evaluates multi turn dialogue, conversational adaptation, behavioural drift, recovery, escalation and failure under realistic interaction conditions.
VCAIT wraps natural conversation in a disciplined testing structure that reveals behavioural vectors, conversational responses under pressure and emergent behaviours that do not appear in single prompt or checklist based testing. It exposes misunderstanding, misalignment, instability, guardrail failures and other behavioural modes that arise only during extended interaction with users.
The methodology follows a defined workflow that includes preparation, vector construction, conversational execution, behavioural observation and structured analysis. This workflow enables organisations to evaluate conversational AI systems with reproducible conditions and verifiable behavioural evidence.
The methodology provides reproducible evaluation conditions that generate verifiable behavioural evidence for conversational AI systems.
AIOBES behavioural evaluation methodologies must meet the following requirements:
• Evidentiary structure Behavioural evidence must be recorded in a structured format that supports audit, assurance and regulatory review.
• Reproducibility Evaluations must be reproducible under equivalent conditions to ensure consistent behavioural evidence.
• Behavioural domain coverage Evaluations must cover operational workflows, conversational interactions, multi-step reasoning, agentic behaviour and guardrail behaviour.
• Operational stability assessment Methodologies must identify behavioural failure modes, including instability, misalignment, hallucination, contamination, workflow collapse and incorrect guardrail behaviour.
These requirements ensure that behavioural evidence produced by AIOBES methodologies is suitable for governance, compliance and assurance processes.
Governance dashboards, policy portals and governance suites cannot generate behavioural evidence because they do not evaluate operational behaviour. These tools record policies, controls, documentation and metadata, but they cannot observe or assess how an AI system behaves during real tasks or interactions. Without behavioural evaluation, governance tools cannot satisfy regulatory obligations relating to behavioural stability, alignment, continuity, coherence or integrity. AIOBES behavioural evaluation methodologies are required to produce the behavioural evidence necessary for defensible governance and regulatory compliance.
AIOBES CAS 1.0 requires organisations to reference the full methodology specifications when conducting behavioural evaluations. The complete operational grade methodology manuals are published on Leanpub and provide the authoritative definitions, workflows and requirements for behavioural evaluation.
• LLM Inquisitor Methodology
Published on Leanpub: https://leanpub.com/llm-inquisitor
• Vectored Conversational AI Testing (VCAIT) Methodology
Published on Leanpub: https://leanpub.com/conversational-ai-testing
These publications constitute the current operational specifications for AIOBES behavioural evaluation. Organisations must ensure that the methodology versions they reference during evaluation correspond to the manuals in use at the time of assessment. This supports methodological accuracy, reproducibility and audit readiness.
Corporate governance systems record policies, controls, documentation and metadata, but they do not evaluate how AI systems behave under operational conditions. They cannot observe multi step reasoning, workflow execution, conversational adaptation, behavioural drift or failure modes. This creates a structural gap between governance intent and operational reality. AIOBES provides the behavioural evaluation layer required to close this gap by producing evidence of real behaviour during authentic tasks and interactions.
Many organisations rely on outsourced prompt based testing services that provide single prompt evaluations, static checklists or narrow benchmarks. These approaches do not capture operational behaviour, do not reveal behavioural drift, do not expose failure modes and do not evaluate real workloads. They cannot demonstrate compliance with regulatory requirements that mandate behavioural evidence. As a result, organisations often pay for testing that does not assess the behaviour of the deployed system and cannot support governance, assurance or regulatory obligations.
Prompt mill testing evaluates isolated prompts rather than operational behaviour. It cannot assess:
• multi turn conversational behaviour
• workflow execution and multi-step reasoning
• behavioural drift across extended interactions
• recovery, escalation or failure under realistic conditions
• guardrail behaviour during real tasks
• emergent responses triggered by conversational pressure or operational complexity
Because these behaviours only appear during real work, prompt mill testing cannot detect them and cannot produce behavioural evidence. This makes it unsuitable for governance, assurance or compliance. Many organisations incorrectly believe they are satisfying their behavioural evaluation obligations by outsourcing prompt based testing. Despite the high fees charged for these services, the outputs do not assess operational behaviour and therefore provide no compliance value.
AIOBES provides formal behavioural evaluation methodologies that assess AI systems under realistic operational conditions. These methodologies produce structured, reproducible and auditable behavioural evidence that supports regulatory, assurance and governance requirements. AIOBES enables organisations to understand system behaviour, identify failure modes, validate operational stability and demonstrate compliance with behavioural obligations defined in regulatory frameworks.
Regulators, auditors and legal frameworks require behavioural evidence, not policy declarations or dashboard outputs. Governance tools cannot generate this evidence because they do not evaluate operational behaviour. AIOBES fills this gap by providing the behavioural evaluation methodologies necessary to produce defensible behavioural evidence for operational AI systems.
AIOBES CAS 1.0 integrates into existing corporate environments without requiring changes to system architecture, application design or deployment pipelines. Organisations can apply AIOBES behavioural evaluation methodologies directly to current AI workloads, workflows and interaction patterns. The methodologies operate independently of model architecture, hosting environment or vendor tooling, enabling immediate adoption across diverse technical landscapes.
AIOBES does not require organisations to restructure governance frameworks, redefine control libraries or replace existing governance processes. It adds the behavioural evaluation layer that governance systems currently lack. Existing governance artefacts, policies and controls remain in place, while AIOBES provides the behavioural evidence required to make those artefacts operationally meaningful and compliant with regulatory expectations.
AIOBES methodologies do not replace governance dashboards, policy portals or compliance suites. They complement these tools by supplying the behavioural evidence that those systems cannot generate. Organisations continue using their existing governance tooling while incorporating AIOBES behavioural evaluation outputs into assurance, compliance and risk management processes.
Regulatory frameworks require behavioural evidence, making behavioural evaluation a mandatory component of AI governance. AIOBES provides this requirement in a lightweight operational form that does not disrupt workflows or impose additional architectural burden. The methodologies integrate into existing development, testing and assurance processes with minimal overhead, providing a practical path to regulatory compliance.
AIOBES CAS 1.0 fills the behavioural evidence gap that exists in corporate governance systems. It enables organisations to evaluate real behaviour, identify failure modes, validate operational stability and demonstrate compliance without altering their governance structure or replacing their tooling. This seamless adoption model ensures that behavioural evaluation becomes a standard part of corporate AI governance without operational disruption.
ISO 42001 establishes management system requirements for responsible AI but does not define how organisations should evaluate operational behaviour. AIOBES provides the behavioural evaluation methodologies needed to generate the behavioural evidence that ISO 42001 assumes. This enables organisations to demonstrate operational alignment, behavioural stability and compliance with ISO 42001 controls using verifiable behavioural records.
The NIST AI RMF identifies behavioural risks such as instability, misalignment, hallucination and drift, but it does not specify how these behaviours should be evaluated. AIOBES supplies the structured behavioural evaluation layer required to observe, document and validate these behaviours under realistic operational conditions. This strengthens risk identification, risk analysis and assurance activities within the NIST framework.
Internal governance systems typically rely on policies, controls, documentation and dashboard outputs. These systems do not evaluate operational behaviour and therefore cannot detect behavioural failure modes or validate behavioural stability. AIOBES integrates directly into existing governance processes by providing behavioural evidence that makes internal controls operationally meaningful and supports assurance, risk management and compliance functions.
The EU AI Act requires behavioural evidence for high risk and general purpose AI systems, including scenario based testing and evaluation under realistic conditions. AIOBES provides the behavioural evaluation methodologies needed to meet these obligations. It enables organisations to produce structured, reproducible and auditable behavioural evidence that satisfies the Act’s requirements for operational testing, behavioural documentation and post deployment monitoring.
ISO 42001, the NIST AI RMF, internal governance systems and EU AI Act compliance processes all assume the existence of behavioural evidence. None of them define how behavioural evaluation should be conducted or how behavioural evidence should be produced. AIOBES fills this gap by providing formal behavioural evaluation methodologies that generate the operational behavioural evidence required for governance, assurance and regulatory compliance.
Behavioural evaluation begins at the design stage, where organisations define intended system behaviour, operational boundaries and behavioural risk assumptions. Early behavioural assessment ensures that design decisions account for behavioural failure modes, conversational drift, workflow instability and other operational risks that cannot be identified through documentation or prompt based testing.
During development, behavioural evaluation is applied to emerging features, workflows and interaction patterns. It enables teams to observe real behaviour as systems evolve, identify behavioural inconsistencies, detect drift and validate alignment with design intent. This ensures that behavioural risks are addressed before systems reach formal testing or deployment.
Behavioural evaluation is a required component of testing. It provides structured, scenario based and conversational evaluation conditions that reveal operational behaviour under realistic workloads. This includes multi step reasoning, workflow execution, conversational adaptation, recovery, escalation and failure. Behavioural evidence produced during testing supports assurance, governance and regulatory compliance.
Deployment requires behavioural evaluation to confirm that systems behave consistently in production environments. This includes validating operational stability, guardrail behaviour, workflow integrity and conversational coherence under real user conditions. Behavioural evidence generated during deployment supports compliance obligations and provides assurance that the system behaves as expected in live operation.
Updates, retraining, fine tuning and configuration changes can alter system behaviour. Behavioural evaluation must be applied after each change to detect behavioural drift, new failure modes or altered operational characteristics. This ensures that updates do not introduce behavioural instability or compromise compliance.
Behavioural evaluation continues throughout post deployment monitoring. It provides ongoing behavioural evidence that supports regulatory requirements for continuous oversight, including the EU AI Act’s obligations for post market monitoring. Continuous behavioural evaluation ensures that systems remain stable, aligned and compliant over time.
Behavioural evaluation is not a single activity. It is a lifecycle requirement that must be applied at design, development, testing, deployment, updates and monitoring. AIOBES provides the methodologies needed to generate behavioural evidence at each stage, ensuring that organisations understand real behaviour, manage behavioural risk and meet regulatory obligations throughout the product lifecycle.
Behavioural evidence must be generated under structured and realistic evaluation conditions. This includes scenario based workflows, conversational interactions, multi-step reasoning sequences and operational tasks that reflect real user behaviour. Evidence generation must capture behavioural outputs, failure modes, drift, recovery, escalation and stability across the full evaluation sequence. Evidence must be produced consistently across repeated runs to ensure reliability.
Behavioural evidence must be documented in a structured format that supports audit, assurance and regulatory review. Documentation must include evaluation conditions, inputs, outputs, behavioural observations, identified failure modes and any deviations from expected behaviour. Records must be clear, complete and suitable for internal governance, external audit and regulatory submission.
Behavioural evaluations must be reproducible under equivalent conditions. Reproducibility requires stable evaluation parameters, consistent scenario execution, controlled conversational vectors and repeatable workflow structures. Organisations must be able to reproduce behavioural evidence on demand to validate findings, support assurance activities and demonstrate compliance.
Behavioural evidence must be tied to specific model versions, configurations and deployment states. Organisations must maintain records that link behavioural outputs to the exact version of the system under evaluation. This ensures that behavioural changes introduced by updates, retraining or configuration modifications can be tracked, compared and validated. Version stability records support regulatory requirements for change management and behavioural oversight.
Behavioural evidence must include a complete and immutable audit trail that records the full context of each behavioural evaluation. Audit trails must provide sufficient detail for auditors and regulators to verify how the evaluation was conducted, confirm the integrity of the behavioural evidence and trace findings back to the specific system version under assessment. Audit trails must be preserved without alteration and maintained for the duration required by regulatory frameworks, supporting internal audit, external audit and regulatory inspection.
Behavioural evidence must be maintained throughout the lifecycle of the AI system. This includes storage, retrieval, version comparison, update tracking and integration with governance systems. Evidence must remain accessible for assurance, compliance and post deployment monitoring. Lifecycle management ensures that behavioural evidence remains valid, traceable and aligned with regulatory expectations.
Behavioural evidence is a formal compliance artefact. It must be generated, documented, reproduced and maintained with the same rigour applied to other regulated artefacts such as risk assessments, control libraries and audit documentation. AIOBES provides the structured workflow required to produce behavioural evidence that meets governance, assurance and regulatory obligations.
Organisations demonstrate conformance to AIOBES CAS 1.0 by producing behavioural evidence generated under structured and realistic evaluation conditions. Behavioural evidence must show how AI systems behave during operational tasks, workflows and interactions. Conformance requires evidence that is complete, verifiable and aligned with the behavioural evaluation methodologies defined in AIOBES.
Behavioural evidence must be tied to specific model versions, configurations and deployment states. Organisations must maintain version referenced behavioural records that allow auditors, regulators and assurance teams to trace behavioural outputs back to the exact system version under evaluation. Version referencing ensures that behavioural changes introduced by updates, retraining or configuration modifications can be identified, compared and validated.
Conformance requires structured assurance documentation that incorporates behavioural evidence into governance, risk and compliance processes. Assurance documentation must include evaluation conditions, behavioural observations, identified failure modes, version references and audit trail information. This documentation must be suitable for internal assurance, external audit and regulatory inspection.
Organisations must demonstrate that behavioural evidence can be reproduced under equivalent conditions. Reproducibility confirms that behavioural findings are reliable and not the result of isolated or non representative evaluation runs. Reproducible behavioural evidence is required for assurance, governance and regulatory compliance.
Conformance requires a complete and immutable audit trail for all behavioural evaluations. Audit trails must record the full evaluation context, including the conditions under which the evaluation was conducted and the behavioural outputs produced. They must provide sufficient detail to allow auditors and regulators to verify how the evaluation was performed, confirm the integrity of the behavioural evidence and trace findings back to the specific system version under assessment. Audit trails must be preserved for the duration required by regulatory frameworks and maintained without alteration.
Organisations demonstrate conformance by integrating behavioural evidence into existing governance, assurance and compliance processes. Behavioural evidence must inform risk assessments, control validation, compliance reporting and post deployment monitoring. Conformance is achieved when behavioural evaluation becomes a standard component of corporate governance without requiring architectural change or tooling replacement.
AIOBES CAS 1.0 defines behavioural evaluation as an operational obligation. Organisations demonstrate conformance by applying AIOBES methodologies across the lifecycle, producing structured behavioural evidence and maintaining version referenced, reproducible and auditable records. Conformance confirms that organisations understand real system behaviour and meet regulatory expectations for behavioural oversight.
Organisations must maintain access to the specific versions of AIOBES referenced in their behavioural evaluations. This includes the methodologies, definitions and requirements used during evidence generation. Version access ensures that behavioural evaluations can be verified, reproduced and audited against the correct standard.
Organisations must apply AIOBES behavioural evaluation methodologies consistently across all relevant systems, workloads and interaction types. Consistency ensures that behavioural evidence is comparable, reliable and aligned with the requirements of CAS 1.0. Selective or partial application does not meet conformance obligations.
Behavioural evidence must be preserved in a complete and immutable form for the duration required by regulatory frameworks. Preservation includes storage, retrieval and protection against alteration or loss. Organisations must ensure that behavioural evidence remains accessible for internal assurance, external audit and regulatory inspection.
Organisations must maintain behavioural traceability across updates, retraining, fine tuning and configuration changes. Behavioural evidence must be linked to specific system versions to allow auditors and regulators to identify behavioural changes introduced by modifications. Traceability ensures that behavioural drift, new failure modes or altered operational characteristics can be detected and validated.
Behavioural evaluation is a lifecycle responsibility. Organisations must apply AIOBES methodologies during design, development, testing, deployment, updates and monitoring. Conformance requires continuous behavioural oversight rather than isolated or one time evaluation.
Organisations must integrate behavioural evidence into governance, assurance and compliance processes. Behavioural evidence must inform risk assessments, control validation, compliance reporting and post deployment monitoring. Integration ensures that behavioural evaluation becomes a standard component of corporate governance.
Adopting AIOBES CAS 1.0 strengthens corporate governance by adding the behavioural evaluation layer that governance systems do not provide. Behavioural evidence makes governance controls operationally meaningful and enables organisations to validate that AI systems behave as intended under real conditions.
AIOBES supports compliance with regulatory frameworks that require behavioural evidence, including ISO 42001, the NIST AI RMF and the EU AI Act. Structured behavioural evaluation enables organisations to demonstrate compliance with behavioural obligations and produce evidence suitable for regulatory inspection.
Behavioural evidence produced under AIOBES methodologies is reproducible, version referenced and supported by complete audit trails. This improves auditability by providing verifiable records that allow auditors and regulators to confirm how evaluations were conducted and how systems behaved under defined conditions.
AIOBES reduces operational risk by identifying behavioural drift, instability, misalignment, workflow failure and guardrail failure before they affect production systems. Behavioural evaluation provides early visibility into failure modes that cannot be detected through documentation, prompt based testing or governance dashboards.
Structured behavioural evidence increases assurance confidence across development, deployment and monitoring. Organisations gain a clear understanding of real system behaviour, enabling more accurate risk assessments, more reliable control validation and more robust assurance reporting.
Applying AIOBES across the lifecycle improves behavioural stability by ensuring that updates, retraining and configuration changes do not introduce behavioural drift or new failure modes. Behavioural traceability supports continuous oversight and long term operational reliability.
AIOBES prepares organisations for current and emerging regulatory requirements by establishing behavioural evaluation as a standard governance practice. Adoption ensures that organisations can produce behavioural evidence on demand and demonstrate responsible AI operation with clarity and confidence.
-Document Ends-