AI-Assisted Engineering

The AI-Native SDLC: Humans, Agents, CI/CD and Automated Verification

When agents write a large share of code, the question 'how do we know it's right?' moves to the centre of the software development lifecycle. This article describes the verification stack of an AI-native SDLC, from executable specifications and property-based tests to contract checks, evals, policy gates and provenance, and how humans fit around it.

By EthicalHat Research · · 3 min read
Infographic: The verification stack of an AI-native SDLC

In a traditional SDLC, the scarce resource is engineering time to write code. In an AI-native SDLC, where agents produce a significant share of implementation, that constraint loosens, and a different one takes its place: the capacity to verify that changes are correct, secure and aligned with intent.

Teams that increase code generation without increasing verification capacity get faster at shipping defects. This article describes the verification stack we see in teams that have got the balance right.

Layer 1: executable specifications

Specifications that can be run are the foundation. They include:

  • Acceptance tests written from requirements before implementation, often by humans or with human review of AI-drafted tests.
  • Examples embedded in specifications (inputs and expected outputs) that become tests automatically.
  • Behaviour-driven scenarios for user-facing flows.

The key rule: expected behaviour comes from intent, never from running the implementation and recording what it does.

Layer 2: stronger tests

Example-based unit tests are necessary but not sufficient when an agent may satisfy them in unexpected ways. Add:

  • Property-based tests that check invariants over many generated inputs:
python
from hypothesis import given, strategies as st

@given(st.lists(st.tuples(st.integers(0, 10_000), st.integers(0, 10_000))))
def test_merge_intervals_properties(raw):
    intervals = [(min(a, b), max(a, b)) for a, b in raw]
    merged = merge_intervals(intervals)
    # 1. output is sorted and non-overlapping
    assert all(merged[i][1] < merged[i + 1][0] for i in range(len(merged) - 1))
    # 2. every input point is covered by some output interval
    for s, e in intervals:
        assert any(ms <= s and e <= me for ms, me in merged)
  • Mutation testing to confirm the tests would catch real faults.
  • Regression tests for every bug fixed, human or agent.

Layer 3: contracts and types

  • Strict type checking catches a surprising share of agent mistakes cheaply.
  • API contracts (OpenAPI, protobuf, GraphQL schemas) with contract tests between services.
  • Schema checks for database migrations, configuration and infrastructure code.
  • Architecture rules that enforce module boundaries and dependency directions.

Layer 4: security verification

  • SAST, dependency and secret scanning on every change.
  • Infrastructure-as-code policy checks.
  • Authorisation tests: "user A cannot access user B's data", run as part of the suite.
  • For changes that affect AI features, evals for quality and safety, including prompt-injection cases.

Layer 5: policy and provenance

Every change should carry a record of how it was made and checked:

  • who or what authored it (a human, an agent and which version, or both),
  • which task or issue it implements,
  • which checks ran and their results, and
  • who reviewed and approved it.

Signed commits and build provenance attestations (in the spirit of SLSA) let you verify this later. Policy gates then use it: for example, agent-authored changes to security-sensitive paths require a code owner's approval, and deployment requires provenance showing all required checks passed.

flowchart LR
  T[Task + spec] --> I[Implementation<br/>human or agent]
  I --> V1[Tests & properties]
  V1 --> V2[Types & contracts]
  V2 --> V3[Security & evals]
  V3 --> P[Provenance record<br/>author, checks, approvals]
  P --> G{Policy gate}
  G -->|meets policy| H[Human review<br/>intent & risk]
  G -->|missing checks| X[Blocked]
  H --> D[Deploy with<br/>progressive rollout]

Layer 6: human judgement

Automation cannot judge everything. Humans concentrate on:

  • intent: does this change do what the business actually needs?
  • risk: what could go wrong in production, and is the rollout safe?
  • design: does this keep the system coherent and maintainable?
  • the unmeasured: usability, ethics, and whether the specification itself was right.

Because the lower layers catch mechanical problems, human reviewers can spend their attention where it matters most.

Progressive delivery completes the loop

Verification continues after merge: feature flags, canary releases, automated rollback on error-budget burn, and production monitoring. Incidents feed back into new tests, properties and policies.

Metrics for an AI-native SDLC

  • change failure rate and time to restore, overall and for agent-authored changes,
  • mutation score for critical modules,
  • percentage of changes with complete provenance,
  • review time per change, and
  • escaped defects by verification layer (which layer should have caught it?).

The bottom line

AI-native development does not reduce the need for engineering rigour; it moves it. When writing code becomes cheap, verifying it becomes the craft. A layered verification stack of executable specifications, strong tests, contracts, security checks, provenance and focused human judgement is what lets organisations turn agent throughput into reliable software.

Our secure SDLC team builds verification-first delivery pipelines, and our application security testing integrates security verification into them.

EthicalHat Research

The EthicalHat research team: penetration testers, SOC analysts and engineers writing about security, software engineering and AI.

Want an expert opinion on your own systems?

Email info@ethicalhat.com or call +1-844-838-4422. Our security engineers are happy to talk it through.