Skip to the content
AI Test AutomationPreview AI Security Learning Center

AI Test AutomationAI, cloud and DevOpsAI pipeline

AI, cloud and DevOps Preview

AI pipeline: where the checks run

Shift left is only real when a tool runs the check before a person has to. This page is the pipeline that does it: the free tools by the moment they run, the four files that wire them in, and the hardening of the pipeline itself, because a CI system that runs agents is the most privileged machine in the company.

AI stations

The AI pipeline: four places a check can run

Four places a check can run, from cheapest to most consequential. The earlier the station, the cheaper the fix; the later the station, the more it has to catch.

  1. 1

    At commit

    Secrets, dependency advisories, your own static rules, infrastructure-as-code checks, instruction files.

    Seconds, on the developer's machine, before the change exists anywhere else.

  2. 2

    At the pull request

    Everything from commit again, plus deeper analysis, policy as code, the bill of materials, model files, and the AI system's own evals.

    Minutes; a required check blocks the merge for people and agents alike.

  3. 3

    Before release

    The slow tests against a production-like environment: integration, end to end, and the agent security tests from this center, each many times.

    An hour on a release candidate; the last gate before real credentials and real users.

  4. 4

    In production

    Monitoring, the audit trail and scheduled re-runs; a new incident becomes a new test.

    Continuous; what no test before release could know.

AI toolkit

AI toolkit: the free tools, by the moment they run

Shift left is only real when a tool runs the check before a person has to. These are the free ones, by the moment they run: in the editor and at commit, where a finding costs seconds; at the pull request, where it blocks a merge; and before a line exists, where it changes the design. Each one maps to a test or a weakness in this center.

In the editor and at commit: seconds, not days

Hooks that run on every commit catch the mistakes that are cheapest to fix right then: a secret pasted into a file, a dependency with a known CVE, a dangerous call, a Terraform change that opens a port. The pre-commit framework runs all of them from one file.

pre-commit

pre-commit.com, open source

A framework for managing git hooks: one .pre-commit-config.yaml lists the checks, and each runs on the files in a commit before it is made.

Example below: the config we use

gitleaks

Zachary Rice, open source (MIT)

Detects secrets such as passwords, API keys and tokens in git repositories, files and standard input, and ships a pre-commit hook so a secret never reaches a commit. Its author calls it feature complete, with future releases limited to security patches, and points new work to Betterleaks.

Weakness: Secrets written into code and commits by agents

TruffleHog

Truffle Security, open source (AGPL-3.0)

Finds leaked credentials in git, chats, wikis, logs, object stores and filesystems, classifies more than 800 secret types and verifies which ones are live; available as a pre-commit hook and a GitHub Action.

Weakness: Secrets written into code and commits by agents

pip-audit

Python Packaging Authority (created by Trail of Bits), open source (Apache 2.0)

Scans Python environments and requirements files for packages with known vulnerabilities, using the Python Packaging Advisory Database through the PyPI JSON API; ships a pre-commit hook and a GitHub Action.

In this center: The build fails on a known, fixed CVE in your AI stack

Semgrep

Semgrep, Community Edition engine open source (LGPL-2.1)

Fast static analysis that searches code with rules written as code patterns, in more than 30 languages, in the IDE, as a pre-commit check and in CI; a team can write its own rule for the mistake its agents keep making, such as a model's output passed to a shell. The free engine analyzes one function or file at a time; cross-file analysis is in the commercial platform.

Weakness: Improper handling of model output (XSS, SQL and command injection)

Checkov

Prisma Cloud (Palo Alto Networks), open source (Apache 2.0)

Static analysis for infrastructure as code (Terraform, CloudFormation, Kubernetes, Helm, Dockerfiles, CI workflow files) with more than 1,000 built-in policies, plus composition analysis for images and packages, so a misconfiguration such as an AI gateway open to the internet fails before it is applied.

Weakness: Exposed AI infrastructure without authentication

At the pull request: the gates nobody can skip

The pull request is where a required check blocks a merge, human or agent. Dependency advisories, deeper static analysis, policy as code, the bill of materials, model files and the AI system's own evals all belong here, in minutes.

OSV-Scanner

Google, open source (Apache 2.0)

Finds known vulnerabilities in a project's dependencies, lockfiles, SBOMs, OS packages and container images against the OSV database, in 11 language ecosystems, with guided remediation; its GitHub Action's pull request workflow reports only the vulnerabilities the pull request introduces.

In this center: The build fails on a known, fixed CVE in your AI stack

Open Policy Agent and conftest

Open Policy Agent, a CNCF graduated project, open source (Apache 2.0)

Policy as code: OPA is a general-purpose policy engine, and conftest tests structured configuration (Kubernetes, Terraform, JSON, YAML) against policies written in its Rego language, so a rule such as every MCP server is pinned fails the build.

In this center: Every MCP server is pinned to an exact version

Syft and the CycloneDX ML-BOM

Anchore, open source (Apache 2.0); OWASP CycloneDX

Syft generates a software bill of materials from container images and filesystems, in CycloneDX or SPDX, and can sign it as an attestation; CycloneDX's ML-BOM adds models and datasets to the bill, for transparency on security, privacy, safety and ethics, so what an AI system is built from is written down and checkable.

Weakness: Unverified model provenance

modelscan and picklescan

Protect AI, open source (Apache 2.0); Matthieu Maitre, open source (MIT)

ModelScan scans model files in several formats (pickle, H5, SavedModel, from PyTorch, TensorFlow, Keras, scikit-learn and XGBoost) for unsafe code that runs when the model loads, and is built for CI pipelines; picklescan checks pickle files, directories, archives and Hugging Face models the same way, with exit codes a pipeline can gate on.

Weakness: Unsafe model serialization (pickle)

Sigstore cosign and model signing

Sigstore, under the OpenSSF, open source

Sigstore lets developers sign and verify release files, container images and binaries, with Cosign as the client; its Model Transparency project's model-signing package signs and verifies model files the same way, so a model's provenance is checked rather than assumed.

Weakness: Unverified model provenance

promptfoo

Promptfoo, open source (action under MIT)

Evals and red-team tests for prompts, models and agents; its GitHub Action runs the evaluation when monitored files change in a pull request and posts a comment with the pass and fail counts, so a prompt change is tested like a code change.

See the tools under Testing AI systems on the AI and agents page

DeepEval

Confident AI, open source (Apache 2.0)

An LLM evaluation framework that works like pytest, with metrics for answer relevancy, hallucination, task completion, tool correctness and plan adherence, so the AI system's tests run in the same suite and the same CI as everything else.

See pass^k, the number to gate on, on the AI and agents page

Before a line exists: threat-model the agent

The cheapest fix is the design that never had the flaw. For an AI agent that means deciding, before it is built, what it reads, what it can reach, what it can do and what it is built on, and which of this center's incidents a setup like that could have been part of.

MAESTRO

Cloud Security Alliance (Ken Huang), February 2025

A threat modeling framework designed for agentic AI, in seven layers: foundation models, data operations, agent frameworks, deployment and infrastructure, evaluation and observability, security and compliance across all of them, and the agent ecosystem.

Blast radius, in the Learning Center, does the same with real incidents

OWASP Threat Dragon

OWASP production project, open source (Apache 2.0)

A modeling tool for creating threat model diagrams as part of a secure development lifecycle, so the model lives with the project and is updated as the design changes.

pytm

OWASP, open source (MIT)

Threat modeling as code, whose stated goal is to shift threat modeling to the left: the system is described in Python, and the tool generates the data flow diagram, the sequence diagram and the relevant threats, so the model is versioned and reviewed like code.

AI files to copy

AI pipeline files: four to copy

Four files to copy. The first runs the commit-time checks on every developer's machine and in CI; the second is the pull request gate; the third is the checklist an agent's pull request has to pass before a person reads it. Pin every tool to a version you have reviewed.

Commit-time checks: .pre-commit-config.yaml
.pre-commit-config.yaml · yaml
# Install once: pip install pre-commit && pre-commit install
# Then run `pre-commit autoupdate` once: it replaces every rev below with the latest tag,
# which you review and commit. The same checks run in CI with `pre-commit run --all-files`.
repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.24.2
    hooks:
      - id: gitleaks            # no secret reaches a commit
  - repo: https://github.com/pypa/pip-audit
    rev: v0.0.0                 # set by pre-commit autoupdate
    hooks:
      - id: pip-audit           # no dependency with a known CVE
        args: ["-r", "requirements.txt"]
  - repo: https://github.com/semgrep/semgrep
    rev: v0.0.0                 # set by pre-commit autoupdate
    hooks:
      - id: semgrep             # your own rules for the mistakes your agents make
        args: ["--config", ".semgrep/", "--error"]
  - repo: https://github.com/bridgecrewio/checkov
    rev: v0.0.0                 # set by pre-commit autoupdate
    hooks:
      - id: checkov             # Terraform and Kubernetes misconfigurations
        args: ["--quiet", "--framework", "terraform"]
  - repo: local
    hooks:
      - id: ai-instruction-files
        name: instruction files have no hidden characters
        entry: python -m pytest -q tests/test_instruction_files.py
        language: system
        files: ^(CLAUDE\.md|AGENTS\.md|\.cursor/rules/.*|\.mcp\.json)$
        pass_filenames: false
The pull request gate: .github/workflows/shift-left-gates.yml
.github/workflows/shift-left-gates.yml · yaml
# Required checks on every pull request. Mark both jobs as required in branch protection so
# that no one, human or agent, can merge around them.
name: shift-left gates
on: [pull_request]
permissions:
  actions: read
  contents: read
  security-events: write   # the OSV results go to the Security tab
jobs:
  known-cves:
    # OSV-Scanner's own workflow: reports vulnerabilities the pull request introduces
    uses: "google/osv-scanner-action/.github/workflows/osv-scanner-reusable-pr.yml@v2.7.0"
    with:
      scan-args: |-
        --recursive
        ./
  gates:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: {fetch-depth: 0}
      - uses: actions/setup-python@v5
        with: {python-version: "3.12"}
      - name: Commit-time checks, again, on every file
        run: pip install pre-commit && pre-commit run --all-files
      - name: Policy as code (every MCP server pinned, no public AI gateway)
        run: |
          curl -sSL https://github.com/open-policy-agent/conftest/releases/latest/download/conftest_Linux_x86_64.tar.gz | tar xz
          ./conftest test .mcp.json infra/*.tf --policy policy/
      - name: The AI system's own evals, as a regression gate
        uses: promptfoo/promptfoo-action@v1
        with:
          config: promptfooconfig.yaml
          github-token: ${{ secrets.GITHUB_TOKEN }}
          openai-api-key: ${{ secrets.OPENAI_API_KEY }}
      - name: The tests from this center (they skip until connected, never fail)
        run: pip install -r requirements-test.txt && python -m pytest -q tests/ai-security/
The agent's definition of done: .github/PULL_REQUEST_TEMPLATE.md
.github/PULL_REQUEST_TEMPLATE.md · markdown
## What this changes

<!-- one paragraph, written for the reviewer, not for the model -->

## Before a person reads this

- [ ] Tests were written first; no existing test was changed to make it pass (a changed test is listed below)
- [ ] `pre-commit run --all-files` is clean on this branch
- [ ] No new dependency, MCP server, skill or extension without an exact version and a link to its source
- [ ] No new hostname, URL or credential in code, config or instruction files that is not already on the allowlist
- [ ] Anything that reaches the network, runs commands or touches production says so here, and why
- [ ] The acceptance criteria this satisfies are linked, and each has a test

## Tests changed or deleted

<!-- every one, with the reason; empty means none -->

## Generated by

<!-- the agent and model, the prompt or task id, and who reviewed the output -->
A hardened workflow that runs an agent: .github/workflows/agent-tests.yml
.github/workflows/agent-tests.yml · yaml
# An AI tester or coding agent in CI, with GitHub's hardening guidance applied: a read-only
# token raised per job, actions pinned to full-length SHAs (replace the placeholders with the
# SHA from each action's own repository), OIDC to the cloud instead of stored keys, no
# pull_request_target, and untrusted text passed through environment variables, never
# interpolated into a shell. Put this file under CODEOWNERS.
name: agent tests
on:
  pull_request:          # not pull_request_target: a pull request is untrusted input
permissions:
  contents: read         # the default for every job; raise only where a job needs more
jobs:
  agent-tests:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write    # only for the OIDC exchange below
    env:
      PR_TITLE: ${{ github.event.pull_request.title }}   # untrusted: an env var, not a run line
    steps:
      - uses: actions/checkout@0000000000000000000000000000000000000000   # pin: actions/checkout
      - uses: aws-actions/configure-aws-credentials@0000000000000000000000000000000000000000   # pin
        with:
          role-to-assume: arn:aws:iam::123456789012:role/ci-agent-tests-readonly
          aws-region: us-east-1
      - uses: actions/setup-python@0000000000000000000000000000000000000000   # pin: actions/setup-python
        with: {python-version: "3.12"}
      - name: Run the agent's tests with an egress allowlist and no secrets in the tree
        env:
          AGENT_ALLOWED_HOSTS: api.example.com,registry.npmjs.org
        run: |
          echo "Testing: $PR_TITLE"          # the variable, never ${{ ... }} inside run
          pip install -r requirements-test.txt
          python -m pytest -q tests/ai-security/

AI hardening

AI pipeline hardening: the workflow that runs agents

The pipeline runs with the keys to the kingdom, and it now runs agents. GitHub's own hardening guidance, in its words where it counts, applied to a workflow that lets an AI tester or coding agent loose on a repository.

  1. Give the workflow token the minimum it needs

    Set the default permission for the GITHUB_TOKEN to read access only for repository contents, then raise permissions per job as needed. An agent step that only needs to read code should not be able to push, open releases or write to the Security tab.

  2. Pin every third-party action to a full-length commit SHA

    A tag can be moved or deleted; a SHA cannot. Verify the SHA comes from the action's own repository rather than a fork, and where a workflow dependency lockfile is available, use it, since GitHub calls it the strongest protection and says it supersedes SHA pinning. Dependabot keeps pinned references current, but it does not raise alerts for actions pinned to SHAs, so review them.

  3. Authenticate to the cloud with OpenID Connect, not stored keys

    Where the cloud provider supports OIDC, stop storing long-lived credentials as secrets; the job gets a short-lived token for exactly the role it needs. A secret that does not exist cannot be read by a hijacked agent.

  4. Keep untrusted code and privileged workflows apart

    Avoid the pull_request_target trigger unless it is necessary, prefer workflow_run for privilege separation, and never let a privileged workflow explicitly check out untrusted pull request code. A pull request from an agent, or from a fork, is untrusted input.

  5. Never interpolate untrusted text into a run step

    A pull request title, an issue body or a commit message can carry a script injection. Pass the expression through an intermediate environment variable rather than into the shell, or use a JavaScript action that receives it as an argument. The same rule protects an agent reading that text.

  6. Own the workflow files

    Put .github/workflows under CODEOWNERS so changes need a designated approval, review the source of third-party actions, check run logs with valid and invalid inputs to confirm secrets are redacted, and use dependency review to see what an action version change brings.

Sources: GitHub Docs: secure use reference for GitHub Actions

AI tests here

AI tests and practices this pipeline enforces

The practices and tests in this center that the pipeline enforces.

Practices: Keep AI agents in CI away from secrets and fork codeCurate dependencies: approve, pin, verify, and re-checkRe-check approved components whenever new advisories landKeep secrets out of reach of models and agentsPin approved MCP servers to their tool list and re-check on every changeNever expose AI infrastructure to the internet

Tests: The build fails on a known, fixed CVE in your AI stackEvery MCP server is pinned to an exact versionAgent instruction files hold no invisible charactersSkills and instruction files name only hosts you ownAn MCP server's tools match the snapshot you reviewedThe agent's sandbox reaches the allowlist and nothing else

AI, cloud and DevOps: the other pages