yapi

CLI-first API testing for HTTP, GraphQL, gRPC, and TCP.

The Workflow

yapi enables test-driven API development. Write the test first, then implement until it passes:

Write the test - Create a .yapi.yml file with the expected behavior
Run it - yapi run file.yapi.yml (it will fail)
Implement/fix - Build the API endpoint
Iterate - Refine assertions, add edge cases

This loop is the core of agentic API development with yapi.

Environment Setup (Do This First)

Before writing any tests, set up your environments. Create yapi.config.yml in your project root:

yapi: v1
default_environment: local

environments:
  local:
    url: http://localhost:3000
    vars:
      API_KEY: dev_key_123

  staging:
    url: https://staging.example.com
    vars:
      API_KEY: ${STAGING_API_KEY}  # from shell env

  prod:
    url: https://api.example.com
    vars:
      API_KEY: ${PROD_API_KEY}
    env_files:
      - .env.prod  # load secrets from file

Now your tests use ${url} and ${API_KEY} - same test, any environment:

yapi run get-users.yapi.yml              # uses local (default)
yapi run get-users.yapi.yml --env staging
yapi run get-users.yapi.yml --env prod

Variable resolution order (highest priority first):

Shell environment variables
Environment-specific vars
Environment-specific env_files
Default vars
Default env_files

A) Smoke Testing

Quick health checks to verify endpoints are alive.

HTTP

yapi: v1
url: ${url}/health
method: GET
expect:
  status: 200

GraphQL

yapi: v1
url: ${url}/graphql
graphql: |
  query { __typename }
expect:
  status: 200
  assert:
    - .data.__typename != null

gRPC

yapi: v1
url: grpc://${host}:${port}
service: grpc.health.v1.Health
rpc: Check
plaintext: true
body:
  service: ""
expect:
  status: 200

TCP

yapi: v1
url: tcp://${host}:${port}
data: "PING\n"
encoding: text
expect:
  status: 200

B) Integration Testing

Multi-step workflows with data passing between requests. Use chains when steps depend on each other.

Authentication Flow

yapi: v1
chain:
  - name: login
    url: ${url}/auth/login
    method: POST
    body:
      email: test@example.com
      password: ${TEST_PASSWORD}
    expect:
      status: 200
      assert:
        - .token != null

  - name: get_profile
    url: ${url}/users/me
    method: GET
    headers:
      Authorization: Bearer ${login.token}
    expect:
      status: 200
      assert:
        - .email == "test@example.com"

CRUD Flow

yapi: v1
chain:
  - name: create
    url: ${url}/posts
    method: POST
    body:
      title: "Test Post"
      content: "Hello World"
    expect:
      status: 201
      assert:
        - .id != null

  - name: read
    url: ${url}/posts/${create.id}
    method: GET
    expect:
      status: 200
      assert:
        - .title == "Test Post"

  - name: update
    url: ${url}/posts/${create.id}
    method: PATCH
    body:
      title: "Updated Post"
    expect:
      status: 200

  - name: delete
    url: ${url}/posts/${create.id}
    method: DELETE
    expect:
      status: 204

Running Integration Tests

Name test files with .test.yapi.yml suffix:

tests/
  auth.test.yapi.yml
  posts.test.yapi.yml
  users.test.yapi.yml

Run all tests:

yapi test ./tests                    # sequential
yapi test ./tests --parallel 4       # concurrent
yapi test ./tests --env staging      # against staging
yapi test ./tests --verbose          # detailed output

C) Uptime Monitoring

Create test suites for monitoring your services in production.

Monitor Suite Structure

monitors/
  api-health.test.yapi.yml
  auth-service.test.yapi.yml
  database-check.test.yapi.yml
  graphql-schema.test.yapi.yml

Health Check with Timeout

yapi: v1
url: ${url}/health
method: GET
timeout: 5s  # fail if response takes longer
expect:
  status: 200
  assert:
    - .status == "healthy"
    - .database == "connected"

Run Monitoring Suite

# Check all monitors in parallel
yapi test ./monitors --parallel 10 --env prod

# With verbose output for debugging
yapi test ./monitors --parallel 10 --env prod --verbose

CI/CD Integration (GitHub Actions)

name: API Health Check
on:
  schedule:
    - cron: '*/5 * * * *'  # every 5 minutes
  workflow_dispatch:

jobs:
  monitor:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install yapi
        run: curl -fsSL https://yapi.run/install/linux.sh | bash

      - name: Run health checks
        env:
          PROD_API_KEY: ${{ secrets.PROD_API_KEY }}
        run: yapi test ./monitors --env prod --parallel 5

Load Testing

Stress test endpoints or entire workflows:

# 1000 requests, 50 concurrent
yapi stress api-flow.yapi.yml -n 1000 -p 50

# Run for 30 seconds
yapi stress api-flow.yapi.yml -d 30s -p 25

# Against production (with confirmation)
yapi stress api-flow.yapi.yml -e prod -n 500 -p 10

D) Async Job Polling with `wait_for`

For endpoints that process data asynchronously, use wait_for to poll until conditions are met.

Fixed Period Polling

yapi: v1
url: ${url}/jobs/${job_id}
method: GET

wait_for:
  until:
    - .status == "completed" or .status == "failed"
  period: 2s
  timeout: 60s

expect:
  assert:
    - .status == "completed"

Exponential Backoff

Better for rate-limited APIs or long-running jobs:

yapi: v1
url: ${url}/jobs/${job_id}
method: GET

wait_for:
  until:
    - .status == "completed"
  backoff:
    seed: 1s       # Initial wait
    multiplier: 2  # 1s -> 2s -> 4s -> 8s...
  timeout: 300s

Async Workflow Chain

Complete example: create job, poll until done, download result:

yapi: v1
chain:
  - name: create_job
    url: ${url}/jobs
    method: POST
    body:
      type: "data_export"
      filters:
        date_range: "last_30_days"
    expect:
      status: 202
      assert:
        - .job_id != null

  - name: wait_for_job
    url: ${url}/jobs/${create_job.job_id}
    method: GET
    wait_for:
      until:
        - .status == "completed" or .status == "failed"
      period: 2s
      timeout: 300s
    expect:
      assert:
        - .status == "completed"
        - .download_url != null

  - name: download_result
    url: ${wait_for_job.download_url}
    method: GET
    output_file: ./export.csv

Webhook/Callback Waiting

Wait for a webhook to be received:

yapi: v1
chain:
  - name: trigger_action
    url: ${url}/payments/initiate
    method: POST
    body:
      amount: 100
    expect:
      status: 202

  - name: wait_for_webhook
    url: ${url}/webhooks/received
    method: GET
    wait_for:
      until:
        - . | length > 0
        - .[0].event == "payment.completed"
      period: 1s
      timeout: 30s

E) Integrated Test Server

Automatically start your dev server, wait for health checks, run tests, and clean up. Configure in yapi.config.yml:

yapi: v1

test:
  start: "npm run dev"
  wait_on:
    - "http://localhost:3000/healthz"
    - "grpc://localhost:50051"
  timeout: 60s
  parallel: 8
  directory: "./tests"

environments:
  local:
    url: http://localhost:3000

Running with Integrated Server

# Automatically starts server, waits for health, runs tests, kills server
yapi test

# Skip server startup (server already running)
yapi test --no-start

# Override config from CLI
yapi test --start "npm start" --wait-on "http://localhost:4000/health"

# See server stdout/stderr
yapi test --verbose

Health Check Protocols

| Protocol | URL Format | Behavior | |----------|------------|----------| | HTTP/HTTPS | http://localhost:3000/healthz | Poll until 2xx response | | gRPC | grpc://localhost:50051 | Uses grpc.health.v1.Health/Check | | TCP | tcp://localhost:5432 | Poll until connection succeeds |

Local vs CI Parity

The same workflow works locally and in CI:

Local development:

yapi test  # starts server, runs tests, cleans up

GitHub Actions:

- uses: jamierpond/yapi/action@main
  with:
    start: npm run dev
    wait-on: http://localhost:3000/healthz
    command: yapi test -a

Commands Reference

| Command | Description | |---------|-------------| | yapi run file.yapi.yml | Execute a request | | yapi run file.yapi.yml --env prod | Execute against specific environment | | yapi test ./dir | Run all *.test.yapi.yml files | | yapi test ./dir --all | Run all *.yapi.yml files (not just tests) | | yapi test ./dir --parallel 4 | Run tests concurrently | | yapi validate file.yapi.yml | Check syntax without executing | | yapi watch file.yapi.yml | Re-run on every file save | | yapi stress file.yapi.yml | Load test with concurrency | | yapi list | List all yapi files in directory |

Assertion Syntax

Assertions use JQ expressions that must evaluate to true.

Body Assertions

expect:
  status: 200
  assert:
    - .id != null                    # field exists
    - .name == "John"                # exact match
    - .age > 18                      # comparison
    - . | length > 0                 # array not empty
    - .[0].email != null             # first item has email
    - .users | length == 10          # exactly 10 users
    - .type == "admin" or .type == "user"  # alternatives
    - .tags | contains(["api"])      # array contains value

Header Assertions

expect:
  status: 200
  assert:
    headers:
      - .["Content-Type"] | contains("application/json")
      - .["X-Request-Id"] != null
      - .["Cache-Control"] == "no-cache"
    body:
      - .data != null

Status Code Options

expect:
  status: 200           # exact match
  status: [200, 201]    # any of these

Protocol Examples

HTTP with Query Params and Headers

yapi: v1
url: ${url}/api/users
method: GET
headers:
  Authorization: Bearer ${API_KEY}
  Accept: application/json
query:
  limit: "10"
  offset: "0"
  sort: "created_at"
expect:
  status: 200

HTTP POST with JSON Body

yapi: v1
url: ${url}/api/users
method: POST
body:
  name: "John Doe"
  email: "john@example.com"
  roles:
    - admin
    - user
expect:
  status: 201
  assert:
    - .id != null

HTTP Form Data

yapi: v1
url: ${url}/upload
method: POST
content_type: multipart/form-data
form:
  name: "document.pdf"
  description: "Q4 Report"
expect:
  status: 200

GraphQL with Variables

yapi: v1
url: ${url}/graphql
graphql: |
  query GetUser($id: ID!) {
    user(id: $id) {
      id
      name
      email
    }
  }
variables:
  id: "123"
expect:
  status: 200
  assert:
    - .data.user.id == "123"

gRPC with Metadata

yapi: v1
url: grpc://${host}:${port}
service: users.UserService
rpc: GetUser
plaintext: true
headers:
  authorization: Bearer ${API_KEY}
body:
  user_id: "123"
expect:
  status: 200
  assert:
    - .user.id == "123"

TCP Raw Connection

yapi: v1
url: tcp://${host}:${port}
data: |
  GET / HTTP/1.1
  Host: example.com

encoding: text
read_timeout: 5
expect:
  status: 200

File Organization

Recommended project structure:

project/
  yapi.config.yml          # environments
  .env                     # local secrets (gitignored)
  .env.example             # template for secrets

  tests/
    auth/
      login.test.yapi.yml
      logout.test.yapi.yml
    users/
      create-user.test.yapi.yml
      get-user.test.yapi.yml

  monitors/
    health.test.yapi.yml
    critical-endpoints.test.yapi.yml

Tips

Start simple: Begin with status code checks, add body assertions as needed
Use watch mode: yapi watch file.yapi.yml for rapid iteration
Validate before running: yapi validate file.yapi.yml catches syntax errors
Keep tests focused: One logical flow per file
Name steps clearly: In chains, use descriptive names like create_user, verify_email
Reference previous steps: Use ${step_name.field} to pass data between chain steps

Yapi

Skill Content