yapi

jamierpond/yapi

109 stars

2 forks

40 views

View on GitHub Add to Favorites

SKILL.md

yapi

CLI-first API testing for HTTP, GraphQL, gRPC, and TCP.

The Workflow

yapi enables test-driven API development. Write the test first, then implement until it passes:

Write the test - Create a .yapi.yml file with the expected behavior
Run it - yapi run file.yapi.yml (it will fail)
Implement/fix - Build the API endpoint
Iterate - Refine assertions, add edge cases

This loop is the core of agentic API development with yapi.

Environment Setup (Do This First)

Before writing any tests, set up your environments. Create yapi.config.yml in your project root:

yapi: v1
default_environment: local

environments:
  local:
    url: http://localhost:3000
    vars:
      API_KEY: dev_key_123

  staging:
    url: https://staging.example.com
    vars:
      API_KEY: ${STAGING_API_KEY}  # from shell env

  prod:
    url: https://api.example.com
    vars:
      API_KEY: ${PROD_API_KEY}
    env_files:
      - .env.prod  # load secrets from file

Now your tests use ${url} and ${API_KEY} - same test, any environment:

yapi run get-users.yapi.yml              # uses local (default)
yapi run get-users.yapi.yml --env staging
yapi run get-users.yapi.yml --env prod

Variable resolution order (highest priority first):

Shell environment variables
Environment-specific vars
Environment-specific env_files
Default vars
Default env_files

A) Smoke Testing

Quick health checks to verify endpoints are alive.

HTTP

yapi: v1
url: ${url}/health
method: GET
expect:
  status: 200

GraphQL

yapi: v1
url: ${url}/graphql
graphql: |
  query { __typename }
expect:
  status: 200
  assert:
    - .data.__typename != null

gRPC

yapi: v1
url: grpc://${host}:${port}
service: grpc.health.v1.Health
rpc: Check
plaintext: true
body:
  service: ""
expect:
  status: 200

TCP

yapi: v1
url: tcp://${host}:${port}
data: "PING\n"
encoding: text
expect:
  status: 200

B) Integration Testing

Multi-step workflows with data passing between requests. Use chains when steps depend on each other.

Authentication Flow

yapi: v1
chain:
  - name: login
    url: ${url}/auth/login
    method: POST
    body:
      email: [email protected]
      password: ${TEST_PASSWORD}
    expect:
      status: 200
      assert:
        - .token != null

  - name: get_profile
    url: ${url}/users/me
    method: GET
    headers:
      Authorization: Bearer ${login.token}
    expect:
      status: 200
      assert:
        - .email == "[email protected]"

CRUD Flow

yapi: v1
chain:
  - name: create
    url: ${url}/posts
    method: POST
    body:
      title: "Test Post"
      content: "Hello World"
    expect:
      status: 201
      assert:
        - .id != null

  - name: read
    url: ${url}/posts/${create.id}
    method: GET
    expect:
      status: 200
      assert:
        - .title == "Test Post"

  - name: update
    url: ${url}/posts/${create.id}
    method: PATCH
    body:
      title: "Updated Post"
    expect:
      status: 200

  - name: delete
    url: ${url}/posts/${create.id}
    method: DELETE
    expect:
      status: 204

Running Integration Tests

Name test files with .test.yapi.yml suffix:

tests/
  auth.test.yapi.yml
  posts.test.yapi.yml
  users.test.yapi.yml

Run all tests:

yapi test ./tests                    # sequential
yapi test ./tests --parallel 4       # concurrent
yapi test ./tests --env staging      # against staging
yapi test ./tests --verbose          # detailed output

C) Uptime Monitoring

Create test suites for monitoring your services in production.

Monitor Suite Structure

monitors/
  api-health.test.yapi.yml
  auth-service.test.yapi.yml
  database-check.test.yapi.yml
  graphql-schema.test.yapi.yml

Health Check with Timeout

yapi: v1
url: ${url}/health
method: GET
timeout: 5s  # fail if response takes longer
expect:
  status: 200
  assert:
    - .status == "healthy"
    - .database == "connected"

Run Monitoring Suite

# Check all monitors in parallel
yapi test ./monitors --parallel 10 --env prod

# With verbose output for debugging
yapi test ./monitors --parallel 10 --env prod --verbose

CI/CD Integration (GitHub Actions)

name: API Health Check
on:
  schedule:
    - cron: '*/5 * * * *'  # every 5 minutes
  workflow_dispatch:

jobs:
  monitor:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install yapi
        run: curl -fsSL https://yapi.run/install/linux.sh | bash

      - name: Run health checks
        env:
          PROD_API_KEY: ${{ secrets.PROD_API_KEY }}
        run: yapi test ./monitors --env prod --parallel 5

Load Testing

Stress test endpoints or entire workflows:

# 1000 requests, 50 concurrent
yapi stress api-flow.yapi.yml -n 1000 -p 50

# Run for 30 seconds
yapi stress api-flow.yapi.yml -d 30s -p 25

# Against production (with confirmation)
yapi stress api-flow.yapi.yml -e prod -n 500 -p 10

D) Async Job Polling with `wait_for`

For endpoints that process data asynchronously, use wait_for to poll until conditions are met.

Fixed Period Polling

yapi: v1
url: ${url}/jobs/${job_id}
method: GET

wait_for:
  until:
    - .status == "completed" or .status == "failed"
  period: 2s
  timeout: 60s

expect:
  assert:
    - .status == "completed"

Exponential Backoff

Better for rate-limited APIs or long-running jobs:

yapi: v1
url: ${url}/jobs/${job_id}
method: GET

wait_for:
  until:
    - .status == "completed"
  backoff:
    seed: 1s       # Initial wait
    multiplier: 2  # 1s -> 2s -> 4s -> 8s...
  timeout: 300s

Async Workflow Chain

Complete example: create job, poll until done, download result:

yapi: v1
chain:
  - name: create_job
    url: ${url}/jobs
    method: POST
    body:
      type: "data_export"
      filters:
        date_range: "last_30_days"
    expect:
      status: 202
      assert:
        - .job_id != null

  - name: wait_for_job
    url: ${url}/jobs/${create_job.job_id}
    method: GET
    wait_for:
      until:
        - .status == "completed" or .status == "failed"
      period: 2s
      timeout: 300s
    expect:
      assert:
        - .status == "completed"
        - .download_url != null

  - name: download_result
    url: ${wait_for_job.download_url}
    method: GET
    output_file: ./export.csv

Webhook/Callback Waiting

Wait for a webhook to be received:

yapi: v1
chain:
  - name: trigger_action
    url: ${url}/payments/initiate
    method: POST
    body:
      amount: 100
    expect:
      status: 202

  - name: wait_for_webhook
    url: ${url}/webhooks/received
    method: GET
    wait_for:
      until:
        - . | length > 0
        - .[0].event == "payment.completed"
      period: 1s
      timeout: 30s

E) Integrated Test Server

Automatically start your dev server, wait for health checks, run tests, and clean up. Configure in yapi.config.yml:

yapi: v1

test:
  start: "npm run dev"
  wait_on:
    - "http://localhost:3000/healthz"
    - "grpc://localhost:50051"
  timeout: 60s
  parallel: 8
  directory: "./tests"

environments:
  local:
    url: http://localhost:3000

Running with Integrated Server

# Automatically starts server, waits for health, runs tests, kills server
yapi test

# Skip server startup (server already running)
yapi test --no-start

# Override config from CLI
yapi test --start "npm start" --wait-on "http://localhost:4000/health"

# See server stdout/stderr
yapi test --verbose

Health Check Protocols

Protocol	URL Format	Behavior
HTTP/HTTPS	`http://localhost:3000/healthz`	Poll until 2xx response
gRPC	`grpc://localhost:50051`	Uses `grpc.health.v1.Health/Check`
TCP	`tcp://localhost:5432`	Poll until connection succeeds

Local vs CI Parity

The same workflow works locally and in CI:

Local development:

yapi test  # starts server, runs tests, cleans up

GitHub Actions:

- uses: jamierpond/yapi/action@main
  with:
    start: npm run dev
    wait-on: http://localhost:3000/healthz
    command: yapi test -a

Commands Reference

Command	Description
`yapi run file.yapi.yml`	Execute a request
`yapi run file.yapi.yml --env prod`	Execute against specific environment
`yapi test ./dir`	Run all `*.test.yapi.yml` files
`yapi test ./dir --all`	Run all `*.yapi.yml` files (not just tests)
`yapi test ./dir --parallel 4`	Run tests concurrently
`yapi validate file.yapi.yml`	Check syntax without executing
`yapi watch file.yapi.yml`	Re-run on every file save
`yapi stress file.yapi.yml`	Load test with concurrency
`yapi list`	List all yapi files in directory

Assertion Syntax

Assertions use JQ expressions that must evaluate to true.

Body Assertions

expect:
  status: 200
  assert:
    - .id != null                    # field exists
    - .name == "John"                # exact match
    - .age > 18                      # comparison
    - . | length > 0                 # array not empty
    - .[0].email != null             # first item has email
    - .users | length == 10          # exactly 10 users
    - .type == "admin" or .type == "user"  # alternatives
    - .tags | contains(["api"])      # array contains value

Header Assertions

expect:
  status: 200
  assert:
    headers:
      - .["Content-Type"] | contains("application/json")
      - .["X-Request-Id"] != null
      - .["Cache-Control"] == "no-cache"
    body:
      - .data != null

Status Code Options

expect:
  status: 200           # exact match
  status: [200, 201]    # any of these

Protocol Examples

HTTP with Query Params and Headers

yapi: v1
url: ${url}/api/users
method: GET
headers:
  Authorization: Bearer ${API_KEY}
  Accept: application/json
query:
  limit: "10"
  offset: "0"
  sort: "created_at"
expect:
  status: 200

HTTP POST with JSON Body

yapi: v1
url: ${url}/api/users
method: POST
body:
  name: "John Doe"
  email: "[email protected]"
  roles:
    - admin
    - user
expect:
  status: 201
  assert:
    - .id != null

HTTP Form Data

yapi: v1
url: ${url}/upload
method: POST
content_type: multipart/form-data
form:
  name: "document.pdf"
  description: "Q4 Report"
expect:
  status: 200

GraphQL with Variables

yapi: v1
url: ${url}/graphql
graphql: |
  query GetUser($id: ID!) {
    user(id: $id) {
      id
      name
      email
    }
  }
variables:
  id: "123"
expect:
  status: 200
  assert:
    - .data.user.id == "123"

gRPC with Metadata

yapi: v1
url: grpc://${host}:${port}
service: users.UserService
rpc: GetUser
plaintext: true
headers:
  authorization: Bearer ${API_KEY}
body:
  user_id: "123"
expect:
  status: 200
  assert:
    - .user.id == "123"

TCP Raw Connection

yapi: v1
url: tcp://${host}:${port}
data: |
  GET / HTTP/1.1
  Host: example.com

encoding: text
read_timeout: 5
expect:
  status: 200

File Organization

Recommended project structure:

project/
  yapi.config.yml          # environments
  .env                     # local secrets (gitignored)
  .env.example             # template for secrets

  tests/
    auth/
      login.test.yapi.yml
      logout.test.yapi.yml
    users/
      create-user.test.yapi.yml
      get-user.test.yapi.yml

  monitors/
    health.test.yapi.yml
    critical-endpoints.test.yapi.yml

Tips

Start simple: Begin with status code checks, add body assertions as needed
Use watch mode: yapi watch file.yapi.yml for rapid iteration
Validate before running: yapi validate file.yapi.yml catches syntax errors
Keep tests focused: One logical flow per file
Name steps clearly: In chains, use descriptive names like create_user, verify_email
Reference previous steps: Use ${step_name.field} to pass data between chain steps

Installation

Option 1: Use slash command in Claude Code

/install-skill https://github.com/jamierpond/yapi

Option 2: Clone to skills directory

# Global (all projects)

git clone https://github.com/jamierpond/yapi ~/.claude/skills/yapi

# Project-specific

git clone https://github.com/jamierpond/yapi .claude/skills/yapi

Add MCP server to .cursor/mcp.json:

{
  "mcpServers": {
    "skillz": {
      "command": "npx",
      "args": ["-y", "skillz-mcp", "https://github.com/jamierpond/yapi"]
    }
  }
}

Restart Cursor after adding the configuration.

Option 1: Use Gemini CLI command

gemini extensions install https://github.com/jamierpond/yapi

Option 2: Clone to extensions directory

git clone https://github.com/jamierpond/yapi ~/.gemini/extensions/yapi

Topics

Related Skills

beads

Beads - A memory upgrade for your coding agent

bloblang-authoring

Fancy stream processing made operationally mundane

schema-designer

A lightweight next-gen data explorer - Postgres, MySQL, SQLite, MongoDB, Redis, MariaDB, Elastic Search, and Clickhouse with Chat interface

prepare-workspace-for-plan

The to-do app to organize your life.

yapi

SKILL.md

yapi

The Workflow

Environment Setup (Do This First)

A) Smoke Testing

HTTP

GraphQL

gRPC

TCP

B) Integration Testing

Authentication Flow

CRUD Flow

Running Integration Tests

C) Uptime Monitoring

Monitor Suite Structure

Health Check with Timeout

Run Monitoring Suite

CI/CD Integration (GitHub Actions)

Load Testing

D) Async Job Polling with wait_for

Fixed Period Polling

Exponential Backoff

Async Workflow Chain

Webhook/Callback Waiting

E) Integrated Test Server

Running with Integrated Server

Health Check Protocols

Local vs CI Parity

Commands Reference

Assertion Syntax

Body Assertions

Header Assertions

Status Code Options

Protocol Examples

HTTP with Query Params and Headers

HTTP POST with JSON Body

HTTP Form Data

GraphQL with Variables

gRPC with Metadata

TCP Raw Connection

File Organization

Tips

D) Async Job Polling with `wait_for`