Back to Documentation

CI/CD Integration

AutoSmoke integrates with your existing CI/CD pipeline so tests run automatically on every pull request. No browser infrastructure to manage — just add a config file and start catching regressions before they ship.

GitHub Actions

Add the following workflow file to your repository at .github/workflows/smoke-tests.yml:

name: Smoke Tests
on:
  pull_request:
    branches: [main]
  push:
    branches: [main]

jobs:
  smoke-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run AutoSmoke Tests
        uses: autosmoke/action@v1
        with:
          api-key: ${{ secrets.AUTOSMOKE_API_KEY }}
          url: ${{ vars.STAGING_URL }}

The action will:

  1. Trigger all tests associated with your project
  2. Wait for them to complete
  3. Post a status check with pass/fail results
  4. Comment a summary with screenshots on the PR

GitLab CI

Add to your .gitlab-ci.yml:

smoke-tests:
  stage: test
  image: node:20-slim
  script:
    - npx @autosmoke/cli run --api-key $AUTOSMOKE_API_KEY --url $STAGING_URL
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

Environment Variables

| Variable | Required | Description | | ------------------- | -------- | ------------------------------------------------- | | AUTOSMOKE_API_KEY | Yes | Your project API key from the AutoSmoke dashboard | | STAGING_URL | Yes | The URL of the environment to test against |

Store secrets using your CI provider's secret management — never commit API keys to the repository.

Running Against Preview Deployments

If you use Vercel, Netlify, or similar platforms that create preview URLs per PR, pass the dynamic preview URL instead of a fixed staging URL:

# Vercel example
- name: Run AutoSmoke Tests
  uses: autosmoke/action@v1
  with:
    api-key: ${{ secrets.AUTOSMOKE_API_KEY }}
    url: ${{ steps.deploy.outputs.preview-url }}

Parallel Execution

Tests run in parallel by default. For large test suites, you can configure concurrency in your project settings to control how many tests execute simultaneously.

Failure Handling

When a test fails in CI:

  • The pipeline is marked as failed
  • A detailed report is generated with screenshots at each step
  • The team is notified via your configured notification channels (Slack, email, or webhook)

Review the screenshots to quickly identify whether the failure is a real bug or a flaky environment issue.

Next Steps