CI/CD Integration

Run AutoSmoke tests automatically on every pull request using GitHub Actions, GitLab CI, or any CI/CD platform.

AutoSmoke integrates with your existing CI/CD pipeline so tests run automatically on every pull request. No browser infrastructure to manage — just add a config file and start catching regressions before they ship.

GitHub Actions#

Add the following workflow file to your repository at .github/workflows/smoke-tests.yml:

name: Smoke Tests
on:
  pull_request:
    branches: [main]
  push:
    branches: [main]

jobs:
  smoke-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run AutoSmoke Tests
        uses: autosmoke/action@v1
        with:
          api-key: ${{ secrets.AUTOSMOKE_API_KEY }}
          url: ${{ vars.STAGING_URL }}

The action will:

Trigger all tests associated with your project
Wait for them to complete
Post a status check with pass/fail results
Comment a summary with screenshots on the PR

GitLab CI#

Add to your .gitlab-ci.yml:

smoke-tests:
  stage: test
  image: node:20-slim
  script:
    - npx @autosmoke/cli run --api-key $AUTOSMOKE_API_KEY --url $STAGING_URL
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

Environment Variables#

Variable	Required	Description
`AUTOSMOKE_API_KEY`	Yes	Your project API key from the AutoSmoke dashboard
`STAGING_URL`	Yes	The URL of the environment to test against

Store secrets using your CI provider's secret management — never commit API keys to the repository.

Running Against Preview Deployments#

If you use Vercel, Netlify, or similar platforms that create preview URLs per PR, pass the dynamic preview URL instead of a fixed staging URL:

# Vercel example
- name: Run AutoSmoke Tests
  uses: autosmoke/action@v1
  with:
    api-key: ${{ secrets.AUTOSMOKE_API_KEY }}
    url: ${{ steps.deploy.outputs.preview-url }}

Parallel Execution#

Tests run in parallel by default. For large test suites, you can configure concurrency in your project settings to control how many tests execute simultaneously.

Failure Handling#

When a test fails in CI:

The pipeline is marked as failed
A detailed report is generated with screenshots at each step
The team is notified via your configured notification channels (email, Slack, or webhook)

Review the screenshots to quickly identify whether the failure is a real bug or a flaky environment issue.

Next Steps#

Getting Started — Set up your first test
Writing Tests — Best practices for reliable tests
Troubleshooting — Common issues and fixes

Command Palette