Validator Pro

When to use Pro vs Free

Same metric definitions, same Python API surface — only the dataset and runtime differ. You can switch from free to pro by adding api_key= to your existing call.

Getting an API key

1. Sign in (or sign up) for an AUSynth account.
2. Buy a credit bundle from /pricing if you don't already have credits.
3. Open your account dashboard → API keys → Create new key. Copy + store in a secrets manager.
4. Export it for local use: export VEROSYNTHEA_API_KEY="..."

Keys are scoped to your account. Rotating a key invalidates the previous one — any running validation jobs continue to completion unaffected.

How the async run works

A pro run takes 5–15 minutes — too long for a synchronous HTTP response. The validator submits the job, gets a job ID back immediately, then polls the status endpoint until the report is ready.

The Python library handles steps 2 and 3 automatically via wait_for_completion() — default 30-min timeout. If runtime exceeds 30 minutes the job is auto-failed and the 50 credits are automatically refunded.

Python: a single pro run

import os
from verosynthea_validator import assert_fair

result = assert_fair(
    model=my_trained_model,
    api_key=os.environ["VEROSYNTHEA_API_KEY"],
    tier="pro",                       # "free" (default) or "pro"
    target_column="income_above_threshold",
    protected_attributes=[
        "sex", "country_of_birth", "age_group",
    ],
    metric="demographic_parity",
    threshold=0.10,
    # Optional — defaults to 30 min, matches server-side timeout
    wait_for_completion=True,
    poll_interval_seconds=15,
    timeout_seconds=30 * 60,
)

if not result.passed:
    for group in result.flagged_groups:
        print(group.attribute, group.values, group.gap)

On success, result contains the per-attribute counterfactual gap distribution, demographic parity, disparate impact, the worst-slices table, and (Pro) rank parity, distribution distance, and variance share, plus a list of any attributes whose weighted mean gap exceeded the threshold. On failure (model error, schema mismatch, timeout) credits are refunded and the exception carries the failure reason.

CI/CD gate

A pro run as a deployment gate: fail the pipeline if the model drifts past a fairness threshold against the full national dataset. Runs in ~10 min, fits inside most CI step timeouts.

# .github/workflows/fairness-gate.yml
name: Fairness gate
on: [push]
jobs:
  audit:
    runs-on: ubuntu-latest
    timeout-minutes: 35   # validator times out at 30; leave a small buffer
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-python@v6
        with: { python-version: "3.11" }
      - run: pip install verosynthea-validator
      - name: Audit against full AUSynth
        env:
          VEROSYNTHEA_API_KEY: ${{ secrets.VEROSYNTHEA_API_KEY }}
        run: |
          python -m my_pipeline.audit \
            --tier pro \
            --threshold 0.10 \
            --fail-on-gap

Store the API key in your CI secrets manager (GitHub Actions secrets, GitLab CI variables, etc.). Each push triggers one audit — that's 50 credits per push on the gated branch, so most teams gate only main / release branches.

Pricing

← Back to the main validator docs (install, quickstart, metric definitions, comparison to fairlearn / aif360).