Certification

This provider is certified: it passes every behavior in the BigFleet conformance program — the same bar every provider must clear — so you can trust it to create, configure, drain, and delete machines correctly under load, failure, and restart. You do not need to run anything here to use it in production; this page exists if you want to reproduce that verdict yourself, locally or in your own CI.

“Certified” here means exactly what it means in the conformance program: the provider passes both the upstream authoritative baseline and this repo’s extension suite, with no failures and no skipped-as-failed behaviors.

One command

make certify-azure

That target (hack/run-certify.sh azure) is fully credential-free. It:

Resolves the bigfleet checkout that owns the authoritative contract — reusing $BIGFLEET_SRC if set, otherwise cloning the exact version pinned in the provider’s go.mod into .cache/bigfleet-src.
Builds ./bin/azure and boots it on 127.0.0.1:9099 with --provider=certify --seed-count=256. It uses --use-fake-backend, so no Azure account is touched — the extension suite consumes a fresh machine per behavior, hence the generous seed.
Runs the upstream baseline (test/conformance/ in the bigfleet repo), then the extension suite (conformance/suite, build-tagged certify), both dialing that one endpoint.
Prints CERTIFIED: azure passed the upstream baseline + the extension suite — or fails non-zero on the first failing behavior, tearing the provider down.

Override the port with PORT=....

What the two suites check

The certification harness is a pure black-box gRPC client: it dials --addr and uses only the wire RPCs of bigfleet.v1alpha1.CapacityProvider — no providerkit imports, no process introspection. It detects what the provider supports through a Capabilities probe and skips inapplicable behaviors with a reason (never failing them).

Upstream baseline — the immovable, authoritative contract maintained in the bigfleet repo. We run it verbatim and never modify it.

Extension suite — the BigFleet conformance program: a frozen registry of 93 behaviors across 11 areas that deepens the baseline:

Area	What it certifies
Lifecycle & state machine	residue-free round-trips; per-edge transitional, cluster, and host invariants
Transition matrix / errors	the out-of-position matrix, idempotent no-ops, code discipline, edge inputs
Fencing	fence-before-everything, per-`(shard_id, machine_id)` isolation, exhaustive `(epoch, sequence)` ordering
Concurrency & idempotency	N parallel retries collapse to one `operation_id` and exactly one effect
Metadata	`shard_metadata` verbatim echo, clear-on-drain, clean replace
Field shape & cost	top-level `instance_type`/`zone`/`capacity_type`; price ≥ 0; `interruption_probability` ∈ [0,1], > 0 for SPOT
List, revision & pagination	filters, `max_results`, `since_revision` deltas, completeness at scale
Timeouts & failure	actuator error / timeout → `FAILED` + `last_error`; a late completion is discarded
Durability / restart	fence marks, idempotency, bindings, and inventory survive a kill + restart
Scale & soak	large inventory, churn-soak, latency budgets, parallel throughput
Property / fuzz	seeded-random lifecycle / fencing / metadata oracles

The field-shape area is where the Azure provider’s design pays off: its Spot machines carry a real, non-zero eviction forecast (see Pricing & interruption), so the SPOT-interruption_probability > 0 assertion holds by construction.

Profiles the Azure provider claims

The harness certifies a provider against the profiles it advertises; behaviors outside a claimed profile skip-as-pass:

core — every provider (lifecycle, errors, fencing, concurrency, metadata, field-shape, list, property).
cloud — implements Delete (Idle → Speculative). The Azure provider does (Delete = VirtualMachines.BeginDelete).
spot — exposes SPOT capacity, so the SPOT interruption rigor applies. The Azure provider offers Spot VMs.
fault — failure / timeout → FAILED handling (from providerkit, by construction).
durable — fence marks, idempotency, bindings, and inventory survive a kill + restart against a --state file.
scale — large inventory, since_revision deltas, churn-soak, latency budgets.

The Azure provider does not claim bare-metal: standalone Azure VMs are always billed, so it never serves a free pool where Delete is meaningless.

make certify-azure runs the credential-free core gate (baseline + the black-box extension). The complete certification — all 93 behaviors across every lane — runs through the bfconformance runner and emits a JUnit + JSON report:

make report-azure PROFILE=core,cloud,spot,fault,durable,scale
# -> VERDICT: CERTIFIED

Certifying a real endpoint

make certify-azure certifies the fake backend in CI. To certify the provider talking to real Azure, run it against your subscription and point the extension suite at it:

# 1. Boot the provider against real Azure (see Install & deploy / Configuration).
./bin/azure \
  --addr 127.0.0.1:9099 \
  --location eastus \
  --subscription-id 00000000-0000-0000-0000-000000000000 \
  --resource-group bigfleet-eastus \
  --subnet-id /subscriptions/.../subnets/nodes \
  --offerings ./offerings.json

# 2. In another shell, run the extension suite against that endpoint.
go -C conformance test -tags=certify -count=1 ./suite/... -target=127.0.0.1:9099

To also run the upstream baseline against the same endpoint, point go test at your bigfleet checkout:

go test -tags=conformance -count=1 -run '^TestConformance_' \
  ./test/conformance/... -target=127.0.0.1:9099   # run from your bigfleet repo

A real run exercises the full lifecycle — VM create → Configure/Drain extensions → Delete — so the endpoint needs the role on the Credentials page. It will create and destroy real VMs; certify in a throwaway subscription or a dedicated test resource group.

Certification

One command

What the two suites check

Profiles the Azure provider claims

Certifying a real endpoint

See also