How we test
A change should not surprise you.
A module is a recipe, and your stack is a meal cooked from it. When we change a recipe, someone else's meal can go wrong in ways a spelling check will never see. So we check the recipe at four distances from reality, cheapest and farthest first, and get close to the real thing only when it counts.
You pin a version, so nothing reaches you by accident. This page is how we make the version you pick worth trusting.
The idea
Different mistakes are visible at different distances. Typos show up in proofreading. Design mistakes show up in rehearsal. Only reality shows you identities, permissions and quotas. So we spend the cheap checks constantly and the expensive one deliberately.
Four checkpoints.
From the lightest check to the closest copy of production.
| Checkpoint | When it runs | What it is like | What it catches |
|---|---|---|---|
| Static checks | Every change. Seconds, no cloud. | Proofreading the recipe. | Typos, wrong types, a missing required input, a setting the provider will reject, and documentation that has drifted from the code. |
| Mocked tests | Every pull request. Seconds, no cloud. | A dress rehearsal with cardboard props. | The shape of the module: its defaults, how many resources it builds, and that it refuses bad input. |
| Live acceptance | On request, and before every release. Minutes. | Cooking for real in a test kitchen, then throwing it out. | What only reality shows: permissions and trust, service limits, unsupported resource types, and timing. |
| Release contract | Always. No cloud. | Dated editions of a menu. | A change reaching you when you did not ask for it. You pin the version, the changelog says what moved, and a migration carries your state across. |
When we change a module.
- 01 The lightest checks run in seconds and tell us whether the recipe is even written correctly. Nothing real has been touched.
- 02 The mocked tests rehearse the whole thing without AWS, so a mistake in the design is caught before it costs anyone anything.
- 03 Before a release, we stand the module up for real in a sandbox account and tear it down. Never against your account, and never against production.
- 04 Only a green live run signs off the version you will pin. The changelog and migration notes travel with it.
What this means for you.
- Pin the version you have tested, and move when you choose.
- Read the changelog for the edition you are moving to.
- The generated reference cannot drift from the code, because it is checked on every change.
- Every release has been built and removed for real at least once before it is tagged.
What this does not replace.
Said plainly, so you know where it stops.
- A fresh install is not an upgrade from your current version. Read the migration notes, and test the move somewhere that is not production.
- A guard against AWS changing underneath everyone. That is why the live check is repeatable rather than a one-time event.
- Your threat model, failover testing and runbook. Those stay yours.