Part 3 of 7 · 2 min read
The four stages
Review, build, pre-production, production.
UDS says a change passes through four stages.
Review. A developer opens a pull request (PR). The code is compiled, checked by static analysis and run against tests, all within a few minutes. To keep tests fast and free of random failures, only small tests that fit on one process or node, such as unit tests, run at this point. Larger ones run later. A team member must approve the PR, and UDS gives a reviewer's checklist: - Are there unit, integration and end-to-end tests where each is needed? - Does it add metrics, logs and traces? - Could it break production by a backward-incompatible change, or by hitting a service limit? - Can it be rolled back safely?
Code is not the only thing to review. Cloud templates, static assets, end-to-end tests and configuration should be in version control and treated like code. A service can then have several pipelines, one per repository. UDS stresses configuration: one of the most common causes of production failures is a configuration change applied everywhere without review or testing.
Build. After the merge, the code is compiled and packed into one release artifact that can be deployed.
Pre-production. The artifact goes to an artificial environment. It lacks the realism of production but catches hard failures, such as a crash at startup because a setting is missing, and runs end-to-end tests. It is faster than production, so bugs show up earlier. There can be several such environments: one made fresh for each artifact for smoke tests, and a lasting one like production that receives a small share of mirrored requests. UDS says AWS uses three, called Alpha, Beta and Gamma. The service there should call the production endpoints of external dependencies to stay stable. Its health should be judged with the same metrics, alerts and tests used in production, so pre-production does not become second-class.
Production. Covered next.
Smoke test
A smoke test is a small, quick test run against freshly deployed software to confirm that the deployment itself worked.
Example: after deploying, call the health endpoint and one read request, and fail the pipeline if either returns an error.
Check yourself: 1. Why do only small tests run at the review stage? 2. Why treat configuration like code? 3. Why should pre-production use the same alerts as production?