Skip to content

Testing

This page explains how the product is tested: which layers exist, what each one runs against and what it proves. It is part of section 9 of the arc42 template.

The rules that matter most in Ochema are not visible in a screen. A company must never read the data of another; a message must never go out without consent; a write must never happen without an audit row. A test that replaces the database with a stand-in cannot prove any of them, because the rule lives in the database. And the product depends on providers of SMS, email and calendars that a test run cannot call.

  • A rule the database enforces is tested against a real PostgreSQL, connected as the same role the application uses.
  • Every area has a test that proves company A can neither read nor change the data of company B.
  • Everything outside the system is replaced by a local stand-in, so the whole suite runs with no provider account and no network.
  • A change ships with its tests. Generated files have a test that fails when they are out of date.
  • A query behind a list, a panel or a search has its execution plan checked against a seed of realistic volume.

The counts are test cases declared in the code today (it(...) and test(...) blocks; a parameterized block counts once and runs more cases than that).

Layer What it runs against, and where Files Test cases
Database A real PostgreSQL with every migration applied. In packages/db/test. 49 858
API The API in process, a real PostgreSQL and a real Redis, with simulators for the providers. In apps/api/test. 162 2,086
Worker The storage setup of the worker process. The queue consumers are tested in the API layer, where they live. In apps/worker/test. 1 2
Office system The React screens in jsdom, with the generated client replaced at its boundary. In apps/web/test. 81 737
Components The components of the design system in jsdom. In packages/ui/test. 9 125
Careers site The pages as the built server answers them, against a stand-in for the API. In apps/careers/test. 9 111
Landing, driver app, documentation The built pages and the screens. In the test folder of apps/landing, apps/driver and apps/docs. 14 76
Shared packages The request helper, the generated client and the lint rules. In the test folder of packages/app-kit, packages/contracts and packages/config. 9 20
End to end The built office system, careers site and landing site in Chromium, against the real API, worker, database and Redis. In e2e/tests. 44 63

Vitest runs every layer except the last, which is Playwright. In total: 334 files and 4,015 declared cases in Vitest, and 44 files and 63 scenarios in Playwright.

packages/db/test connects as ochema_app, the role of the application, and as the other roles when the test is about them. It covers:

  • Isolation by area (isolation*.test.ts, nine files): for each table, reading, writing and foreign keys across two companies, the exact privileges of each role per table and per column, and the policies.
  • Roles and privileges (privileges.test.ts): no runtime role is a superuser or can bypass row level security, and every table has row level security enabled and forced.
  • The functions that cross the boundary (gate.test.ts, public-context.test.ts, wave*-functions.test.ts, staff-*.test.ts): what each definer function returns, to whom, and what it refuses.
  • Upgrades (upgrade*.test.ts, 14 files): each migration that changes existing data is applied over a database that already has companies, and the result is checked.
  • Plans (volume-plans.test.ts): the search functions keep every company to itself and use a plan per call.

See Tenant isolation.

apps/api/test starts the real application and calls it over HTTP. global-setup.ts recreates a dedicated database with the migrations before each run, so the development database is never touched. Each test file creates its own companies and uses its own Redis key prefix, which lets the files run in parallel. Six files that change rows shared by every company, such as the catalog of compliance rules, run afterwards, one at a time (apps/api/vitest.config.ts).

Besides the behaviour of each route, the suite has tests of a specific kind:

Kind What it proves, with examples
Isolation A session of one company gets nothing of another through the API. isolation.test.ts, and the cases with two companies in the test files of the modules.
Access Every route declares who may call it, and each role gets exactly its permissions. route-boundary.test.ts, role-access.test.ts, permissions.test.ts.
Concurrency Simultaneous requests end in a correct state, never in an error 500. scheduling-concurrency.test.ts (200 bookings of one time: one interview, 199 refusals) and lead-distribution-concurrency.test.ts (300 leads among three recruiters: 100 each).
Volume A routine keeps its totals over thousands of rows. import-volume.test.ts, broadcast-volume.test.ts, automation-volume.test.ts, compliance-engine-volume.test.ts.
Timing A promised time is kept on the real worker. first-response-timing.test.ts, domain-events.test.ts.
Provider adapters The adapter of each real provider, against local stand-ins that answer what the documentation of the provider shows. No test calls the network. twilio-send.test.ts, twilio-webhook.test.ts, ses-send.test.ts, smtp-send.test.ts, calendar-adapters.test.ts.
Architecture The structure of the code. lint-boundary.test.ts and module-cycles.test.ts for the module boundaries, openapi.test.ts for the contract.

The screens of the office system run in jsdom with Testing Library. The tests replace the functions of @ochema/contracts, the generated client, and nothing below or above it (apps/web/test/api-mock.ts), so a screen is tested with the real types of the contract. Elements are found by role and accessible name, the way a screen reader finds them.

e2e/tests drives a browser against the built apps and the real API and worker, with a database recreated for the run. journey.spec.ts follows a new customer through the screens a person uses, from the landing page to a driver who is eligible to drive and the audit export of the driver file. Other scenarios cover one area each, the top bar at 390 pixels wide (narrow.spec.ts) and two tabs in different companies (two-tabs.spec.ts). Every scenario also fails when the browser console shows an error or a warning (console-guard.spec.ts proves the guard itself).

Outside system Stand-in in tests and in development
SMS provider A simulator that keeps the message and marks it delivered; test routes inject replies. In apps/api/src/modules/platform-messaging.
Email provider An outbox that writes each email to a local folder. In the same module and in platform-mail.
Calendar providers An in-memory gateway, fake.gateway.ts in apps/api/src/modules/platform/calendar, which production refuses.
Object storage An S3-compatible server in development (docker-compose.yml); an in-process fake in the API tests (apps/api/test/fake-s3.ts).

See ADR-0013.

pnpm --filter @ochema/db seed:volume fills a separate database with synthetic companies at a chosen scale. The scripts explain:wave01 and explain:wave04 of the same package run the SQL of the critical queries of the API against that database, as the application role with a company in context, and report the plan and the time of each one. The measurements taken this way, with their conditions, are in Quality requirements.

scripts/check.sh runs the same steps on a developer machine and in continuous integration, in this order: format, lint, type check, tests, build, and the end to end suite when asked. The workflow .github/workflows/ci.yml runs them and also builds the container images.

Lint is part of the test strategy. Three rules described elsewhere are enforced by lint and not by review: module boundaries (Building blocks), the generated client (API contract) and the design tokens (Design system).

  • No real provider is called. The adapters for Twilio, Amazon SES, SMTP, Google Calendar and Microsoft Graph are tested against local stand-ins. Sending through a real account is Not verified.
  • No load test. The suite proves correctness under concurrency and the plans of the critical queries. It does not measure the API under many simultaneous users, and it never runs more than one instance of the API or of the worker.
  • Code coverage is not measured. No coverage tool is configured.
  • One browser. The end to end suite runs in Chromium only.
  • Accessibility is tested by role and name, not audited. No automated accessibility scanner runs, and no screen reader pass is recorded. See Quality requirements.
  • Timing tests depend on the machine. The measurements come from a development machine and are stated with that condition.