µLang v1

Guides

Testing contracts

A contract is code, so test it like code. Decision tests state, in the application's own terms, what each rule is for.

test/deploy.test.tsexample project
// Decision tests for policy/deploy.ul. Run: node --test test/*.test.ts
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import { test } from 'node:test';
import { loadPolicy } from '../src/policy.ts';

const policy = loadPolicy(readFileSync(new URL('../policy/deploy.ul', import.meta.url), 'utf8'), 'policy/deploy.ul');
const green = JSON.parse(readFileSync(new URL('../facts/green.json', import.meta.url), 'utf8')).DeployRequest;
const decide = (action: string, change: Record<string, unknown> = {}) =>
  policy.decide(action, { DeployRequest: { ...green, ...change } });

// Changing the contract changes this digest: the test fails until someone reviews and updates it.
test('the reviewed contract is the one being tested', () => {
  assert.equal(policy.digest, '686358ddc154b77b5b7b3a24870b8dd6f79b23d376ed039d6824a923284e4059');
});

test('a green, reviewed deploy is permitted', () => {
  assert.deepEqual(decide('deploy'), { verdict: 'PERMIT', clause_id: 'ProductionDeploy.deploy.permit', contract_sha256: policy.digest });
});

const denials: [string, Record<string, unknown>, string][] = [
  ['change freeze in production', { change_freeze: true }, 'change_freeze'],
  ['author approved their own change', { author_approved_own_change: true }, 'no_self_approval'],
  ['error budget exhausted', { error_rate_pct: 2.5 }, 'error_budget'],
  ['tests failing', { tests_passed: false }, 'tests_green'],
  ['one approval', { approvals: 1 }, 'two_approvals'],
];
for (const [name, change, clause] of denials) {
  test(`deny: ${name}`, () => {
    assert.deepEqual(decide('deploy', change), { verdict: 'DENY', clause_id: `ProductionDeploy.${clause}`, contract_sha256: policy.digest });
  });
}

test('the freeze applies to production only', () => {
  assert.equal(decide('deploy', { change_freeze: true, environment: 'staging' }).verdict, 'PERMIT');
});

test('rollbacks are not blocked by the freeze', () => {
  assert.equal(decide('rollback', { change_freeze: true }).verdict, 'PERMIT');
});

test('bad facts are rejected, not decided', () => {
  assert.deepEqual(decide('deploy', { approvals: '2' }), { error: 'invalid type or domain for DeployRequest.approvals' });
});
example projectreal output · timings elided
$ node --test --test-reporter=spec test/*.test.ts
✔ the reviewed contract is the one being tested (… ms)
✔ a green, reviewed deploy is permitted (… ms)
✔ deny: change freeze in production (… ms)
✔ deny: author approved their own change (… ms)
✔ deny: error budget exhausted (… ms)
✔ deny: tests failing (… ms)
✔ deny: one approval (… ms)
✔ the freeze applies to production only (… ms)
✔ rollbacks are not blocked by the freeze (… ms)
✔ bad facts are rejected, not decided (… ms)
ℹ tests 10
ℹ suites 0
ℹ pass 10
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms …

What to test

Pin the digest

The first test fails whenever the contract changes in a way that could affect a decision, which makes updating the expected digest an explicit, reviewable step. Layout and comment changes keep the digest, so they do not trip it.

check prints the digest:

example projectreal output
$ ./ulang-lite check policy/deploy.ul
policy/deploy.ul: ok ProductionDeploy (6 clauses; actions deploy, rollback; sha256 686358ddc154b77b5b7b3a24870b8dd6f79b23d376ed039d6824a923284e4059)