Write tests

Orchard tests should verify workflow code and orchestration contracts. Unit tests can use fake apps; runtime integration tests can use createLocalRuntime() directly — no database or shared fixture needed.

Run deterministic smoke checks

Use deterministic checks before spending time or quota on real Codex/Cursor/Claude calls:

pnpm check:cli
pnpm --filter @snevins/orchard-cli exec vitest run --config vitest.config.ts test/workflow-artifacts.test.ts

These tests cover generation, repair, preview, source-hash validation, and artifact typechecking with fake harnesses. They are the no-LLM smoke path for the generated-artifact loop.

Test generated artifacts statically

Use the CLI package artifact generation/preview internals for code-mode behavior in repository tests:

const artifact = await generateWorkflowArtifact({
  prompt: 'summarize a repo',
  harness: fakeHarness,
  cwd: process.cwd(),
  workflowDir: tmpdir,
})

expect(artifact.validation.taskResolution.ok).toBe(true)

These tests assert that generated TypeScript imports allowed dependencies, exports registerWorkflow, typechecks, avoids top-level side effects, and uses queue-aware workflow/task registration.

Test workflow primitives with fake apps

For task, workflow, spawnAndAwaitTask, and grouped run options, use fake runtime apps (typed as Orchard's Absurd interface) that record registrations and spawns when a test should not exercise real task scheduling. Keep the test about the Orchard contract, not runtime internals.

Test harness calls with fake harnesses

const harness: Harness = {
  name: 'fake',
  async runStructuredTurn() {
    return {
      sessionId: 's1',
      text: JSON.stringify({ summary: 'ok', approved: true }),
    }
  },
  async runUnstructuredTurn() {
    throw new Error('structured turn should be used')
  },
}

Then assert harnessCall validates output, records through ctx.step, and resumes the same session during repair.

Test workflow execution with the in-process runtime

Use createLocalRuntime — exported from @snevins/orchard-core — when a test needs a real in-process runtime app instead of a fake one. Task registration, spawn, ctx.step memoization, retries, and the event bus all run for real, in memory:

import { createLocalRuntime } from '@snevins/orchard-core'

const runtime = createLocalRuntime()
try {
  const workflow = registerWorkflow({
    app: runtime.app,
    harness: fakeHarness,
    cwd: process.cwd(),
    queues: { workflow: 'test-workflow', tasks: 'test-tasks' },
  })
  const run = await workflow.start({ prompt: 'test' })
  const snapshot = await runtime.awaitRootTask(run.taskID)
  expect(snapshot.state).toBe('completed')
} finally {
  await runtime.close()
}

Each createLocalRuntime() call is a fresh, isolated instance: tests get isolation for free, but there is nothing to assert about persistence — runtime.close() discards all in-memory state.

What not to test in Orchard

Do not recreate a scheduler, worker pool, or cross-process retry engine in Orchard tests — the in-process runtime's execution model (single process, ctx.step memoization, attempt-based retries) is the actual contract to test against, not a stand-in for one.

Related

• Run real-harness tests
• Inspect a local run