0

Testing

Run deterministic tests, transport checks, and opt-in live provider tests.

Te
Testing
Chapter 020ReleaseQualityIndex
01Orientation

This project separates tests into tiers so mixed environments are more reliable.

Use this when

Run deterministic tests, transport checks, and opt-in live provider tests.

  • Test tiers
  • Recommended local commands
  • Reliability notes
  • CI parity
02 / syntax starter
pytest -q -m "not integration and not live"
03 / manual

Read the system

The complete source manual—syntax, examples, linked references, edge cases, and implementation notes.

Documentation index ↗

This project separates tests into tiers so mixed environments are more reliable.

Test tiers

  • fast unit tests: deterministic, local-only, no live provider requirements
  • integration tests: transport/socket/conformance style tests
  • live tests: real external provider/network credentials (opt-in)

Markers:

  • integration
  • live

Run fast tests only:

bashpytest -q -m "not integration and not live"

Run integration tests:

bashpytest -q -m "integration and not live"

Run everything except live:

bashpytest -q -m "not live"

Reliability notes

  • pytest.ini enforces:
  • --strict-markers
  • workspace-local base temp root (tmp/pytest_tmp)
  • collection hygiene (norecursedirs ignores transient and lock-prone paths)
  • tests should prefer workspace temp roots (tmp/...) over OS global temp paths
  • cleanup logic should be permission-aware and retry-safe on Windows/sandboxed filesystems
  • network/server startup checks should use wait loops instead of fixed sleeps where possible

CI parity

CI runs:

  • fast tests on ubuntu, windows, macos
  • integration tests on representative ubuntu + windows

This catches common path/permission/socket differences early.

01

Read

Understand where Testing sits in the agent loop before adding abstractions.

02

Wire

Connect the smallest useful provider, registry, store, or event stream first.

03

Observe

Expose events, logs, results, and failure states while the runtime is still moving.

04

Harden

Add policy, tests, retries, and audit traces after the behavior is visible.

Next docMTP Native Tool Call Syntax