Changelog

Release notes

What shipped, and when. Two open-source libraries on one trust thesis — SafeAct (safe actions) and wobbly (label-free output checks).

SafeAct safeact · safeact-mcp · safeact-langchain

safeact-langchain 0.1.0new package
  • Make any LangChain / LangGraph tool exactly-once with one decorator, keyed on the injected tool_call_id — hidden from the LLM schema.
  • Crash-safe under LangGraph: it persists tool_call_ids in its checkpoint, so a resume re-invokes with the same id and the side effect does not repeat.
  • Sync and async tools; durable audit trail + honest in-doubt.
safeact 0.1.0  ·  safeact-mcp 0.1.0feature
  • Async support: @safe and safe_tool auto-detect async def tools and await them — same exactly-once guarantee (agent & MCP tools are often async).
safeact-mcp 0.0.1new package
  • Make any MCP tool exactly-once and auditable in one decorator (safe_tool), composing under MCPServer.tool().
  • Honest in-doubt on a mid-action crash; durable audit record per call.
safeact 0.0.2chore
  • Packaging: repo-level Apache-2.0 LICENSE; fixed the PyPI homepage link.
safeact 0.0.1initial
  • The core: @safe exactly-once execution, honest IN_DOUBT state (never a blind retry), and Saga compensation / rollback.
  • One SQLite file, no server. Mutation-tested — every guarantee has a crash-test that fails if removed.

wobbly metamorphic testing for AI outputs

0.4.0feature
  • length_bias() + length_invariant() — catch verbosity bias (does your judge/model prefer longer answers?) with no labels.
  • wobbly judge CLI — measure position bias in your own LLM judge straight from the terminal.
0.3.0feature
  • judge_bias() — label-free position-bias measurement for an LLM-as-judge (resolves the judge's letter to the chosen answer before comparing).
  • coverage="rotate" — Latin-square positional rotation so every option visits every position (random stays the default).
  • Opt-in real-LLM validation script for order / paraphrase relations.
0.2.0feature
  • Robustness catalog: order_invariant, distractor_robust, formatting_invariant, paraphrase_invariant — each a fair relation a correct system can't fail.
  • equivalent() assertion for free-text outputs (you inject your own embedder; baseline-spread aware).
  • Baseline-variance control in check() with per-relation violation rates; assert_robust() pytest/CI helper; cached_system() wrapper; a minimal wobbly CLI.
0.1.0initial
  • Metamorphic testing core — check + Relation + Report: find wrong AI outputs with no answer key.
  • The 535-receipt experiment (flagged outputs 2.7× more likely wrong, zero labels); unchanged() works on any output type; non-receipt classifier example.