AI agents
Building, tracing, and evaluating LLM agents: observability, eval harnesses, and verification patterns for systems where a model chooses the actions.
Auditing tool-call traces in MCP agents
An outcome grader tells you the refund never happened. Only the trace tells you which of the four layers between the model and the payment API broke.
How to measure a grader
A grader is a role in an eval rather than a technology. Deterministic checks and LLM judges both fill it, and each carries a different measurement cost.
The state of the OpenTelemetry GenAI semantic conventions (July 2026)
The OTel GenAI conventions moved to a dedicated repository with no versioned release yet, nothing GenAI-specific is stable, and frameworks emit several generations at once. The current map.
How to build an eval harness for LLM agents
A working recipe for evaluating agentic workflows: isolated trials, outcome-first grading, invariant gates, reliability across repeated runs, and a benchmark you can trust.
Tracing a local LLM agent end to end: Strands Agents, Ollama, and OpenTelemetry
Instrument a fully local tool-calling agent with OpenTelemetry: Strands Agents plus Ollama, traces in Jaeger or Grafana, no cloud APIs and no observability SaaS.
Agentic systems change what engineers review
When a model chooses tools and actions at runtime, requirements and code review do not disappear. Engineers review the decision environment: tools, permissions, traces, and evals.
planner-lab: a financial planning agent where the LLM never does the math
planner-lab turns a typed household case file into a retirement-readiness memo. Deterministic code does the math; a critic gate blocks any memo with untraceable numbers.
APAB 0.3.0: observability and provenance for agentic phased-array design
APAB 0.3.0 makes agentic phased-array design auditable: OpenTelemetry traces of every MCP tool call, runtime provenance manifests, real LLM providers with uniform cost tracking, and a golden-task eval harness.
An agentic workflow for phased-array design, from unit cell to link margin
I built APAB, an LLM that orchestrates a chain of phased-array physics tools through MCP and emits an auditable run bundle. The agent drives; the solvers compute.