AI agents
Building, tracing, and evaluating LLM agents: observability, eval harnesses, and verification patterns for systems where a model chooses the actions.
MCP transport is a lifecycle decision
All four of my open-source MCP servers default to stdio, and one flag away they speak streamable HTTP. The real decision is who owns the server process and where its files live.
A SysML v2 model that runs its own verification
One command reads a SysML v2 model, runs a phased-array study, and checks five requirements with margins. How the binding convention works and what does not survive.
Data provenance for agentic AI workflows
An agent picks its own inputs at runtime, so the list of what a result depends on can only be recorded while it happens. What that record has to hold, read from my own run bundles.
An agent designed a maritime search radar and beat my reference on cost
I gave a coding agent a calibrated X-band search radar task with nine scored requirements. It passed on the first attempt with a design trick my own solver never tried.
How many runs does an agent benchmark need?
Three attempts per arm cannot produce a significant result under Fisher's exact test, whatever the outcome. Here is the arithmetic, applied to my own published numbers, and what more attempts would cost.
The stack I'm building for agent-driven RF and antenna design
Nine repositories, four design rules, and one dependency graph with the aspirations drawn as dashed lines. What an agentic hardware design stack needs before the agent shows up.
An agent found the free lunch in my phased-array benchmark
I wrote a 2-bit phase quantization task I thought was hard. A constant 45 degree offset made it exactly solvable, and the agent found it in about fifteen minutes.
Auditing tool-call traces in MCP agents
An outcome grader tells you the refund never happened. Only the trace tells you which of the four layers between the model and the payment API broke.
How to measure a grader
A grader is a role in an eval rather than a technology. Deterministic checks and LLM judges both fill it, and each carries a different measurement cost.
The state of the OpenTelemetry GenAI semantic conventions (July 2026)
The OTel GenAI conventions moved to a dedicated repository with no versioned release yet, nothing GenAI-specific is stable, and frameworks emit several generations at once. The current map.
How to build an eval harness for LLM agents
A working recipe for evaluating agentic workflows: isolated trials, outcome-first grading, invariant gates, reliability across repeated runs, and a benchmark you can trust.
Tracing a local LLM agent end to end: Strands Agents, Ollama, and OpenTelemetry
Instrument a fully local tool-calling agent with OpenTelemetry: Strands Agents plus Ollama, traces in Jaeger or Grafana, no cloud APIs and no observability SaaS.
Agentic systems change what engineers review
When a model chooses tools and actions at runtime, requirements and code review do not disappear. Engineers review the decision environment: tools, permissions, traces, and evals.
planner-lab: a financial planning agent where the LLM never does the math
planner-lab turns a typed household case file into a retirement-readiness memo. Deterministic code does the math; a critic gate blocks any memo with untraceable numbers.
APAB 0.3.0: observability and provenance for agentic phased-array design
APAB 0.3.0 makes agentic phased-array design auditable: OpenTelemetry traces of every MCP tool call, runtime provenance manifests, real LLM providers with uniform cost tracking, and a golden-task eval harness.
An agentic workflow for phased-array design, from unit cell to link margin
I built APAB, an LLM that orchestrates a chain of phased-array physics tools through MCP and emits an auditable run bundle. The agent drives; the solvers compute.