Evaluating LLM Workflows: A Field Guide
What mature LLM evaluation actually looks like in 2026, and why squinting at outputs in a spreadsheet doesn't count.
Read essay →I'm Matt Ferrante. For 15+ years I've shipped software in AdTech, algorithmic trading, and now AI. The kind that has to actually work on Monday morning, not just demo on Friday.
Pick the outcome you need. Each is a complete engagement, and they compose — most clients start with one and add a second.
Your buyers are asking ChatGPT, Perplexity, and Google AI Overviews for recommendations. Be the answer they get — not the competitor who got the structured data right first.
For teams whose Monday looks like 2024 even though they've 'adopted AI.' Audit the workflow, replace the tedious parts with agents and automation, and ship something your team actually uses.
Ship AI-native features, agent APIs, chat surfaces, and MCP endpoints that survive real customer traffic. Draws on 18 MCP servers in production, one at 100k requests a day.
What mature LLM evaluation actually looks like in 2026, and why squinting at outputs in a spreadsheet doesn't count.
Read essay →A checklist for making a codebase pleasant for coding agents to operate in: file layout, naming, docs.
Read essay →Most software companies think AI is a feature. The real question is whether AI can replace what you sell.
Read essay →Start with a 30-minute consult. $20, refunded if it isn't useful. Bring your worst architecture decision, a stalled rollout, or a demo you can't put in front of customers. You'll leave with a plan.