MCP/Agents/Skills Meetup - January 14, 2027

Join our virtual meetup to hear talks from experts on cutting-edge topics across AI, ML, and computer vision.

In this session, Adonai will show how AI agents can take on the slow, repetitive parts of working with visual data, from finding and curating the right samples to labeling, evaluating models and tracking down failure modes. You'll see live agent workflows in the Voxel51 visual AI data platform, along with practical lessons on where agents help most, where a human should stay in the loop, and how to try it on your own data.

Your agent works perfectly in the demo. Then in production it gives a wrong answer and nobody can tell why, token costs spike, it slows to a crawl under load, and it calls a tool the user was never allowed to use. Building an agent is the easy part; the real work is running it like any other production service: tested, observable, scalable and secure.

Spring AI lets you do that with the Spring idioms and tooling you already know. This code-first session grows one example step by step. We start with a simple ChatClient, add advisors, memory and RAG, comparing the options Spring AI offers for each, and then let the agent act through tool calling and MCP.

Then come the production questions:
- Testing: we catch regressions with LLM-as-judge evaluators and typed decision models like Jev for fast, cheap verdicts.
- Observability: we add Micrometer traces, logs and token usage across model and tool calls.
- Scaling: we compare options for handling more concurrent users, from self-hosting to a managed runtime such as Amazon Bedrock AgentCore.
- Security: we use Spring Security to authenticate users and make sure only the MCP tools they're allowed to use ever get called. You'll leave with reusable Spring patterns for agents that reason and act, and the discipline to test, observe, scale and secure them in production.

Every team shipping agents is quietly standing up two systems: the agent harness that plans, retrieves, calls tools, and edits code — and the evaluation harness that decides whether any of it can be trusted. Most teams only evaluate the first one's final output.

That is not enough. An agent can produce a correct answer through an unreliable, wasteful, or out-of-policy execution path, and outcome-only checks will pass it. This talk argues for evaluating the whole agent system: execution trajectories — tool correctness, evidence and grounding, state handling, policy compliance, and efficiency — alongside final outcomes, using deterministic specification checks an agent cannot game, model-based judges, and human-calibrated benchmarks.

We separate capability evaluations from regression evaluations so harness or model changes can be gated in CI. Then we turn the lens around: the evaluators themselves need evaluating — LLM judge calibration, disagreement analysis, and where human review still belongs.

Attendees leave with a production checklist for gating agent, model, and harness changes on system-level evidence instead of final answers.