How-to: Enhance your Agentic workflows with Telemetry Data!

Why Your Agent Workflows Are Flying Blind

You've built an AI agent in Xano. It runs, it processes tasks, it does something — but beyond a success or failure response, you have no idea what's actually happening inside. How many steps did it take? Did it retry a tool call three times before succeeding? Which route did it choose when it had options? What was it "thinking" when it picked one user over another for a task assignment?

Without visibility into that inner loop, debugging is guesswork and optimization is impossible. You're essentially handing your agent the wheel and hoping it drives straight. That's fine when you're prototyping, but the moment your agent is doing real work for real users, flying blind becomes a real liability.

This is exactly the problem that agent telemetry solves — and Xano has made it genuinely easy to set up.

What Agent Telemetry Actually Gives You

Telemetry, in the context of AI agents, means collecting observability data about what your agent does at runtime. Not just the final output, but the entire trace: every tool call, every decision branch, every retry, every intermediate result.

Think of it like application performance monitoring, but for your agent's reasoning process. Instead of just knowing that a function returned a value, you can see the full chain of events that led to that value.

Xano currently supports three telemetry integrations out of the box: LangSmith, Langfuse, and Braintrust. More integrations are on the way. Each of these platforms has its own UI and feature set, but the setup process in Xano is nearly identical across all three — you mainly just need an API key and a project name.

The best part? You don't have to rebuild your agent or restructure your workflows. Telemetry plugs into your existing setup with a few clicks and a couple of environment variables.

Setting Up Telemetry in Xano Step by Step

Here's how to get this running, using Braintrust as the example.

Step 1: Create a Braintrust account and get your API key

Head to brain.dev and create an account if you don't have one. Once you're in, navigate to Settings in the top left, then find API Keys. Create a new API key and copy it immediately — you won't see it again.

While you're in Braintrust, also make note of your project name. You can find it on your project dashboard. If you need to confirm or rename it, hover over the project and select Rename Project to see the exact string. Copy that too.

Step 2: Store your credentials as environment variables in Xano

Back in Xano, go to your Settings and open your Environment Variables by clicking the blue Manage button. Add two new variables:

• One for your telemetry API key (the key you just copied from Braintrust)
• One for your project name

Storing them as environment variables keeps your credentials secure and makes it easy to swap services later without touching your agent configuration directly.

Step 3: Enable the OpenTelemetry integration on your agent

Now navigate to the AI tab in Xano and select Agents. Open the agent you want to monitor. In the bottom right of the agent's details card, you'll see an OpenTelemetry Integration option. Click it to slide out the configuration panel.

Toggle the integration to Enabled, then select your destination — in this case, Braintrust. Xano will prompt you for the relevant fields:

• API Key → your telemetry environment variable
• Project Name → your project name environment variable

Hit Save. That's it. Your agent is now instrumented.

Step 4: Run your agent and check the logs

Trigger your agent as you normally would — run a function, process a task, whatever your workflow does. Then head back to Braintrust and open your project. You'll see a live log populating with the agent's full trace: tool calls, queries, user lookups, updates, the works. You can drill into each step and see exactly what happened and in what order.

Things That Can Trip You Up

A few things worth knowing before you dive in:

Variable naming matters. When you reference your environment variables inside the telemetry configuration, make sure the names match exactly what you saved. A typo in the variable name means Xano will send an empty or null value to the integration, and your telemetry just won't work — often without an obvious error.

Project names are case-sensitive. Braintrust (and other platforms) can be particular about matching project names. Copy the name exactly as it appears in the dashboard rather than typing it from memory.

Don't assume all three platforms behave the same. LangSmith, Langfuse, and Braintrust all capture similar core data, but their UIs, filtering capabilities, and depth of insight vary. If you evaluate one and feel like it's missing something, try another before assuming the integration is broken.

Telemetry adds a small amount of overhead. It's minimal and generally negligible, but if you're running agents at high volume, be aware that tracing data is being sent externally. This shouldn't be a dealbreaker — it's standard for observability tooling — but it's worth knowing.

What You Actually Gain When You Can See Inside Your Agent

Once telemetry is running, something shifts in how you work with agents. You stop guessing and start seeing. That slow agent that you thought had a prompting problem? Turns out it's retrying a tool call four times because of a flaky data condition. That task assignment that seemed random? You can now trace exactly which users were queried, in what order, and why one was selected over another.

This kind of visibility is what separates agents you can trust in production from agents you're nervous about. You can catch inefficiencies before they become user-facing problems. You can validate that your agent is following the logic you intended. And when something does go wrong, you have a full trace to diagnose rather than a blank stare at an output that doesn't make sense.

The setup takes maybe five minutes. The payoff is the ability to actually understand — and improve — what your agents are doing every time they run.