A short and sweet guide to choosing an agent observability platform
Most buyer guides for agent observability are marketing from the platforms themselves. Here is our short and sweet take on which platform to pick, depending on your stack, budget and industry.
Here at Moyai, being agent reliability people, we’re sometimes asked which observability platform we recommend for people building agents. When I went online to find a buyer’s guide, it felt like every resource was a marketing tool from an agentic observability platform. So if you’re in the market, here’s my honest opinion on picking an observability platform.
Let’s keep it short and sweet.
| If you… | Pick | Why |
|---|---|---|
| Use a major LLM SDK (OpenAI, Vercel AI, Anthropic) and want a strong free tier | Langfuse | OTel-native, familiar UX, free up to 50k traces/mo (2 seats), then $29 for 100k + $8 per extra 100k |
| Want to self-host | Langfuse (self-hosted) | Open source and feature-complete, Helm chart for Kubernetes, BYO inference (needs Postgres, ClickHouse, Redis, S3) |
| Build on LangChain/LangGraph | LangSmith | One-click instrumentation and native ecosystem integration; not the cheapest |
| Build on Pydantic / FastAPI | Pydantic Logfire | Python-first, near-zero setup for Pydantic projects |
| Work in a regulated industry or need real-time guardrails | Galileo AI | Governance-first, stops agents before mistakes; premium pricing |
Default: Langfuse. Since it’s OTel-based, you can switch providers later if pricing changes.
And now the details, if you’re interested.
Path 1:
The best for most people
Generous free tier, budget-friendly scaling
AND/OR
You build with one of the major LLM SDKs (OpenAI SDK, Vercel AI SDK, Anthropic SDK, etc.)
You need:
- OTel compatibility to avoid vendor lock-in after you scale
- A simple UX that gets you from identifying a customer issue to finding the relevant trace as quickly as possible
- A good enough free tier that you can onboard your first customers without paying
Our choice: Langfuse
- Langfuse is OTel native, and offers a generous free tier (2 seat limit, 50k traces per month).
- The UX is familiar if you’ve used observability platforms before, and has a modest learning curve if you haven’t.
- You get 100,000 traces/month for $29, and then $8 for every 100,000 after that.
- Fully open-source.
- In general, Langfuse tends to be the go-to option in the space.
Path 2:
You want to self-host your observability
You need:
- OTel compatibility
- An open-source project that won’t get rug-pulled
Our choice: Self-hosted Langfuse
- Langfuse offers a feature-complete self-hosted solution. You’ll need to host the following components: Postgres, ClickHouse, Redis and S3/blob storage.
- You can BYO inference for integrated AI tools.
- Langfuse maintains a Helm chart for simple deployment via Kubernetes.
Path 3:
You build in the LangChain/LangGraph ecosystem
You need:
- Tooling that plugs directly into the services you already use
- One-click instrumentation for your LangChain/LangGraph project
- A development team that’s building specifically for your use case
Our choice: LangSmith
- While LangSmith isn’t the cheapest, it’s hard to beat if you’re already in the ecosystem.
- LangSmith integrates directly with all the tools you already use.
Path 4:
You build in the Pydantic ecosystem
You need:
- Tooling that plugs directly into the services you already use
- Python-first
Our choice: Pydantic Logfire
- Our personal choice, given our Python-driven stack at Moyai
- Great tooling for FastAPI-heavy projects
- Almost zero lift for Pydantic projects
Path 5:
You work in a regulated industry and/or need real-time guardrails
You need:
- Guardrails that work in real time and stop your agent before it makes a mistake
- Governance-first observability
- An observability platform that understands regulated environments
Our choice: Galileo AI
- Galileo (now owned by Cisco) focuses on real-time guardrails.
- Not a budget option, but fills a niche and is well reviewed.
Conclusion
Langfuse comes out on top of a lot of these options because of its competitive pricing, solid product offering, and open-source foundation. That being said, it’s not unheard of for sweet-spot offerings like Langfuse to change their pricing once they’re dominant. The beauty of OTel is that regardless of what happens, you’ll always be able to switch providers down the line if you need to.
And here’s the obligatory Moyai promo (that might actually save you a lot of time):
What we find when we talk to our customers is that while observability helps customers track down what went wrong in individual cases, agent builders still feel blocked on real QA.
Building an agent that’s reliable enough to trust with hard-won customers takes more than whack-a-mole-ing the few issues you can track down. Moyai integrates with the telemetry you already have, sorts through all your traces, and gives you a prioritized list of issues that you or your agent can solve right now.
Rather than making your customers your QA, tackling individual issues and getting nowhere, Moyai lets you zoom out and address real behavioral patterns. Here’s a no-nonsense walkthrough of the workflow if you’re interested: moyai.ai/demos/customer-support
Agents look fine in demos. Production disagrees.
Moyai helps teams catch reliability issues early with clustering, evaluation and actionable alerts — so a failure surfaces before a customer or a regulator finds it for you.
If that sounds like the kind of tooling you want to run — try Moyai or join us on Discord.
We build Moyai for teams running AI in production: practical tools, fast feedback loops, and reliability you can operate.