Launch HN: Relvy (YC F24) – On-call runbooks, automated (relvy.ai)
34 points by behat 7 hours ago
Hey HN! We are Bharath, and Simranjit from Relvy AI (https://www.relvy.ai). Relvy automates on-call runbooks for software engineering teams. It is an AI agent equipped with tools that can analyze telemetry data and code at scale, helping teams debug and resolve production issues in minutes. Here’s a video: [[[https://www.youtube.com/watch?v=BXr4_XlWXc0]]]
A lot of teams are using AI in some form to reduce their on-call burden. You may be pasting logs into Cursor, or using Claude Code with Datadog’s MCP server to help debug. What we’ve seen is that autonomous root cause analysis is a hard problem for AI. This shows up in benchmarks - Claude Opus 4.6 is currently at 36% accuracy on the OpenRCA dataset, in contrast to coding tasks.
There are three main reasons for this: (1) Telemetry data volume can drown the model in noise; (2) Data interpretation / reasoning is enterprise context dependent; (3) On-call is a time-constrained, high-stakes problem, with little room for AI to explore during investigation time. Errors that send the user down the wrong path are not easily forgiven.
At Relvy, we are tackling these problems by building specialized tools for telemetry data analysis. Our tools can detect anomalies and identify problem slices from dense time series data, do log pattern search, and reason about span trees, all without overwhelming the agent context.
Anchoring the agent around runbooks leads to less agentic exploration and more deterministic steps that reflect the most useful steps that an experienced engineer would take. That results in faster analysis, and less cognitive load on engineers to review and understand what the AI did.
How it works: Relvy is installed on a local machine via docker-compose (or via helm charts, or sign up on our cloud), connect your stack (observability and code), create your first runbook and have Relvy investigate a recent alert.
Each investigation is presented as a notebook in our web UI, with data visualizations that help engineers verify and build trust with the AI. From there on, Relvy can be configured to automatically respond to alerts from Slack
Some example runbook steps that Relvy automates: - Check so-and-so dashboard, see if the errors are isolated to a specific shard. - Check if there’s a throughput surge on the APM page, and if so, is it from a few IPs? - Check recent commits to see if anything changed for this endpoint.
You can also configure AWS CLI commands that Relvy can run to automate mitigation actions, with human approval.
A little bit about us - We did YC back in fall 2024. We started our journey experimenting with continuous log monitoring with small language models - that was too slow. We then invested deeply into solving root cause analysis effectively, and our product today is the result of about a year of work with our early customers.
Give us a try today. Happy to hear feedback, or about how you are tackling on-call burden at your company. Appreciate any comments or suggestions!
taoh 27 minutes ago
Congratulations! The difference between pure agentic exploration and deterministic steps is spot on. Runbooks give ops more confidence on the data exploration and save time/context.
Curious how much savings do you observe from using runbook versus purely let Claude do the planning at first. Also how the runbooks can self heal if results from some steps in the middle are not expected.
behat 13 minutes ago
>> how the runbooks can self heal if results from some steps in the middle are not expected.
Yeah this is a very interesting angle. Our primary mechanism here is via agent created auto-memories today. The agent keeps track of the most useful steps, and more importantly, dead end steps as it executes runbooks. We think this offers a great bridge to suggest runbook updates and keep them current.
>> Curious how much savings do you observe from using runbook versus purely let Claude do the planning at first.
Really depends on runbook quality, so I don't have a straightforward answer. Of course, it's faster and cheaper if you have well defined steps in your runbooks. As an example, `check logs for service frontend, faceted by host_name`, vs. `check logs`. Agent does more exploration in the latter case.
We wrote about the LLM costs of investigating production alerts more generally here, in case helpful: https://relvy.ai/blog/llm-cost-of-ai-sre-investigating-produ...
atarus an hour ago
Interesting! In my experience using custom harnesses has worked better eg: Stripe etc all did it custom largely because of the sensitive integrations. How would you handle that?
hangrymoon01 12 minutes ago
Re: custom harnesses, imo maintaining them can be time consuming especially when things are changing very fast with AI. Bringing up a prototype is easy but a robust harness that handles the edge cases needs time and effort.
behat 23 minutes ago
Do you mean how we connect to internal data? Today, you can connect any API endpoint to Relvy, so if you have internal business data / dashboards that you look at while debugging, Relvy can do the same if there's an API for it.
Most of our deployments are self-hosted, in which case the data stays locally (your chosen LLM provider exempted), if that's what you are asking.
willchen 2 hours ago
Interesting! tbh, we don't have any runbooks and pretty minimal telemetry set up (we're a very small team :), do you have any recommendations on which telemetry service to use to get started? right now, our services run on a combination GCP Cloud Run + Vercel
behat 2 hours ago
Nice to see you here, Will! I’d generally recommend using open telemetry for instrumentation so that you keep the option of switching between telemetry vendors.
Re: runbooks, yeah even larger teams don’t have good ones to begin with. Relvy helps debug without runbooks as well - it might take longer to explore, but once you are happy with a particular investigation path the AI took, you can save it as a runbook for more deterministic future executions.
hrimfaxi 6 hours ago
How does this differ from cursor cloud agents where I can hook up MCPs, etc and even launch the agent in my own cloud to connect directly to internal hosts like dbs?
behat 6 hours ago
Thanks. Yeah, Cursor / Claude code + MCP is powerful. We differentiate on two fronts, mainly:
1) Greater accuracy with our specialized tools: Most MCP tools allow agents to query data, or run *ql queries - this overwhelms context windows given the scale of telemetry data. Raw data is also not great for reasoning - we’ve designed our tools to ensure that models get data in the right format, enriched with statistical summaries, baselines, and correlation data, so LLMs can focus on reasoning.
2) Product UX: You’ll also find that text based outputs from general purpose agents are not sufficient for this task - our notebook UX offers a great way to visualize the underlying data so you can review and build trust with the AI.
hrimfaxi 6 hours ago
To be clear, are the main differentiators basically better built-in MCPs and better UX? Not knocking just trying to understand the differences.
I have had incredible success debugging issues by just hooking up Datadog MCP and giving agents access to it. Claude/cursor don't seem to have any issues pulling in the raw data they need in amounts that don't overload their context.
Do you consider this a tool to be used in addition to something like cursor cloud agents or to replace it?
behat 6 hours ago
esafak 6 hours ago
They claim a 12% lead (from 36% to 48%) over Opus 4.6 in a RCA benchmark: https://www.relvy.ai/blog/relvy-improves-claude-accuracy-by-...
behat 6 hours ago
heh, I was just about to post the following on your previous comment re: reproducible benchmark results. Thanks for posting the blog.
With the docker images that we offer, in theory, people can re-run the benchmark themselves with our agent. But we should document and make that easier.
At the end of it, you really would have to evaluate on your own production alerts. Hopefully the easy install + set up helps.
ramon156 6 hours ago
Congrats on the launch! I dig the concept, seems like a good tool :)
behat 6 hours ago
Thank you :)
rishav 5 hours ago
Woohoo!!! Congrats on the big launch y'all
Harnoor_Kaur 4 hours ago
This is a big one!! Congratulations guys :) Rooting for you!