Hi all. I’ve never really posted my own content here, but I feel compelled to now. I’ve been working at a startup in San Francisco, which is to say I’m surrounded by people, teams, and companies building cool agentic stuff to optimize and automate the software and product development lifecycle. Coding agents may be the largest and most valuable application of AI we’ve discovered so far (besides ads...). Meanwhile, the market is still scratching its head, wondering when all of this AI investment is going to generate real returns for real companies. This is my reflection on the matter.

Author here - you are right that this architecture is not going to deliver super fast queries. But that's a tradeoff we're making: Agents don't need super fast queries to triage your software issues. In fact, the agents are extremely good at triage by fanning out to explore hypotheses against the telemetry. What they need is a datastore that allows them to run a _ton_ of queries in parallel, on the cheap. Data Lake architectures like ours provide exactly this.

At the end of the day, we're less focused on traditional database query metrics. We're optimizing for higher level outcomes, think mean time to remediation and such.

> The agents themselves are not likely going to be doing the high cardinality queries or they will keel over. They have limited memory buffers. They will take many seconds to return results. They are likely going to be limited in terms of QPS.

You're right that if you simply give an LLM a tool to query a massive high cardinality dataset, it's going to blow itself (its context window) up. That's not what we're doing: instead we harness the llm with purpose-built tools + prompt + context + other engineering to ensure the agent can explore the data and make progress, even if it does run a dumb query on occasion.