The Missing Layer in Your AI Stack: Context, Not Just State
Newsletter Content
The Missing Layer in Your AI Stack: Context, Not Just StateFrom SQL to Semantics: The Rise of the Context Graph for AI Agents
Join The Great Data Debate to get answers to questions the data & AI industry is so curious about right now:
Why Data Engineers Must Think in Graphs, Not Just TablesIf you have been following the “Systems of Record” debate on tech Twitter, you likely saw the clash between the “Agents kill SaaS” camp and the “Long live the Database” camp. But for data engineers, the reality is more nuanced—and far more interesting. As we move from dashboards to autonomous agents, we are hitting a wall. It turns out that knowing the state (what happened) is not the same as knowing the reasoning (why it happened). Drawing on recent insights from Foundation Capital, Jamin Ball (Altimeter), OpenAI’s internal engineering team, and the TrustGraph manifesto, this post explores the emergence of the Context Graph. This missing architectural layer will likely redefine how we build data platforms in the agentic era. The Problem: State Machines vs. Decision TracesFor the past decade, our role as data engineers has been to centralize data in the warehouse (or Lakehouse). We built ETL pipelines to move data from Salesforce, NetSuite, and Zendesk into a “Single Source of Truth.” However, traditional Systems of Record (SoR) effectively act as “state machines.” They record the final output: the organization closed a deal, applied a discount, and escalated a ticket. But they fail to capture the decision traces. As Foundation Capital notes, the reasoning behind a decision—the Slack threads, the cross-system synthesis, the VP’s verbal override of a policy—is rarely captured in the database. A CRM might show a “20% discount,” but it won’t tell an AI agent why that exception was granted (e.g., “Customer represents a strategic entry into the APAC market”). Without these traces, agents fly blind. They have the rules (”Do not give discounts >10%”), but they lack historical context on when and why they were violated. The Solution: The Truth Registry and the Context GraphTo address this, we observe a bifurcation in the modern data stack, as illustrated by the Hybrid Agentic Architecture (see Figure 1 below).
This architecture consists of two distinct but integrated planes: 1. The Warehouse as the “Truth Registry.”Jamin Ball argues that systems of record aren’t dying; they are becoming “boring, rock-solid sources of truth”. In an agentic world, the warehouse must evolve into a “Truth Registry” that encodes semantic contracts. Agents are fragile. If an agent hallucinates the definition of “Churn,” it can automate disastrous decisions. Therefore, we must clean and canonize data before the agent sees it. In the architecture above, the flow is from Raw (Variant) to Silver (Extracted) to Gold (Canonical Model).
2. The Context Graph as the “Reasoning Layer.”While the warehouse handles facts, the Context Graph handles relationships. TrustGraph defines a context graph as a “triples-representation of data (Subject → Predicate → Object) optimized for AI”. Why a graph? Because structure is information. When you feed an LLM structured data (like RDF or Cypher), the structure itself encodes meaning. This allows agents to traverse relationships that SQL joins struggle to represent—stitching together a user’s support ticket, their billing status, and their web activity into a single, queryable context. Case Study: Inside OpenAI’s Data AgentOpenAI recently reported that standard metadata was insufficient for their internal data agent. They had to build a custom “Context Layer” that closely resembles the architecture above. Their agent failed when it relied solely on table schemas. To fix this, they added:
This confirms a major trend: The metadata is the model. Providing agents with a semantic ontology (machine-readable definitions of terms) is just as important as the data itself. The “Front Door” is MovingThe implications for the industry are massive. Historically, if you owned the System of Record (like Salesforce), you owned the “Front Door” (the UI). But as agents take over workflows, the UI is unbundling from the data. Jamin Ball compares this to the travel industry: GDS systems (Sabre, Amadeus) remained the backend source of truth, but Online Travel Agencies (Expedia, Booking) captured the front door—and the value. In our new stack, the Agents become the OTAs. They are the new interface. The Warehouse/Lakehouse becomes the GDS—the invisible, essential infrastructure layer. What This Means for Data Engineers
As agents grow more capable, the infrastructure beneath them must evolve. The Context Graph offers a powerful new foundation—not just for smarter agents, but for more transparent, explainable, and aligned systems. It’s time for data teams to build not just pipelines, but reasoning engines. Referenceshttps://x.com/KirkMarple/status/2003944353342149021 https://x.com/KirkMarple/status/2005443843848856047 https://foundationcapital.com/context-graphs-ais-trillion-dollar-opportunity/ https://trustgraph.ai/news/context-graph-manifesto/ https://openai.com/index/inside-our-in-house-data-agent/ 
|
Details
Source:
Other
Date:
Jan 31, 2026 11:13
Category:
Technical