Known as a Streamhouse, this architecture helps capture, process, govern and serve the current state of a business, giving decisions and actions a common foundation

Key Facts

• A Streamhouse is a new data architecture that lets AI agents and applications access and act on a governed view of the current state of the business at machine speed. 
• Confluent’s Data Streaming Platform adds new Query capabilities to become a complete and unified platform for building a Streamhouse.
• Lightning Tables lets customers query live streaming data directly in Confluent with low latency, reducing the need to copy data into a separate database to serve applications.

Confluent, an IBM company, today announced new capabilities for building a Streamhouse, a new type of data architecture that keeps business context current for AI agents, applications and analytics as they make decisions and take action. With Lightning Tables now generally available on AWS, Confluent completes its five-pillar Data Streaming Platform, bringing capabilities to Stream, Connect, Process, Govern and now Query data together in a unified platform for building a Streamhouse. Confluent sits at the center of IBM’s data strategy, combining real-time data streaming with IBM’s integration, governance and AI capabilities across hybrid environments.

Why enterprises need a Streamhouse

Data lakes, warehouses and lakehouses were built primarily with reporting or other batch jobs in mind. As AI agents take on more work across the enterprise, they must make decisions using information spread across systems that update at different times. When those systems have different views of the current state of the business, agents can act on outdated or conflicting information. Those actions can carry into other workflows.

A Streamhouse addresses this architectural problem by continuously capturing, processing, governing and serving current business context across agents, applications and analytics. Consider the following illustrative example of an airline managing a canceled flight:

• Today, it has different systems holding conflicting views of the same operation. Flight operations track where the aircraft is, scheduling tracks when it will depart, and reservations tracks which seats are available. When those systems update at different times, an autonomous rebooking agent can assemble an itinerary that looks valid in each system but does not work in practice: a seat is available, but the aircraft cannot arrive in time for the passenger’s connection. Customer-service teams and notification systems can then act on those same conflicting views, spreading the disruption.
• With a Streamhouse, each change can update a shared picture of the airline’s operations. When an aircraft’s location changes, a departure is delayed or a seat is booked, the source system sends that update into a continuous flow of events. A Streamhouse combines those updates into current business context and makes it available to rebooking agents, customer-service teams and applications as they work. Instead of piecing together separately updated information, they can check aircraft location, departure time and seat availability together before offering a passenger a new itinerary.

Enterprises are already seeing results. Accenture uses a Streamhouse architecture powered by Confluent to connect data across its global business.

“Operating across more than 120 countries, our global enterprise was previously constrained by legacy batch and point-to-point architectures where duplicative data extractions drove up operational overhead and analytics data lagged. Moving to a Streamhouse architecture powered by Confluent decouples data producers from consumers, transforming high-volume event streams into governed, reusable real-time data products that flow seamlessly across operational and analytical systems. With Confluent handling infrastructure automation and providing seamless enterprise connectivity, our developers operate in a standardized environment without the friction of building custom integrations or micro-managing data movement. This continuous data foundation has dropped our time-to-insight from months to days and slashed reporting times, empowering our teams to focus on delivering high-value AI for our 800,000 employees.”
— Thomas Mahaffey, SAI Chief Data Strategist, Accenture.

What makes a Streamhouse different

A Streamhouse is built on three architectural principles: current, mission-critical and decentralized. It maintains continuously updated business context, is engineered to production service levels, and meets data where it lives without requiring consolidation first. It works alongside existing data warehouses and lakehouses, adding a real-time operational layer for decisions and actions.

“As an AI-native commerce platform delivering personalized shopper experience and operational and financial advantage to merchants, Extend requires exceptional adaptability across rapid merchant onboarding, real-time pricing adjustments, and instant fraud detection. We initially struggled with traditional message queues and static data lakes that lacked event replay capabilities, persistence, and strict schema enforcement at scale. Implementing a 100% serverless Streamhouse architecture anchored by Confluent Cloud turns Kafka into our operational source of truth, enabling downstream microservices to asynchronously consume and query real-time data without breaking contracts. This inherently decoupled streaming foundation directly provides the flexibility and responsiveness required to scale our operations, adapt to live market signals, and power real-time agentic AI workflows.”
— Jonathan Kropp, Director of AI and Architecture, Extend

Confluent’s Data Streaming Platform Now Offers a Complete Streamhouse

Confluent’s Data Streaming Platform (DSP) provides a complete, unified foundation for building a Streamhouse. At its core is Apache Kafka®, which captures and streams business events. On its own, Kafka leaves teams to assemble everything else needed to turn those events into current business context. The DSP brings that together in one platform, now adding Query as its fifth pillar alongside Stream, Connect, Process and Govern. Core capabilities and latest additions to each platform pillar include:

• Stream, built on Apache Kafka and the Kora engine, provides the durable, scalable foundation for capturing and streaming business events across systems with sub-millisecond latency in Confluent internal testing; actual latency varies by configuration and workload.
• Connect, with 150+ fully managed and custom connectors, brings data from enterprise systems into Kafka and delivers it downstream. Hybrid connector management extends how teams manage those connections.
• Process uses Apache Flink to combine, enrich and derive business context close to the source as data changes. Additional batch execution capabilities help initialize streaming workloads from existing data.
• Govern helps teams discover, classify and secure data through lineage, catalog, quality, encryption, schema and access-control capabilities.
• Query serves current business context to analytics, agents and applications.

Query puts streaming data to work for analytics, agents and applications, each through the interface that fits the job:

• Confluent Tableflow makes streaming data available to lakehouses and warehouses as open table formats, ready for analytics.
• Real-Time Context Engine serves current business context to AI agents through the Model Context Protocol (MCP).
• Lightning Tables gives applications low-latency SQL and API access to the current state of the business, so they can respond as conditions change.

Lightning Tables: Fast access to the current state of the business

Generally available on AWS, Lightning Tables on Confluent Cloud is a read-only serving layer that durably stores and materializes Apache Kafka data into tables, keeping them current as changes arise. It makes the current state of the business available for low-latency queries. It features:

• Data freshness and query performance: In Confluent internal testing, approximately 100 milliseconds from new data creation to serving, with query latency of 10-100 milliseconds and throughput of 1,000+ queries per second.
• Lower cost and complexity: Serve continuously with a fully managed serving layer, without separate ETL pipelines, databases, or caches.
• Multiple query interfaces: Query through SQL and a direct API.
• Unified governance: Apply schemas, lineage and access policies from data capture through serving.
• Schema evolution: Manage schema changes to help prevent breaks in downstream applications.
• Feature store: Compute features in Flink and store them for machine learning use cases, including real-time personalization.

“The next era of enterprise AI will be won by the agents that know what’s happening in the business the moment they act,” said Shaun Clowes, General Manager, Confluent and Data, IBM. “That’s what a Streamhouse can deliver. With Lightning Tables, our Data Streaming Platform gives each agent and application a live, governed view of the business. This means companies can act on fraud signals sooner, serve customers in the moment and run operations on what’s true now versus what was true yesterday.”

Supporting Materials

• Introducing the Streamhouse Working Group
• Q4 Product Bundle blog
• Streamhouse vision blog
• Streamhouse webinar
• Streamhouse Quickstart

About Confluent

Confluent, an IBM company, is the data streaming platform that is pioneering a fundamentally new category of data infrastructure that sets data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion—designed to be the intelligent connective tissue enabling real-time data from multiple sources to constantly stream across an organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital frontend customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, visit www.confluent.io.

About IBM

IBM is a leading provider of global hybrid cloud and AI, and consulting expertise. We help clients in more than 175 countries capitalize on insights from their data, streamline business processes, reduce costs and gain the competitive edge in their industries. Thousands of governments and corporate entities in critical infrastructure areas such as financial services, telecommunications and healthcare rely on IBM's hybrid cloud platform and Red Hat OpenShift to affect their digital transformations quickly, efficiently and securely. IBM's breakthrough innovations in AI, quantum computing, industry-specific cloud solutions and consulting deliver open and flexible options to our clients. All of this is backed by IBM's long-standing commitment to trust, transparency, responsibility, inclusivity and service. Visit www.ibm.com for more information.

View source version on businesswire.com: https://www.businesswire.com/news/home/20261104894527/en/