Advanced pattern support Interceptors

Advanced patterns in Conduktor Gateway Interceptors: cache data, handle large messages, filter and project topic records with topic views, and concentrate topic partitions.

Conduktor Gateway advanced pattern Interceptors let you do things standard Kafka can’t: cache repeated fetches, send oversized messages, filter and reshape topic content server-side, and collapse many virtual topics onto fewer physical ones. Producers and consumers keep using standard Kafka clients — Gateway handles it at the proxy layer.

Available patterns

Pattern	Description
Data caching	Cache fetch responses at the proxy layer to reduce broker load and repeated consumer reads
Large messages	Split oversized messages at produce time and reassemble them transparently at consume time
Topic views	Present a non-materialized, filtered and projected view of a physical topic using SQL, without duplicating the underlying data
Topic concentration	Map multiple virtual topics onto a single physical Kafka topic to reduce partition count and broker overhead

Topic views replace SQL topics and CEL topics, which are deprecated. Use topic views for new work — the deprecated plugins will be removed once topic views cover the remaining functionality.

When to use advanced patterns

• Reduce broker load — use caching to serve repeated fetch requests without hitting the broker every time
• Work around Kafka limits — large message handling lets you produce and consume messages that exceed message.max.bytes without reconfiguring brokers
• Control data exposure — topic views let you expose a filtered, projected subset of a topic’s records to specific consumers without duplicating the underlying data
• Optimize partition usage — topic concentration consolidates logical namespaces onto fewer physical partitions, reducing overhead in multi-tenant environments

Related resources

• Logical topics overview
• Topic views
• TopicView resource reference
• Migrate a Kafka Streams transformer to a Topic View
• Large message use case
• Cache data use case
• Data quality Interceptors