Data Pipeline Plumbing

For the last two decades, the amount of data we store, process, and analyze is ever growing. The last decade shows a higher focus on immediate feedback loop data pipeline, using technologies such as Complex Event Processing (CEP), Stream Processing, and Change Data Capture (CDC). Services such as Kafka or NATS are to be found in almost every new system (at least to some extent).

To build a data pipeline, the number of technologies, frameworks, and platforms are endless. Getting the initial grasp of it all is much harder than expected, but together we can tackle it!

Christoph Engelbert is an open-source developer and consultant known for his work on Java serialization libraries. He frequently speaks at conferences and meetups about modern Java, data serialization, and distributed systems.

When he's not coding, he probably enjoys hiking in the Paderborn region or sipping a fine Belgian Duvel beer.