Streaming ETL in Practice with PostgreSQL, Apache Kafka, and KSQL
Presented by:
Viktor Gamov
Viktor Gamov is a Developer Advocate at Confluent, the company that makes an event streaming platform based on Apache Kafka. Back in his consultancy days, Viktor developed comprehensive expertise in building enterprise application architectures using open source technologies. He enjoys helping architects and developers to design and develop low latency, scalable and highly available distributed systems. He is a professional conference speaker on distributed systems, streaming data, JVM and DevOps topics, and is regular on events including JavaOne, Devoxx, OSCON, QCon, and others. He co-authored O’Reilly’s «Enterprise Web Development.» He blogs at http://gamov.io and co-hosts DevRelRad.io. Follow Viktor on Twitter @gamussa, where he posts there about gym life, food, open source, and, of course, Kafka and Confluent!
Have you ever thought that you needed to be a programmer to do stream processing and build streaming data pipelines? Think again!
Companies new and old are all recognizing the importance of a low-latency, scalable, fault-tolerant data backbone, in the form of the Apache Kafka® streaming platform.
With Kafka, developers can integrate multiple sources and systems, which enables low latency analytics, event-driven architectures and the population of multiple downstream systems.
These data pipelines can be built using configuration alone.
In this talk, we’ll see how easy it is to stream data from a database such as PostgreSQL into Kafka using CDC and Kafka Connect.
Besides, we’ll use KSQL to filter, aggregate and join it to other data, and then stream this from Kafka out into multiple targets such as Elasticsearch and S3.
All of this will be accomplished without a single line of code!
Why should programming buffs have all the fun?
- Date:
- 2019 March 22 14:00 EDT
- Duration:
- 50 min
- Room:
- New York Ballroom East
- Conference:
- Postgres Conference
- Language:
- Track:
- Data
- Difficulty:
- Medium