Hans-Peter Grahsl is a Staff Developer Advocate at Decodable. He is an open-source community enthusiast and in particular passionate about event-driven architectures, distributed stream processing systems and data engineering. For his code contributions, conference talks and blog post writing at the intersection of the Apache Kafka and MongoDB communities, Hans-Peter received multiple community awards. He likes to code and is a regular speaker at developer conferences around the world.
As the saying goes: nothing is older than yesterday’s news, uhm, data. Join us for an immersive hands-on lab to explore real-time ETL using the triumphant trio Apache Flink, Debezium, and LangChain4j.
Participants will gain practical experience in setting up different end-to-end real-time data pipelines, streaming data from an operational database to an analytics data store—continuously, efficiently, and with a very low latency—enabling use cases such as full-text search and live dashboarding, enriched with LLM-derived metadata.
In the lab, you will learn how to:
- Build a real-time data pipeline from Postgres to OpenSearch, based on Apache Flink and Debezium for change data capture (CDC)
- Use Flink's connector capabilities to set up seamless real-time ETL pipelines between various data sources and sinks
- Implement data transformations, filtering, and aggregations on top of CDC streams in real time with the help of streaming SQL
- Integrate a large language model (LLM) for sentiment analysis based on LangChain4j, enabling deeper insights into the processed data
Join this lab to advance your skills in working with real-time data and learn how robust and leading open-source technologies support your business-critical stream processing workloads.
please pull the following Docker images onto your laptop before.
This will save some time and network bandwidth on the day of the event:
docker image pull quay.io/debezium/example-postgres:2.7.3.Final
docker image pull quay.io/debezium/tooling:latest
docker image pull docker.io/opensearchproject/opensearch:1.3.19
docker image pull docker.io/flink:1.19.1-scala_2.12-java17
docker image pull docker.io/hpgrahsl/hol-devoxxbe-model-serving-app:1.0.0
docker image pull docker.io/hpgrahsl/hol-devoxxbe-review-app:1.0.1
docker image pull docker.io/hpgrahsl/data-generator:1.1.4
Searching for speaker images...