Batch Processing at Scale: a hands-on JobRunr workshop with experiments on Raspberry Pi
Hands-On Lab (BEGINNER level)
BOF 1
Batch processing can be quite a challenge in Java. How do you manage task coordination and data consistency? What to do with the complexities of concurrency, especially when moving away from outdated mainframe systems?
This workshop offers a practical exploration of JobRunr, an open-source Java library recommended by ThoughtWorks. It leverages your existing infrastructure and facilitates efficient job scheduling using a Java 8 lambda. The workshop is designed to provide you with a thorough understanding of JobRunr's capabilities in a real-world application context, helping you transition from other frameworks or mainframe batch processing to modern, distributed batch job processing.
Throughout the session, you'll learn to integrate JobRunr into Java applications, with a focus on distributed batch processing. We will use some Raspberry Pis to create a cluster of computers that work together to generate and handle jobs. The lab will cover JobRunr setup and configuration, creating one-off jobs, scheduling future jobs, maintaining recurring jobs, and scaling job throughput with Kubernetes. You'll also see how JobRunr adapts across various Java frameworks including Spring Boot and Quarkus and how we can use Pi4J to convert button presses to jobs and display the number of handled jobs on a small display.
By the end of this workshop, you'll be ready to manage batch processing in Java using JobRunr, while enjoying the fun of interacting with electronic components connected to a Raspberry Pi. You'll learn to set up, schedule jobs, and distribute across various devices. You'll be able to move away from mainframes and enhance your Java applications with modern, distributed job scheduling.
This workshop offers a practical exploration of JobRunr, an open-source Java library recommended by ThoughtWorks. It leverages your existing infrastructure and facilitates efficient job scheduling using a Java 8 lambda. The workshop is designed to provide you with a thorough understanding of JobRunr's capabilities in a real-world application context, helping you transition from other frameworks or mainframe batch processing to modern, distributed batch job processing.
Throughout the session, you'll learn to integrate JobRunr into Java applications, with a focus on distributed batch processing. We will use some Raspberry Pis to create a cluster of computers that work together to generate and handle jobs. The lab will cover JobRunr setup and configuration, creating one-off jobs, scheduling future jobs, maintaining recurring jobs, and scaling job throughput with Kubernetes. You'll also see how JobRunr adapts across various Java frameworks including Spring Boot and Quarkus and how we can use Pi4J to convert button presses to jobs and display the number of handled jobs on a small display.
By the end of this workshop, you'll be ready to manage batch processing in Java using JobRunr, while enjoying the fun of interacting with electronic components connected to a Raspberry Pi. You'll learn to set up, schedule jobs, and distribute across various devices. You'll be able to move away from mainframes and enhance your Java applications with modern, distributed job scheduling.
Frank Delporte
Azul
Frank Delporte is a Java Champion, developer, and technical writer working at Azul, blogs on his own site and Foojay, author of "Getting Started with Java on the Raspberry Pi", co-organizer of BeJUG, and contributor to Pi4J. He blogs and talks about his experiments with Java, sometimes combined with electronic components, on the Raspberry Pi.