Pushing Java to the Limits: Processing a Billion Rows in under 2 Seconds
Deep Dive (INTERMEDIATE level)
Room 5
Last January a challenge was posted online by Gunnar Morling:
How fast can you parse a file with one billion rows of weather data using Java?
Little did we know this deceivingly simple question would lead us down a path that covered: parallelism, memory mapped files, SWAR techniques (SIMD as a register), bit twiddling, branchless code, mechanical sympathy, Graal native compilation and finally... turning to the dark side: using sun.misc.Unsafe.
Join Thomas and Roy during this deep dive where they explain all the improvements and tricks that enabled them to go from a >4 minute reference implementation to processing a billion rows in under two seconds.
Who knew Java could be _this_ fast?
How fast can you parse a file with one billion rows of weather data using Java?
Little did we know this deceivingly simple question would lead us down a path that covered: parallelism, memory mapped files, SWAR techniques (SIMD as a register), bit twiddling, branchless code, mechanical sympathy, Graal native compilation and finally... turning to the dark side: using sun.misc.Unsafe.
Join Thomas and Roy during this deep dive where they explain all the improvements and tricks that enabled them to go from a >4 minute reference implementation to processing a billion rows in under two seconds.
Who knew Java could be _this_ fast?
AI-generated (Experimental): may contain inaccuracies, please verify facts.
Roy van Rijn
OpenValue
Roy van Rijn is a director at OpenValue Rotterdam and a Java Champion. He worked on numerous projects all over the Netherlands as developer, architect and agile coach. He talks at conferences all around the world, CodeOne (Rockstar), Devoxx BE & UK & PL, GOTO, Joy of Coding, and local JUG events. You can read more at his blog (http://www.royvanrijn.com) or follow him on Twitter/X (@royvanrijn).
Thomas Wuerthinger
Oracle
Thomas Wuerthinger is a Vice President at Oracle Labs leading programming language implementation teams for languages including Java, JavaScript, Ruby, and R. He is the architect of the Graal compiler and the Truffle self-optimizing runtime system. Previously, he worked on the Crankshaft optimizing compiler of V8 at Google, and the Maxine research virtual machine at Sun Microsystems. He received a PhD degree from JKU Linz for his research about dynamic code evolution.