Talk

---

The goal of this talk is to create denormalized and localized views close to users in order to improve performance, with the majority of requests being reads. This is known as Command Query Responsibility Segregation (CQRS). This architecture uses caches to reduce latency and improve efficiency, by storing a canonical version of the data in a Postgres database and replicating it to different caches in different geographies. The talk focused on Phoenix Pin, a Redis sponsored project. Infinity Spin is a highly flexible and versatile cache that can be used for in-memory requests, deployed in clusters, and used with Java.net, Node.js, and more. It is fault-tolerant with replicable data and can be queried like a database. It can also be embedded in applications, allowing for scalability and stateless applications. The challenge of keeping the data in sync between the caches and the system of record was addressed. Dual writes can be used to update multiple resources, such as a database and cache, but it is not always the best idea due to availability concerns. CDC is a solution proposed for distributed transactions between multiple resources. It works by tapping into the transaction log of a database and extracting changes such as inserts, updates, and deletes. Division is an open source project for implementing CDC and is typically used with Apache Kafka. It also supports near caching to keep data in-memory, and cross-site replication to keep multiple clusters of Infinite Spin in different locations in sync. This presentation discussed ways to reduce latency for read requests by using local caches and denormalization with Kafka streams. The advantages of this approach include lower latencies, reduced load on the primary database, and increased availability of the system. However, this approach adds complexity and requires understanding of eventual consistency. Resources for further exploration are provided.

---