Speaker(s): Vinay Mayar
We use streaming Beam pipelines to index data into an object store (S3) and compute summaries of new data with periodic batch pipelines. These summaries are analogous to indices in a database and allow for efficient lookup along a fixed set of axes. The result is a scalable and inexpensive data storage solution for applications that serve data with limited query patterns.