Sessions

Using the Mm FLaNK Stack for Edge AI (Flink, NiFi, Kafka, Kudu)

Wiggly air and data processing at Spotify

Digital signal processing (DSP) has been made easy with the help of many Python libraries, allowing engineers and researchers to quickly and effortlessly analyze audio, images, and video. However, scaling these algorithms and models to process millions of files has not been equally as seamless. At Spotify, we’re trying to address scaling DSP over our catalog of over 50 million songs. This talk will discuss the challenges we’ve encountered while building the infrastructure needed to support audio processing at scale.

Workshop: Build a Unified Batch and Stream Processing Pipeline with Apache Beam on AWS

We will explore an end to end example that combines batch and streaming aspects in one uniform Beam pipeline, and deploy it to a fully managed environment with Amazon Kinesis Data Analytics.

Workshop: Building big data pipelines for deep learning

We will work through an end-to-end example of using the Python Beam SDK to build maintainable, reliable, and scalable production pipelines for deep learning.

Workshop: First steps with Apache Beam - Writing portable pipelines using Java, Python, Go, Kotlin

This workshop is for participants who are getting started with Apache Beam. We will get hands-on writing Beam pipelines, as well as discuss the fundamentals of Beam programming model and SDKs.

Workshop: Implement a streaming data pipeline with Google Cloud Dataflow

We will implement a Google Cloud Dataflow pipeline through a series of labs where we will explore functionalities like auto-scaling, monitoring and optimization.

Workshop: Using Conda on Apache beam - there is nothing your cant do with your workers

Learn how to install Conda on your worker machine, personalize your workers’ environment and integrate with your Apache Beam pipeline.

Workshop: Writing custom components for TensorFlow Extended with Apache Beam

Step-by-step walkthrough of how to write custom TFX components with Apache Beam to customize your ML pipelines beyond the standard components and tailor the components for their ML pipelines.