Scaling TensorFlow to 300 million predictions per second

09/20/2021

∙

by   Jan Hartman, et al.

∙

0

∙

share

We present the process of transitioning machine learning models to the TensorFlow framework at a large scale in an online advertising ecosystem. In this talk we address the key challenges we faced and describe how we successfully tackled them; notably, implementing the models in TF and serving them efficiently with low latency using various optimization techniques.