Efficient Sampled Softmax for Tensorflow

04/10/2020

∙

This short paper discusses an efficient implementation of sampled softmax loss for Tensorflow. The speedup over the default implementation is achieved due to simplification of the graph for the forward and backward passes.

READ FULL TEXT

Efficient Sampled Softmax for Tensorflow

Sampled Softmax with Random Fourier Features

On the Effectiveness of Sampled Softmax Loss for Item Recommendation

Memory-Efficient Training of RNN-Transducer with Sampled Softmax

Softmax Optimizations for Intel Xeon Processor-based Platforms

TensorFlow Agents: Efficient Batched Reinforcement Learning in TensorFlow

Noisy Softmax: Improving the Generalization Ability of DCNN via Postponing the Early Softmax Saturation

A Constant-time Adaptive Negative Sampling

Efficient Sampled Softmax for Tensorflow

Related Research

Sampled Softmax with Random Fourier Features

On the Effectiveness of Sampled Softmax Loss for Item Recommendation

Memory-Efficient Training of RNN-Transducer with Sampled Softmax

Softmax Optimizations for Intel Xeon Processor-based Platforms

TensorFlow Agents: Efficient Batched Reinforcement Learning in TensorFlow

Noisy Softmax: Improving the Generalization Ability of DCNN via Postponing the Early Softmax Saturation

A Constant-time Adaptive Negative Sampling