Theano

What is Theano?

Theano is an open source numerical computation library for Python designed to run on both CPU and GPU architectures. It allows users to define, optimize, and evaluate mathematical expressions, including multi-dimensional arrays (numpy.ndarray). 

Theano combines components of a computer algebra system (CAS) with those of an optimizing compiler. It can also generate customized C code for some operations. This combination of CAS and optimizing compilation is useful for tasks involving complicated mathematical expressions being evaluated repeatedly and where evaluation speed is important. 

What are Theano’s Unique Features?

According to Theano’s developers, the compiler applies many optimizations symbolic expressions. Some of the most important optimizations they list are:

  • Using GPU for computations
  • Constant folding
  • Merging of similar subgraphs, to avoid redundant calculation
  • Arithmetic simplification (e.g. X*y/x -> y, --x -> x)
  • Inserting efficient bias operations (e.g. Gemm) in a variety of contexts
  • Using memory aliasing to avoid calculation
  • Using in-place operations wherever it does not interfere with aliasing
  • Loop fusion for element-wise sub-expressions
  • Numerical stability