What is Tracy-Widom Distribution?
Tracy–Widom distribution is a universal rule describing the distribution of any set of correlated variables. Instead of the smooth bell curve found with Gaussian distribution for uncorrelated variables, this distribution gives an asymmetrical statistical bump for correlated variables. The left side is steeper than the right and its summit sits at a scalable, universal value of √2N. Or the square root of twice the number of variables in the underlying systems. This is also referred to as the transition point or crossover function between between stability and instability in various systems.
How is the Tracy-Widom Distribution Used in Machine Learning?
Finding this transition point is quite useful for building stabile and faster machine learning programs. One of the simplest uses is just to test if a complex system’s variables are truly uncorrelated. Tracy-Widom distribution can also be calculated in any neural network fairly quickly, since the summit of the distribution corresponds to the largest eigenvalue in a matrix, which is the largest in a series of numbers calculated from both the matrix’s rows and columns.
Even in a randomly generated matrix, the N eigenvalues tend to fall along the real number line in a distinct pattern, with the largest random eigenvalue fluctuating around this average value.