Binary Classification as a Phase Separation Process
share
We propose a new binary classification model called Phase Separation Binary Classifier (PSBC). It consists of a discretization of a nonlinear reaction-diffusion equation coupled with an ODE, and is inspired by fluid behavior, namely, on how binary fluids phase separate. Hence, parameters and hyperparameters have physical meaning, whose effects are carefully studied in several different scenarios. PSBC's coefficients are trainable weights, chosen according to a minimization problem using Gradient Descent; optimization relies on a classical Backpropagation with weight sharing. The model can be seen under the framework of feedforward networks, and is endowed with a nonlinear activation function that is linear in trainable weights but polynomial in other variables, yielding a cost function that is also polynomial. In view of the model's connection with ODEs and parabolic PDEs, forward propagation amounts to an initial value problem. Thus, stability conditions are established using the concept of Invariant regions. Interesting model compression properties are thoroughly discussed. We illustrate the classifier's qualities by applying it to the subset of numbers "0" and "1" of the classical MNIST database, where we are able to discern individuals with more than 94% accuracy, sometimes using less only about 10% of variables.
READ FULL TEXT
