Attempted Blind Constrained Descent Experiments

02/18/2021
by   Prasad N R, et al.
0

Blind Descent uses constrained but, guided approach to learn the weights. The probability density function is non-zero in the infinite space of the dimension (case in point: Gaussians and normal probability distribution functions). In Blind Descent paper, some of the implicit ideas involving layer by layer training and filter by filter training (with different batch sizes) were proposed as probable greedy solutions. The results of similar experiments are discussed. Octave (and proposed PyTorch variants) source code of the experiments of this paper can be found at https://github.com/PrasadNR/Attempted-Blind-Constrained-Descent-Experiments-ABCDE- . This is compared against the ABCDE derivatives of the original PyTorch source code of https://github.com/akshat57/Blind-Descent .

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/20/2020

Blind Descent: A Prequel to Gradient Descent

We describe an alternative to gradient descent for backpropogation throu...
research
06/23/2020

Spherical Perspective on Learning with Batch Norm

Batch Normalization (BN) is a prominent deep learning technique. In spit...
research
07/27/2023

Scaling Session-Based Transformer Recommendations using Optimized Negative Sampling and Loss Functions

This work introduces TRON, a scalable session-based Transformer Recommen...
research
10/09/2019

Loss Surface Sightseeing by Multi-Point Optimization

We present multi-point optimization: an optimization technique that allo...
research
10/09/2019

Loss Landscape Sightseeing with Multi-Point Optimization

We present multi-point optimization: an optimization technique that allo...
research
08/21/2023

GBM-based Bregman Proximal Algorithms for Constrained Learning

As the complexity of learning tasks surges, modern machine learning enco...
research
01/24/2023

Model soups to increase inference without increasing compute time

In this paper, we compare Model Soups performances on three different mo...

Please sign up or login with your details

Forgot password? Click here to reset