On the Convergence of Tsetlin Machines for the XOR Operator

01/07/2021
by   Lei Jiao, et al.
0

The Tsetlin Machine (TM) is a novel machine learning algorithm with several distinct properties, including transparent inference and learning using hardware-near building blocks. Although numerous papers explore the TM empirically, many of its properties have not yet been analyzed mathematically. In this article, we analyze the convergence of the TM when input is non-linearly related to output by the XOR-operator. Our analysis reveals that the TM, with just two conjunctive clauses, can converge almost surely to reproducing XOR, learning from training data over an infinite time horizon. Furthermore, the analysis shows how the hyper-parameter T guides clause construction so that the clauses capture the distinct sub-patterns in the data. Our analysis of convergence for XOR thus lays the foundation for analyzing other more complex logical expressions. These analyses altogether, from a mathematical perspective, provide new insights on why TMs have obtained state-of-the-art performance on several pattern recognition problems

READ FULL TEXT

page 1

page 2

page 3

page 4

07/28/2020

On the Convergence of Tsetlin Machines for the IDENTITY- and NOT Operators

The Tsetlin Machine (TM) is a recent machine learning algorithm with sev...
09/17/2021

On the Convergence of Tsetlin Machines for the AND and the OR Operators

The Tsetlin Machine (TM) is a novel machine-learning algorithm based on ...
10/29/2020

Infinite Time Solutions of Numerical Schemes for Advection Problems

This paper addresses the question whether there are numerical schemes fo...
01/28/2019

Support Feature Machines

Support Vector Machines (SVMs) with various kernels have played dominant...
12/09/2015

Distributed Training of Deep Neural Networks with Theoretical Analysis: Under SSP Setting

We propose a distributed approach to train deep neural networks (DNNs), ...
09/30/2019

Towards Scalable Koopman Operator Learning: Convergence Rates and A Distributed Learning Algorithm

In this paper, we propose an alternating optimization algorithm to the n...
02/08/2021

Functional Space Analysis of Local GAN Convergence

Recent work demonstrated the benefits of studying continuous-time dynami...