DeepAI AI Chat
Log In Sign Up

Coalesced Multi-Output Tsetlin Machines with Clause Sharing

by   Sondre Glimsdal, et al.

Using finite-state machines to learn patterns, Tsetlin machines (TMs) have obtained competitive accuracy and learning speed across several benchmarks, with frugal memory- and energy footprint. A TM represents patterns as conjunctive clauses in propositional logic (AND-rules), each clause voting for or against a particular output. While efficient for single-output problems, one needs a separate TM per output for multi-output problems. Employing multiple TMs hinders pattern reuse because each TM then operates in a silo. In this paper, we introduce clause sharing, merging multiple TMs into a single one. Each clause is related to each output by using a weight. A positive weight makes the clause vote for output 1, while a negative weight makes the clause vote for output 0. The clauses thus coalesce to produce multiple outputs. The resulting coalesced Tsetlin Machine (CoTM) simultaneously learns both the weights and the composition of each clause by employing interacting Stochastic Searching on the Line (SSL) and Tsetlin Automata (TA) teams. Our empirical results on MNIST, Fashion-MNIST, and Kuzushiji-MNIST show that CoTM obtains significantly higher accuracy than TM on 50- to 1K-clause configurations, indicating an ability to repurpose clauses. E.g., accuracy goes from 71.99 to 89.66 memory). While TM and CoTM accuracy is similar when using more than 1K clauses per class, CoTM reaches peak accuracy 3× faster on MNIST with 8K clauses. We further investigate robustness towards imbalanced training data. Our evaluations on imbalanced versions of IMDb- and CIFAR10 data show that CoTM is robust towards high degrees of class imbalance. Being able to share clauses, we believe CoTM will enable new TM application domains that involve multiple outputs, such as learning language models and auto-encoding.


page 1

page 2

page 3

page 4


The Regression Tsetlin Machine: A Tsetlin Machine for Continuous Output Problems

The recently introduced Tsetlin Machine (TM) has provided competitive pa...

Extending the Tsetlin Machine With Integer-Weighted Clauses for Increased Interpretability

Despite significant effort, building models that are both interpretable ...

Massively Parallel and Asynchronous Tsetlin Machine Architecture Supporting Almost Constant-Time Scaling

Using logical clauses to represent patterns, Tsetlin machines (TMs) have...

A Regression Tsetlin Machine with Integer Weighted Clauses for Compact Pattern Representation

The Regression Tsetlin Machine (RTM) addresses the lack of interpretabil...

Increasing the Inference and Learning Speed of Tsetlin Machines with Clause Indexing

The Tsetlin Machine (TM) is a machine learning algorithm founded on the ...

Dynamic Training of Liquid State Machines

Spiking Neural Networks (SNNs) emerged as a promising solution in the fi...

Code Repositories


PyCUDA implementation of the Coalesced Multi-Output Tsetlin Machine

view repo