Target Curricula via Selection of Minimum Feature Sets: a Case Study in Boolean Networks

06/15/2017
by   Shannon Fenn, et al.
0

We consider the effect of introducing a curriculum of targets when training Boolean models on supervised Multi Label Classification (MLC) problems. In particular, we consider how to order targets in the absence of prior knowledge, and how such a curriculum may be enforced when using meta-heuristics to train discrete non-linear models. We show that hierarchical dependencies between targets can be exploited by enforcing an appropriate curriculum using hierarchical loss functions. On several multi output circuit-inference problems with known target difficulties, Feedforward Boolean Networks (FBNs) trained with such a loss function achieve significantly lower out-of-sample error, up to 10% in some cases. This improvement increases as the loss places more emphasis on target order and is strongly correlated with an easy-to-hard curricula. We also demonstrate the same improvements on three real-world models and two Gene Regulatory Network (GRN) inference problems. We posit a simple a-priori method for identifying an appropriate target order and estimating the strength of target relationships in Boolean MLCs. These methods use intrinsic dimension as a proxy for target difficulty, which is estimated using optimal solutions to a combinatorial optimisation problem known as the Minimum-Feature-Set (minFS) problem. We also demonstrate that the same generalisation gains can be achieved without providing any knowledge of target difficulty.

READ FULL TEXT
research
07/14/2023

HuCurl: Human-induced Curriculum Discovery

We introduce the problem of curriculum discovery and describe a curricul...
research
06/05/2020

Hierarchical Class-Based Curriculum Loss

Classification algorithms in machine learning often assume a flat label ...
research
12/09/2018

Theory of Curriculum Learning, with Convex Loss Functions

Curriculum Learning - the idea of teaching by gradually exposing the lea...
research
04/08/2019

Component-Wise Boosting of Targets for Multi-Output Prediction

Multi-output prediction deals with the prediction of several targets of ...
research
05/02/2022

Skeptical binary inferences in multi-label problems with sets of probabilities

In this paper, we consider the problem of making distributionally robust...
research
12/01/2017

Precision Learning: Towards Use of Known Operators in Neural Networks

In this paper, we consider the use of prior knowledge within neural netw...

Please sign up or login with your details

Forgot password? Click here to reset