End-to-end Robustness for Sensing-Reasoning Machine Learning Pipelines

02/28/2020
by   Zhuolin Yang, et al.
25

As machine learning (ML) being applied to many mission-critical scenarios, certifying ML model robustness becomes increasingly important. Many previous works focuses on the robustness of independent ML and ensemble models, and can only certify a very small magnitude of the adversarial perturbation. In this paper, we take a different viewpoint and improve learning robustness by going beyond independent ML and ensemble models. We aim at promoting the generic Sensing-Reasoning machine learning pipeline which contains both the sensing (e.g. deep neural networks) and reasoning (e.g. Markov logic networks (MLN)) components enriched with domain knowledge. Can domain knowledge help improve learning robustness? Can we formally certify the end-to-end robustness of such an ML pipeline? We first theoretically analyze the computational complexity of checking the provable robustness in the reasoning component. We then derive the provable robustness bound for several concrete reasoning components. We show that for reasoning components such as MLN and a specific family of Bayesian networks it is possible to certify the robustness of the whole pipeline even with a large magnitude of perturbation which cannot be certified by existing work. Finally, we conduct extensive real-world experiments on large scale datasets to evaluate the certified robustness for Sensing-Reasoning ML pipelines.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 11

05/06/2020

Testing the Robustness of AutoML Systems

Automated machine learning (AutoML) systems aim at finding the best mach...
07/17/2021

Towards autonomic orchestration of machine learning pipelines in future networks

Machine learning (ML) techniques are being increasingly used in mobile n...
06/10/2019

Making Classical Machine Learning Pipelines Differentiable: A Neural Translation Approach

Classical Machine Learning (ML) pipelines often comprise of multiple ML ...
06/11/2021

Knowledge Enhanced Machine Learning Pipeline against Diverse Adversarial Attacks

Despite the great successes achieved by deep neural networks (DNNs), rec...
05/22/2019

The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System Development

As machine learning is applied more and more widely, data scientists oft...
11/24/2019

On the Robustness of Deep Learning-predicted Contention Models for Network Calculus

The network calculus (NC) analysis takes a simple model consisting of a ...
03/04/2019

Learning Ex Nihilo

This paper introduces, philosophically and to a degree formally, the nov...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.