Uncheatable Machine Learning Inference

08/08/2019 ∙ by Mustafa Canim, et al. ∙ ibm 0

Classification-as-a-Service (CaaS) is widely deployed today in machine intelligence stacks for a vastly diverse set of applications including anything from medical prognosis to computer vision tasks to natural language processing to identity fraud detection. The computing power required for training complex models on large datasets to perform inference to solve these problems can be very resource-intensive. A CaaS provider may cheat a customer by fraudulently bypassing expensive training procedures in favor of weaker, less computationally-intensive algorithms which yield results of reduced quality. Given a classification service supplier S, intermediary CaaS provider P claiming to use S as a classification backend, and customer C, our work addresses the following questions: (i) how can P's claim to be using S be verified by C? (ii) how might S make performance guarantees that may be verified by C? and (iii) how might one design a decentralized system that incentivizes service proofing and accountability? To this end, we propose a variety of methods for C to evaluate the service claims made by P using probabilistic performance metrics, instance seeding, and steganography. We also propose a method of measuring the robustness of a model using a blackbox adversarial procedure, which may then be used as a benchmark or comparison to a claim made by S. Finally, we propose the design of a smart contract-based decentralized system that incentivizes service accountability to serve as a trusted Quality of Service (QoS) auditor.



There are no comments yet.


page 1

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

The market for Machine Learning as as Service (MLaaS) has been exploding recently111https://www.marketsandmarkets.com/Market-Reports/deep-learning-market-107369271.html

, with breakthroughs in techniques, availability of datasets, demand for machine-intelligence solutions, and power of computation rapidly increasing. With this, we’ve seen a massive demand for resources to support such computation, and these resources aren’t cheap. Indeed, much of machine learning commercialized today relies on deep learning models trained and ran on large GPU clusters, requiring a great deal of time, energy, and memory to train.

From a business perspective, it is optimal for the service entity to minimize costs while maximizing revenue. However, when considering ways to minimize costs, a Classification-as-a-Service (CaaS) supplier may be tempted to cheat by delivering cheaper services, classifications made by less computationally-intensive models, while charging for and promising classification services performed by better models that require greater resources. A classification service supplier exposes an API that takes as input an image and responds with a softmax probability distribution over the possible labels. The supplier claims to be using a deep neural network model with hundreds of layers trained on Tencent ML-Images


but instead uses a Support Vector Machine (SVM) model trained on Cifar-10

333https://www.cs.toronto.edu/ kriz/cifar.html. SVMs are typically less computationally intensive than deep learning models, but also often do not perform as well, especially on computer vision tasks. The overall effect is a sub-standard level of quality, where the service accuracy, efficiency, speed, and/or robustness is below the standard set by the Service Level Agreements (SLA) guarantees or QoS claims that a CaaS provider makes.

Likewise, intermediary providers claiming to use a high-quality, but expensive, service as the classification backend may be tempted to instead use a cheaper or homegrown classification service. For example, Google’s Bidirectional Encoder Representations from Transformers (BERT) [5]

model for natural language processing (NLP) is used as a pretraining procedure, generating 768-dimensional embeddings for input text. Predictions using these high-dimensional embeddings are often of high quality, but can be more expensive to generate and train with. A classification provider claims to use BERT via Google’s NLP API in the backend for pretraining to classify input text, but instead uses a homegrown model to generate 50-dimensional embeddings. The classifications resulting from models with this pretraining procedure will likely be of sub-standard quality.

The aim of this paper is to define these and similar problems that may arise more formally and to propose methods that address each of them.

Fig. 1: A trusted intermediary queries the promised supplier when a request from is received.

Ii Problem Definition

We define a classification service supplier to be a service entity that exposes an API that uses a model that, when trained, computes a function which takes as input an input object (e.g., a sequence of DNA, an image, a sequence of text, an identity graph) and returns as output a classification vector , in which

corresponds to the model’s probability estimate of

being an instance of object class , which is described by the ground truth label vector . We define a classification service provider to be an intermediary entity that takes as input a classification query , performs some process, and returns as output a probability vector . If makes an accurate claim to be sourcing classification service from , then the process performed would be sending along to and defining to be the output received from . For now, we put aside scenarios where some transformation is applied by to the input or output, as these would likely require case-by-case solutions. Finally, we define a client to be an entity with an object that it wants a classification vector for.

There are many scenarios in which validating the QoS of a classification service supplier or provider becomes important. For the following scenarios, assume is untrained.

Fig. 2: An untrusted, cheating intermediary does not query the promised supplier when a request from is received, possibly returning cheaper classification results.

Scenario 1: S trusted, P untrusted, M blackbox

In this scenario, requests classification of object from , who in turn claims to use a service as a classification backend. also does not have access to . However, we consider several scenarios here with different types of interaction with . One scenario we consider is one where is capable of steganography and can in turn perform reasonable computations on its own. Another scenario is where has the ability to interact directly with and an instance of . This again may be broken into two scenarios, one where some modifications or features may be added, and one where that is not the case. A final scenario is one where quality-proofs by accompanying the output with metadata, such as a certificate or explanation of the classification.

Scenario 2: S trusted, P untrusted, M whitebox

Here, again requests classification of object from , but this time, the model is known. We now split this into two scenarios. In the first, we assume that is reasonably computable by . In the second, we assume that is not reasonably computable by , but that a decentralized system may compute it and communicate with .

Scenario 3: S untrusted, M blackbox

Now we consider a scenario in which makes a performance guarantee, but is untrusted. Hence, (which may be , in this case) must be able to verify these performance guarantees. In this scenario, is unknown, and we consider performance metrics of accuracy, robustness, and latency. In particular, we’re concerned with the absolute metrics as well as their consistency with respect to .

Iii Related Work

Machine Learning-as-a-Service (MLaaS) is a subset of cloud computing, for which Service Level Agreements and Quality of Service measures have been well-studied. The authors of [12] have proposed a framework for considering the technological, legal, consumer-adoption, and infrastructure security problems that arise in cloud computing, proposing that these issues necessitate research into topics such as optimal risk transfer and SLA contract design. We take this as a call-to-action, as we feel that SLAs for MLaaS applications can be better designed with respect to the possibility of intermediary fraud. The authors of [11] define web service level agreements formally with three entities: parties, which describe the service provider, the service consumer, and third parties; SLA parameters, which describe resource metrics and composite metrics, which provide insightful and contextual information where raw resource metrics fall short, and Service Level Objectives (SLOs), which are a set of formal if: then expressions. We acknowledge the necessity for this formal, deterministic language in our context as well when it is applicable, but also note that in some cases where only probabilistic measures are available for QoS, the SLAs which address these must be more expressive to accommodate. We also note that, for instance, Google’s ML Engine SLA only covers metrics such as uptime, and not metrics related to model accuracy or robustness444https://cloud.google.com/ml-engine/sla. Our work seeks to identify methods for determining whether some promise in a MLaaS provider’s SLA or other guarantee mechanism is fraudulent. The first method we describe uses the ideas of steganography in the deep learning domain, proposed by [3], to embed hidden messages into an input request that a supplier would correctly identify while a fraudulent provider would not be able to identify it. This work draws inspiration from steganographic methods for images that do not necessarily involve deep neural networks [13]

as well as autoencoders for dimensionality reduction

[8] to propose a deep neural network architecture that embeds a message image into a cover image such that the message image is not visible to a human eye, and can be extracted by a separate component of the network. Where supplier services offer explanations with their predictions, we propose the use of explainability rating for our service access framework. The author of [7] gives a clear overview of the challenges and current approaches to explainability and interpretability in machine learning, which we use. Supplier services also have limitations when it comes to robustness, and these limitations can be indicative of the robustness quality of the service provided. To study this, we reference the approach proposed in [6]

to generate adversarial examples with respect to different classes offered for classification by the supplier. In these models, we also utilize seeding capabilities offered by many machine learning libraries and frameworks such as Tensorflow


, PyTorch


, and Caffe

[9]. These seeding methods allow for reproducibility, giving the client or a federated accountability entity the ability to verify claims being made by MLaaS providers. In proposing a federated accountability entity, we build on the work by the authors of [2] and [4]—we conjecture that a permissioned blockchain with peers which verify model outputs and reach consensus on the validity of the output of a provider is a well-suited model for a federated accountability entity.

Fig. 3: We propose as a solution a set of five main methods described in section III.

Iv Proposed Solutions

In this section, we present an overview of our solutions to the scenarios described in II.

Iv-a Watermarking, Seeding, Benchmarking, and Explainability

In Scenario 1, we considered the case where the CaaS supplier is trusted, but is untrusted. Likewise, the supplier’s model

is blackbox; that is, the model’s architecture and hyperparameters are unknown. Within this scenarios, we identified several sub-scenarios that will allow for solutions involving different techniques and approaches.

Iv-A1 Deep Steganography

For our first sub-scenario, we consider the case where employs a model that supports deep steganography. Steganography, a term that dates back to 1499555https://en.wikipedia.org/wiki/Steganography, is the practice of concealing a message using a medium. Examples of steganographic messages are those written on envelopes in the area covered by postage stamps, or those written within the lowest bits of noisy images. Unfortunately, typical algorithmic steganography techniques fail here, since any steganographic protocol introduced by intended to be made known to is also known by . In order to address this, we build on a body of work on deep stegangraphy proposed by Shumeet Baluja in [3]. In this model, which takes an approach similar to auto-encoding networks, the hidden message is nearly impossible to detect and decode without explicit knowledge of the trained revealing/classification model . An outline of the procedure is as follows:

  1. creates a steganographic service that takes as input and returns as output , where are classification probabilities for object classes, and is the classification probability for the message class.

  2. defines a procedure for creating a model that will take as input an instance of any object class and return as output an instance of the message class that “looks” like the original object class—that is, would not classify it as the message class with high probability. This procedure can be likened to the generative adversarial models described by Goodfellow et al. in [6].

  3. leverages to generate instances of steganographic objects and queries with some random sequence of steganographic objects and nonsteganographic objects.

  4. If the classification results for the steganographic objects are mostly correct, it may be deemed that is likely not fraudulent in its claim. However, if the classification results for the steganographic objects are mostly incorrect, it may be deemed that is likely fraudulent and not using .

Fig. 4: The flow of our proposed steganography-based MLaaS accountability testing method.

The deep steganography model shown in 4 has three components, though they are trained as one model: the prep network, the hiding network, and the revealing network, which performs classification.

The prep network is responsible for distributing the bits of the message class instance among the object class instance, or “cover”, as well as efficiently encoding the message class instance. The hiding network takes as input the output of the prep network, which is the message class instance, processed for embedding into the cover, as well as the cover itself. This network outputs an instance of a “container” which is the desired steganographic object. is responsible for providing a cover and message, as well as running these two networks. The reveal network is run by

and takes as input the hiding network and decodes it to reveal the hidden message. The overall model implements the loss function

where is the cover instance, is the container instance, is the secret message, is the reconstructed message, and is a hyperparameter used to weigh reconstruction errors. Intuitively, this penalizes large distances between the input cover and the container instances, as well as large distances between the secret message and the reconstruction. -norm loss functions for different choices of could be used as well. As these networks are trained as one model, is intended to instantiate and train the full model and publicly release the prep network and hiding network for use by , as well as instances of the message class (which could conceivably be extended to multiple classes). In describing this procedure, we’ve assumed that generating a cheaper model that is not and that can discriminate between inputs of the message class and object class is not viable.

Iv-A2 Deterministic Benchmarking

For this sub-scenario, we consider the case where

makes use of seeding. A seed is a number or vector used to initialize a pseudorandom number generator. Hence, any pseudostochastic process (such as stochastic gradient descent) or pseudorandom generation (such as weight initialization) used in machine learning models can be made reproducible by seeding. As a result, we propose that a supplier

make available the seed configuration used to initialize for any provider , so that a client can initialize another instance directly with using the same seed configuration and compare pairs yielded from and . We have determinism due to seeding, so if each pair is identical, we can say with high probability that is not fraudulent. On the other hand, if any pair is not identical, we can say that is fraudulent.

Iv-A3 Probabilistic Benchmarking

Now we consider the case where , while trusted, has a strictly blackbox model. Because we’re unable to compare pairs exactly, we must settle for a more probabilistic approach of performance benchmarking. As an example, we consider accuracy as the performance metric under scrutiny. prepares pairs of identical inputs for which it has ground truth labels, sends and each one of each , retrieves the outputs , and calculates the overall accuracy for each model. If is shown to have lower accuracy than

(this may be evaluated with variance in mind, which is controlled by

), it is likely that is fraudulent. If they show roughly the same accuracy (again considered with respect to the size of ), then it’s likely that is not fraudulent.

Iv-A4 Metaresults

Finally, we consider the case where may provide information along with the results of a classification, such as an explanation of the results or a “quality assurance” key that may be used to verify a classification event with . In this case, would need to know of the quality assurance measures taken by in order to take advantage of them. This being the case, it would be very difficult for an untrusted provider to forge the metaresults returned from a verified classification event with . Likewise if indeed queried , all it would need to do is pass the metaresults along to . This method is predicated on the assumption that it is difficult for any provider without access to to create valid metaresults.

Iv-B Accountability with Known M

Let’s now consider the scenario in which is a whitebox model. That is, has made publicly known the architecture, hyperparameters, and training procedure. Here, we have two subscenarios. In the first, we assume that is also available or reasonably computable by . In this case, note that can use the performance metric comparison method described in IV-A3 by instantiating its own and comparing the metrics given by identical trials run with and its own model.

On the other hand, we consider a scenario in which is not available or reasonably computable by . This particular scenario is one of the more common scenarios in today’s CaaS market, along with blackbox scenarios, as the service suppiers often release research papers and open-source code detailing their work. Here, we introduce a decentralized system that functions as a trusted QoS auditor. The QoS auditor is a blockchain-based model with peers being CaaS accounters, which are oracles who compute and post transactions with data including performance metrics of the model and performance metrics of . Majority consensus mechanisms here are designed to ensure that accurate performance metrics of a service are maintained on-chain. Clients who desire a verification of a model send requests with tokens to the chain to receive verified performance metrics on , and oracles that cast votes on performance metrics of which are in the majority are rewarded with tokens. The blockchain is not open to the public, since the services provided to the client are more valuable than the tokens the client pays for verification. Hence, view access would be managed by a trusted gatekeeper. An assumption we make here is that the oracles are trusted and/or mutually untrusting, and would therefore not collude to cheat the system.

Fig. 5: An outline of a decentralized MLaaS service accountability system based on the permissioned blockchain model.

Iv-C Benchmarking Performance Claims of Untrusted S

Robustness is a desirable characteristic in classification models. Recent work by Ian Goodfellow and others explores the methods and implications of real world and computationally-generated adversarial examples against blackbox models, such as [14][10]. These models take advantage of the fact that the function

is differentiable and by studying the gradient with respect to the input, the output can be controlled in some sense. The canonical multilayer perceptron example from Goodfellow’s 2014 paper between an adversarial network and a discriminator network illustrates this. Suppose G is a differentiable function that represents the generator multilayer perceptron. In particular, if

is the set of model parameters for ,

is the prior probability distribution of the input

, then let map to the data space , described by a multilayer perceptron. This model wants to learn the generated distribution, , over the data. Now, if is the set of model parameters for and , let map to the space of real scalars, and note that is also described by a multilayer perceptron. Then represents the probability that came from the data, and not . We then formulate the minimax game to represent the objective, with respect to the value function , where is the true probability distribution of the data:

For our purposes, we are n concerned only with a dynamic generator model, where the discriminator model does not respond to adversarial adaptation. Hence, we want

where the sum being maximized is the expected absolute error of the discriminator. To illustrate the idea behind generative adversarial models for a robustness metric, consider figure 6.

Fig. 6: A visual representation of the process by which adversarial inputs are generated.

Here, is a binary classification model that is tasked with assigning probabilities of an input example being labeled as “cheetah” (class 1) or “mouse” (class 2). The model correctly predicts the label of “cheetah” for the input example above. However, we introduce an adversarial model for , , which uses the gradient of the outputs of to generate an additive transformation (seen in the center panel) which, when applied to the input, yields an output that is classified as “mouse” with high probability, though this adversarial example is clearly, to the human eye, still depicting a cheetah. To establish a measure of robustness, we concern ourselves with the gradient of convergence to a certain robustness threshold given an input image that is uniformly sampled from the space of all input images. For a simple guarantee metric here, we look at the average number of input queries the adversarial model needs to make to the blackbox model before we reach a certain classification probability for class , from a uniformly sampled image. In the figure 7, we see that an adversarial model has generated an example that has passed the robustness threshold for class 2.

Fig. 7: One way of testing the robustness of a classification service is to study the gradient of the outputs during adversarial training, with respect to each class.

In this scenario, may release the architecture for an adversarial model

, as well as a robustness score (perhaps a sigmoidal function of the average number of examples needed for

to generate before reaching a certain robustness threshold) for each class that the service offers classification for. This need not be limited to images, but can be extended to any type of input.

V Conclusion

In this paper, we’ve examined the problem of Machine Learning-as-a-Service (MLaaS) accountability and have proposed a suite of accountability methods that may be used to verify the Quality of Service (QoS) guaranteed by a MLaaS provider in their Service Level Agreements (SLAs) or otherwise. These methods build upon techniques in deep steganography, seeding for reproducibility in models, explainability in artificial intelligence (XAI), generative adversarial models for robustness measurements, and permissioned blockchains for federated accountability entities. With these methods, a client will be able to determine a probability of whether a MLaaS provider is cheating, and MLaaS providers will be able to use these methods to create improved SLAs that reflect respective verifiable QoS measures.


  • [1] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, et al. (2016) Tensorflow: a system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pp. 265–283. Cited by: §III.
  • [2] E. Androulaki, A. Barger, V. Bortnikov, C. Cachin, K. Christidis, A. De Caro, D. Enyeart, C. Ferris, G. Laventman, Y. Manevich, et al. (2018) Hyperledger fabric: a distributed operating system for permissioned blockchains. In Proceedings of the Thirteenth EuroSys Conference, pp. 30. Cited by: §III.
  • [3] S. Baluja (2017) Hiding images in plain sight: deep steganography. In Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), pp. 2069–2079. External Links: Link Cited by: §III, §IV-A1.
  • [4] C. Cachin (2016) Architecture of the hyperledger blockchain fabric. In Workshop on distributed cryptocurrencies and consensus ledgers, Vol. 310, pp. 4. Cited by: §III.
  • [5] J. Devlin, M. Chang, K. Lee, and K. Toutanova (2018) BERT: pre-training of deep bidirectional transformers for language understanding. CoRR abs/1810.04805. External Links: Link, 1810.04805 Cited by: §I.
  • [6] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio (2014) Generative adversarial nets. In Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger (Eds.), pp. 2672–2680. External Links: Link Cited by: §III, item 2.
  • [7] D. Gunning (2017) Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency (DARPA), nd Web 2. Cited by: §III.
  • [8] G. E. Hinton and R. R. Salakhutdinov (2006) Reducing the dimensionality of data with neural networks. science 313 (5786), pp. 504–507. Cited by: §III.
  • [9] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell (2014) Caffe: convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM international conference on Multimedia, pp. 675–678. Cited by: §III.
  • [10] A. Kurakin, I. J. Goodfellow, and S. Bengio (2016) Adversarial examples in the physical world. CoRR abs/1607.02533. External Links: Link, 1607.02533 Cited by: §IV-C.
  • [11] H. Ludwig, A. Keller, A. Dan, R. P. King, and R. Franck (2003) Web service level agreement (wsla) language specification. Ibm corporation, pp. 815–824. Cited by: §III.
  • [12] S. Marston, Z. Li, S. Bandyopadhyay, J. Zhang, and A. Ghalsasi (2011) Cloud computing—the business perspective. Decision support systems 51 (1), pp. 176–189. Cited by: §III.
  • [13] T. Morkel, J. H. Eloff, and M. S. Olivier (2005) An overview of image steganography.. In ISSA, pp. 1–11. Cited by: §III.
  • [14] N. Papernot, P. D. McDaniel, I. J. Goodfellow, S. Jha, Z. B. Celik, and A. Swami (2016) Practical black-box attacks against deep learning systems using adversarial examples. CoRR abs/1602.02697. External Links: Link, 1602.02697 Cited by: §IV-C.
  • [15] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer (2017) Automatic differentiation in pytorch. Cited by: §III.