1 Introduction
Neural networks (NNs) have led to stateoftheart performance in myriad areas, such as computer vision, speech recognition, and machine translation. Due to the presence of millions of parameters and floatingpoint operations (FLOPs), NNs are typically too computationallyintensive to be deployed on resourceconstrained platforms. Many efforts have been made recently to design compact NN architectures. Examples include NNs presented in
[24, 21, 28], which have significantly cut down the computation cost and achieved a much more favorable tradeoff between accuracy and efficiency. However, a compact model design still faces challenges upon deployment in realworld applications [33]:
Different platforms have diverging hardware characteristics. It is hard for a single NN architecture to run optimally on all the different platforms. For example, a Hexagon v62 DSP prefers convolution operators with a channel size that is a multiple of 32, as shown later, whereas this may not be the case on another platform.

Realworld applications may face very different constraints. For example, realtime video frame analysis may have a very strict latency constraint, whereas InternetofThings (IoT) edge device designers may care more about runtime energy consumption for longer battery life. It is infeasible to have one NN that can meet all these constraints simultaneously. This makes it necessary to adapt the NN architecture to the specific use scenarios before deployment.
There are two common practices for tackling these challenges. The first practice is to manually craft the architectures based on the characteristics of a given platform. However, such a trialanderror methodology might be too timeconsuming for largescale crossplatform NN deployment and may not be able to effectively explore the design space. Moreover, it also requires substantial knowledge of the hardware details and driver libraries. The other practice focuses on platformaware neural architecture search (NAS) and sequential modelbased optimization (SMBO) [37]. Both NAS and SMBO require computationallyexpensive network training and measurement of network performance metrics (e.g., latency and energy) throughout the entire search and optimization process. For example, the latencydriven mobile NAS (MNAS) architecture requires hundreds of GPU hours to develop [28], which becomes unaffordable when targeting numerous platforms with various resource budgets. Moreover, it may be difficult to implement the new celllevel structures discovered by NAS because of their complexity [37].
In this paper, we propose an efficient, scalable, and automated NN architecture adaptation methodology. We refer to this methodology as Chameleon. It does not rely on new celllevel building blocks nor does it use computationallyintensive reinforcement learning (RL) techniques. Instead, it takes into account the traits of the hardware platform to allocate computation resources accordingly when searching the design space for the given NN architecture with existing building blocks. This adaptation reduces search time. It employs predictive models (namely accuracy, latency, and energy predictors) to speed up the entire search process by enabling immediate performance metric estimation. The accuracy and energy predictors incorporate Gaussian process (GP) regressors augmented with Bayesian optimization and imbalanced quasi MonteCarlo (QMC) sampling. It also includes an operator latency lookup table (LUT) in the latency predictor for fast, yet accurate, latency estimation. It consistently delivers higher accuracy and less runtime latency against stateoftheart handcrafted and automatically searched models across several hardware platforms (e.g., mobile CPU, DSP, Intel CPU, and Nvidia GPU) under different resource constraints. For example, on a Samsung Galaxy S8 with Snapdragon 835 CPU, our adapted model yields 1.7% higher top1 accuracy and 1.75
speedup compared to MnasNet [28].Our contributions can be summarized as follows:

We show that computation distribution is critical to model performance. By leveraging existing efficient building blocks, we adapt models with significant improvements over stateoftheart handcrafted and automatically searched models under a wide spectrum of devices and resource budgets.

We propose a novel algorithm that searches for optimal architectures through efficient accuracy and resource predictors. At the core of our algorithm lies an accuracy predictor built based on GP with Bayesian optimization that enables a more effective search over a similar space to RLbased NAS.

Our proposed algorithm is efficient and scalable. With a onetime building cost, it only takes minutes to search for models under different platforms/constraints, thus making them suitable for largescale heterogeneous deployment.
2 Related work
Efficient NN design and deployment is a vibrant field. We summarize the related work next.
Model simplication: An important direction for efficient NN design is model simplification. Network pruning [11, 29, 6, 34, 32, 7] has been a popular approach for removing redundancy in NNs. For example, NetAdapt [33] utilizes a hardwareaware filter pruning algorithm and achieves up to 1.2 speedup for MobileNetV2 on the ImageNet dataset [8]. AMC [13] employs RL for automated model compression and achieves 1.53 speedup for MobileNet_v1 on a Titan XP CPU. Quantization [10, 17] has also emerged as a powerful tool for significantly cutting down computation cost with no or little accuracy loss. For example, Zhu et al. [36] show that there is only a 2% top5 accuracy loss for ResNet18 when using a 3bit representation for weights compared to its fullprecision counterpart.
Compact architecture: Apart from simplifying existing models, handcrafting more efficient building blocks and operators for mobilefriendly architectures can also substantially improve the accuracyefficiency tradeoffs [18, 30]. For example, at the same accuracy level, MobileNet [15] and ShuffleNet [31] cut down the computation cost substantially compared to ResNet [12] by utilizing depthwise convolution and lowcost group convolution, respectively. Their successors, MobileNetV2 [24] and ShuffleNetV2 [21], further shrink the model size while maintaining or even improving accuracy. In order to deploy these models on different realworld platforms, Andrew et al. propose linear scaling in [15]. This is a simple but widelyused method to accommodate various latency constraints. It relies on thinning a network uniformly at each layer or reducing the input image resolution.
NAS and SMBO: Platformaware NAS and SMBO have emerged as a promising direction for automating the synthesis flow of a model based on direct metrics, making it more suitable for deployment [14, 22, 3, 19, 20, 4]. For example, MnasNet [28]
yields an absolute 2% top1 accuracy gain compared to MobileNetV2 1.0x with only a 1.3% latency overhead on Google Pixel 1 using TensorFlow Lite. As for SMBO, Stamoulis et al. use a Bayesian optimization approach and reduce the energy consumed for VGG19
[25] on a mobile device by up to 6. Unfortunately, it is difficult to scale NAS and SMBO for largescale platform deployment, since the entire search and optimization needs to be conducted once per network per platform per use case.3 Methodology
We first give a highlevel overview of the Chameleon framework, after which we zoom into predictive models.
3.1 Platformaware Model Adaptation
We illustrate the Chameleon approach in Fig. 1
. The adaptation step takes a default NN architecture and a specific use scenario (i.e., platform and resource budget) as inputs and generates an adapted architecture as output. Chameleon searches for a variant of the base NN architecture that fits the use scenario through efficient evolutionary search (EES). EES is based on an adaptive genetic algorithm
[26], where the gene of an NN architecture is represented by a vector of hyperparameters (e.g., #Filters and #Bottlenecks), denoted as
, where is the number of hyperparameters of interest. In each iteration, EES evaluates the fitness of each NN architecture candidate based on inputs from the predictive models, and then selects architectures with the highest fitness to breed the next generation using mutation and crossover operators. EES terminates after a predefined number of iterations. Finally, Chameleon rests at an adapted NN architecture for the target platform and use scenario.We formulate EES as a constrained optimization problem. The objective is to maximize accuracy under a given resource constraint on a target platform:
(1) 
(2) 
where , , , and refer to mapping of x to network accuracy, mapping of x to the network performance metric (e.g., latency or energy), target platform, and resource constraint determined by the use scenario (e.g., 20ms), respectively. We merge the resource constraint as a regularization term in the fitness function as follows:
(3) 
where is the Heaviside step function, and and are positive constants. Consequently, the aim is to find the network gene x that maximizes R:
(4) 
We next estimate and to solve the optimization problem. Choices of depends on constraints of interest. In this work, we mainly study based on direct latency and energy measurements, as opposed to an indirect proxy, such as FLOPs, that has been shown to be suboptimal [33]. Thus, for each NN candidate x, we need three performance metrics to calculate its : accuracy, latency, and energy consumption.
Extracting the above metrics through network training and direct measurements on hardware, however, is too timeconsuming [19]. To speed up this process, we bypass the training and measurement process by leveraging accuracy, latency, and energy predictors, as shown in Fig. 1. These predictors enable metric estimation in less than one CPU second. We give details of our accuracy, latency, and energy predictors next.
3.2 Efficient Accuracy Predictor
To significantly speed up NN architecture candidate evaluation, we utilize an accuracy predictor to estimate the final accuracy of a model without actually training it. There are two desired objectives of such a predictor:

Reliable prediction: The predictor should minimize the distance between predicted and real accuracy, and rank models in the same order as their real accuracy.

Sample efficiency: The predictor should be built with as few trained network architectures as possible. This saves computational resources in training instance generation.
Next, we explain how we tackle these two objectives through GP regression and Bayesian optimization based sample architecture selection for training.
3.2.1 Gaussian Process Model
We choose a GP regressor as our accuracy predictor to model as:
(5)  
where i denotes the index of a training vector among s training vectors and ’s refer to noise variables with independent distributions. is drawn from a GP prior characterized by covariance matrix
. We use a radial basis function kernel for
:(6) 
A GP regressor provides two benefits. First, it offers reliable predictions when training data are scarce. As an example, we compare several regression models for MobileNetV2 accuracy prediction in Fig. 2. The GP regressor has the lowest mean squared error (MSE) among all six regression models. Second, a GP regressor produces predictions with uncertainty estimations, which offers additional guidance for new sample architecture selection for training. This helps boost the convergence speed and improves sample efficiency, as shown next.
3.2.2 Iterative Sample Selection
As mentioned earlier, our objective is to train the GP predictor with as few NN architecture samples as possible. We summarize our efficient sample generation and predictor training method in Algorithm 1. Since the number of unique architectures in the adaptation search space can still be large, we first sample representative architectures from this search space to form an architecture pool. We adopt the QMC sampling method [2], which is known to provide similar accuracy to Monte Carlo sampling but with orders of magnitude fewer samples. We then build the accuracy predictor iteratively. In each iteration, we use the current predictor as a guide for selecting additional sample architectures to add to the training set. We train these sample architectures and then upgrade the predictor based on new architectureaccuracy observations.
To improve sample efficiency, we further incorporate Bayesian optimization into the sample architecture selection process. This enables Algorithm 1 to converge faster with fewer samples [27]. Specifically, we select both exploitation and exploration samples in each iteration:

Exploitation samples: We choose sample architectures with high accuracy/FLOPs ratios. These desirable architectures, or ‘samples of interest,’ are likely to yield higher accuracy with less computation cost. They typically fall in the top left part of the accuracyFLOPs tradeoff graph, as shown in Fig. 3.

Exploration samples: We choose samples with large uncertainty values. This helps increase the prediction confidence level of the GP regressor over the entire adaptation search space [27].
Based on these rules, we show the selected sample architectures from the architecture space in Fig. 3. It can be observed that we have higher sampling density in the area of ‘samples of interest,’ where adaptation typically rests.
3.3 Latency Predictor
Recently, great efforts have been made towards developing more efficient and compact NN architectures for better accuracylatency tradeoffs. Most of them optimize NNs based on FLOPs, which is an oftenused proxy [37, 35]. However, optimization based on direct latency measurement instead of FLOPs can better explore hardware traits and hence offer additional advantages. To illustrate this point, we show the measured latency surface of a 11 convolution operator with varying numbers of input and output channels in Fig. 4. The latency is measured using the Caffe2 library on a Snapdragon 835 mobile CPU and a Hexagon v62 DSP. It can be observed that FLOPs, though generally effective in providing guidance for latency reduction, may not capture desired hardware characteristics upon model deployment.
(a)  (b) 
56 and stride 1 on (a) Snapdragon 835 GPU and (b) Hexagon v62 DSP.
Extracting the latency of an NN architecture during EES execution through direct measurement, however, is very challenging. Platformspecific latency measurements can be slow and difficult to parallelize, especially when the number of available devices is limited [33]. Therefore, largescale latency measurements might be expensive and become the computation bottleneck for Chameleon. To speed up this process, we construct an operator latency LUT for the target device to enable fast and reliable latency estimations of NN candidates throughout EES execution. The LUT is supported by an operator latency database, where we benchmark operatorlevel latency on real devices with different input dimensions. For an NN model, we sum up all its operatorlevel latencies as an estimate of the networklevel latency:
(7) 
Building the operator latency LUT for a given device is just a onetime cost, but can be substantially reused across various NN models, different tasks, and different applications of architecture search, model adaptation, and hyperparameter optimization.
Latency estimation based on operator latency LUT can be completed in less than one CPU second, as opposed to real measurements on hardware that usually take minutes. Moreover, it also supports parallel query, hence significantly enhancing simultaneous latency extraction efficiency across multiple NN candidates. This enables latency estimation in EES to consume very little time. We compare the predicted latency value against real measurement in Fig. 5. The distance between the predicted value and real measurement is quite small. We plan to opensource the operator latency LUT for mobile CPUs and DSPs.
3.4 Energy Predictor
Batterypowered devices (e.g., smart watches, mobile phones, and AR/VR products) have limited energy budgets. Thus, it is important to adjust and finetune the NN model to fit the energy constraint before deployment [5]. To solve this problem, we incorporate energy constraintdriven adaptation in Chameleon for different platforms and use scenarios.
We build the energy predictor in a similar manner to the accuracy predictor. We build a GP energy predictor that incorporates Bayesian optimization and acquire the energy values from direct measurements on hardware. However, for the energy predictor, we only select exploration samples in each iteration (i.e., samples with large uncertainty). The concept of ‘samples of interest’ is not applicable in this scenario. We show the performance of our energy predictor for MobileNetV2 in Fig. 6.
4 Experiments
In this section, we apply Chameleon to various NN architectures over a wide spectrum of platforms and resource budgets. We use PyTorch
[23] and Caffe2 for the implementation. For mobile platforms, we train the model with full precision (float32) and quantize to int8. We report full precision results for accuracy comparison but we see no or minimal loss of accuracy for quantized models with fake quantization finetuning. We benchmark the latency using Caffe2 int8 backend with Facebook AI Performance Evaluation Platform [1]. We report results on the ImageNet dataset [8], which is a wellknown benchmark consisting of 1.2M training and 50K validation images classified into 1000 distinct classes. Thus, all our models share the same 1000 final output channels. We randomly reserve 50K images from the training set (50 images per class) to build the accuracy predictor. We measure latency and energy with the same batch size of 1.
In the EES process, we set and for latency and energyconstrained adaptation, respectively, and . We set the initial QMC architecture pool size to . We generate the accuracy and energy predictors with 240 samples selected from the architecture pool. Latency estimation is supported by a operator latency LUT with approximately 350K records. In evolutionary search, the population size of each generation is set to 96. We pick the top 12 candidates for the next generation. The total number of search iterations is set to 100. We present our experimental results next.
4.1 Adaptation for Mobile Models
This section presents the adaptation results leveraging the efficient inverse residual building block from MobileNetV2, which is the stateoftheart handcrafted architecture for mobile platforms. It utilizes inverted residual and linear bottleneck to significantly cut down on the number of operations and memory needed per inference [24]. We first show the adaptation search space used in our experiments in Table 1, where , , , and refer to the expansion factor, number of output channels, number of repeated blocks, and stride, respectively. The adaptation search range for each hyperparameter is denoted as [], where denotes the lower bound and the upper bound. The default values used in MobileNetV2 1.0x are also shown next to our search ranges, following the notation rule used in [24].
Input resolution  224 [96, 224]  

stage  
conv2d    32 [8,48]  1  2 
bottleneck  1  16 [8,32]  1  1 
bottleneck  6 [2,6]  24 [8,40]  2 [1,2]  2 
bottleneck  6 [2,6]  32 [8,48]  3 [1,3]  2 
bottleneck  6 [2,6]  64 [16,96]  4 [1,4]  2 
bottleneck  6 [2,6]  96 [32,160]  3 [1,3]  1 
bottleneck  6 [2,6]  160 [56,256]  3 [1,3]  2 
bottleneck  6 [2,6]  320 [96,480]  1  1 
conv2d    1280 [1024,2048]  1  1 
avgpool      1   
fc    1000     
We target two different platforms in our experiments: Snapdragon 835 mobile CPU (on a Samsung S8) and Hexagon v62 DSP (800 MHz frequency with internal NN library implementation). We evaluate Chameleon under a wide range of latency constraints: 4ms, 6ms, 10ms, 15ms, 20ms, and 30ms.
We compare our adapted ChamNetMobile models with stateoftheart models, including MobileNetV2, ShuffleNetV2, and MnasNet in Fig. 7 and 8 on mobile CPU and DSP, respectively. Models discovered by Chameleon outperform all the previous manually designed or automatically searched architectures on both platforms consistently. Our ChamNet has an 8.5% absolute accuracy gain compared to MobileNetV2 0.5x with an input resolution of 9696, while both models share the same 4.0ms runtime latency on the mobile CPU.
4.2 Adaptation for Server Models
We also evaluate Chameleon for server models on both CPU and GPU. We choose residual building blocks from ResNet as the base for adaptation because of its high accuracy and widespread usage in the cloud. The target platforms are the Intel Xeon Broadwell CPU with 2.4 GHz frequency and Nvidia GTX 1060 GPU with 1.708 GHz frequency. We use CUDA 8.0 and CUDNN 5.1 in our experiments.
We show the detailed adaptation search space in Table 2, where the notations are identical to the ones in Table 1. This search space for ChamNetRes includes #Filters, expansion factor, and #Bottlenecks per layer. Note that the maximum number of layers in the adaptation search space is 152, which is the largest reported depth for the ResNet family [12].
Input resolution  224  

stage  t  c  n  s 
conv2d    64 [16,64]  1  2 
bottleneck  4 [2,6]  64 [16,64]  3 [1,3]  2 
bottleneck  4 [2,6]  128 [32,128]  4 [1,8]  2 
bottleneck  4 [2,6]  256 [64,256]  6 [1,36]  2 
bottleneck  4 [2,6]  512 [128,512]  3 [1,3]  2 
avgpool      1   
fc    1000     
We set a wide spectrum of latency constraints for both Intel CPU and Nvidia GPU to demonstrate the generality of our framework. The latency constraints for the CPU are 50ms, 100ms, 200ms, and 400ms, while the constraints for the GPU are 2.5ms, 5ms, 10ms, and 15ms. We compare the adapted model with ResNets on CPU and GPU in Fig. 9 and 10, respectively. Again, Chameleon improves the accuracy by a large margin on both platforms.
4.3 EnergyDriven Adaptation
Next, we study the energyconstrained use scenario for ChamNetMobile on mobile phones. We obtain the energy measurements from the Snapdragon 835 CPU. We first replace the battery of the phone with a Monsoon power monitor with a constant voltage output of 4.2V. During measurements, we ensure the phone is kept in the idle mode for 18 seconds, then run the network 1000 times and measure the current at 200s intervals. We then deduct the baseline current from the raw data in a postprocessing step and calculate the energy consumption per forward pass.
To demonstrate Chameleon’s applicability under a wide range of constraints, we set six different energy constraints in our experiment: 15mJ, 30mJ, 50mJ, 75mJ, 100mJ, and 150mJ. We use the same adaptation search space as ChamNetMobile.
We show the accuracy of ChamNet and compare it with MobileNetV2 in Fig. 11. We achieve significant improvement in accuracyenergy tradeoffs. For example, compared to the MobileNetV2 0.75x with input resolution 9696 baseline (58.8% accuracy at 19mJ per run), our adapted models achieves 60.0% accuracy at only 14mJ per run. Therefore, our model is able to reduce energy by 26% while simultaneously increasing accuracy by 1.2% on a smartphone.
Model  Method  Direct  Scaling  Latency(ms)  Top1 
metrics based  complexity  accuracy (%)  
MobileNetV2 1.3x [24]  Manual  33.8  74.4  
ShuffleNetV2 2.0x [21]  Manual  Y  33.3  74.9  
ChamNetA  EES  Y  29.8  75.4  
MobileNetV2 1.0x [24]  Manual  21.7  71.8  
ShuffleNetV2 1.5x [21]  Manual  Y  22.0  72.6  
CondenseNet (G=C=4) [16]  Manual  28.7  73.8  
MnasNet 1.0x [28]  RL  Y  23.8  74.0  
AMC [13]  RL  Y  
MorphNet [9]  Regularization  N  
ChamNetB  EES  Y  19.9  73.8  
MobileNetV2 0.75x [24]  Manual  16.6  69.8  
ShuffleNetV2 1.0x [21]  Manual  Y  14.9  69.4  
MnasNet 0.75x [28]  RL  Y  18.4  71.5  
NetAdapt [33]  Pruning  Y  16.6 (63.6)  70.9  
ChamNetC  EES  Y  15.0  71.9  
MobileNetV2 0.5x [24]  Manual  10.6  65.4  
MnasNet 0.35x [28]  RL  Y  10.7  62.4  
ChamNetD  EES  Y  10.0  69.0  
MobileNetV2 0.35x [24]  Manual  9.3  60.3  
ShuffleNetV2 0.5x [21]  Manual  Y  8.8  60.3  
ChamNetE  EES  Y  6.1  64.1  
We report five of our ChamNet models with A30ms, B20ms, C15ms, D10ms, and E6ms latency constraints. , , and refer to the number of network  
models, distinct platforms, and use scenarios with different resource budgets, respectively. : Ref. [33] reports 63.6ms latency with TensorFlow Lite on Pixel 1.  
For a fair comparison, we report the corresponding latency in our experimental setup with Caffe2 on Samsung Galaxy S8 with Snapdragon 835 CPU. : The  
inference engine is faster than other models. 
4.4 Comparisons with Alternative Adaptation and Compression Approaches
In this section, we compare Chameleon with relevant work, including:

MNAS [28]: this is an RLbased NN architecture search algorithm for mobile devices.

AutoML model compression (AMC) [13]: this is an RLbased automated network compression method.

NetAdapt [33]: this is a platformaware filter pruning algorithm that adapts a pretrained network to a specific hardware under a given latency constraint.

MorphNet [9]: this is a network simplification algorithm based on sparsifying regularization.
Table 3 compares different model compression, adaptation, and optimization approaches on the Snapdragon 835 CPU, where , , and refer to the number of network models, distinct platforms, and use scenarios with different resource budgets, respectively. ChamNet yields the most favorable accuracylatency tradeoffs among all models. Moreover, most existing approaches need to be executed at least once per network per device per constraint [28, 33, 9, 13], and thus have a total training cost of . Chameleon only builds accuracy predictors and resource predictors (e.g., latency LUT), and thus reduces the cost to . The search cost is negligible once the predictors are built. Such onetime costs can easily be amortized when the number of use scenarios scales up, which is generally the case for largescale heterogeneous deployment.
We compare the FLOPs distribution at each stage (except for avgpool and fc) for MobileNetV2 0.5x and ChamNet with similar latency in Fig. 12. Our model achieves 71.9% accuracy at 15.0ms compared to the MobileNetV2 that has 69.8% accuracy at 16.6ms. We have two observations:

ChamNet redistributes the FLOPs from the early stages to late stages. We hypothesize that this is because when feature map size is smaller in the later stages, more filters or a larger expansion factor are needed to propagate the information.

ChamNet has a better utilization of computation resources. We estimate the CPU processing speed at each stage using the ratio of FLOPs and latency, as shown with the green curve in Fig. 12. The operators in early stages with large input image size have significantly lower FLOPs per second, hence incur higher latency given the same computational load. A possible reason is incompatibility between cache capacity and large image size. Through better FLOPs redistribution, ChamNet enables 2.1% accuracy gain while reducing runtime latency by 5% against the baseline MobileNetV2.
5 Conclusions
This paper proposed a platformaware model adaptation framework called Chameleon that leverages efficient building blocks to adapt a model to different realworld platforms and use scenarios. This framework is based on very efficient predictive models and thus bypasses the expensive training and measurement process. It significantly improves accuracy without incurring any latency or energy overhead, while taking only CPU minutes to perform an adaptation search. At the same latency or energy, it achieves significant accuracy gains relative to both handcrafted and automatically searched models.
References
 [1] Facebook AI performance evaluation platform. https://github.com/facebook/FAIPEP, 2018.
 [2] S. Asmussen and P. W. Glynn. Stochastic Simulation: Algorithms and Analysis, volume 57. Springer Science & Business Media, 2007.
 [3] B. Baker, O. Gupta, R. Raskar, and N. Naik. Accelerating neural architecture search using performance prediction. arXiv preprint arXiv:1705.10823, 2017.
 [4] J. S. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl. Algorithms for hyperparameter optimization. In Proc. Advances in Neural Information Processing Systems, pages 2546–2554, 2011.
 [5] E. Cai, D.C. Juan, D. Stamoulis, and D. Marculescu. Neuralpower: Predict and deploy energyefficient convolutional neural networks. arXiv preprint arXiv:1710.05420, 2017.
 [6] X. Dai, H. Yin, and N. K. Jha. NeST: A neural network synthesis tool based on a growandprune paradigm. arXiv preprint arXiv:1711.02017, 2017.
 [7] X. Dai, H. Yin, and N. K. Jha. Grow and prune compact, fast, and accurate LSTMs. arXiv preprint arXiv:1805.11797, 2018.

[8]
J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, and L. FeiFei.
ImageNet: A largescale hierarchical image database.
In
Proc. IEEE Conf. Computer Vision and Pattern Recognition
, pages 248–255. Ieee, 2009.  [9] A. Gordon, E. Eban, O. Nachum, B. Chen, H. Wu, T.J. Yang, and E. Choi. MorphNet: Fast & simple resourceconstrained structure learning of deep networks. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2018.
 [10] S. Han, H. Mao, and W. J. Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding. arXiv preprint arXiv:1510.00149, 2015.
 [11] S. Han, J. Pool, J. Tran, and W. Dally. Learning both weights and connections for efficient neural network. In Proc. Advances in Neural Information Processing Systems, pages 1135–1143, 2015.
 [12] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, pages 770–778, 2016.
 [13] Y. He, J. Lin, Z. Liu, H. Wang, L.J. Li, and S. Han. AMC: AutoML for model compression and acceleration on mobile devices. In Proc. European Conf. Computer Vision, pages 784–800, 2018.

[14]
J. M. HernándezLobato, M. A. Gelbart, R. P. Adams, M. W. Hoffman, and
Z. Ghahramani.
A general framework for constrained Bayesian optimization using
informationbased search.
J. Machine Learning Research
, 17(1):5549–5601, 2016.  [15] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861, 2017.
 [16] G. Huang, S. Liu, L. van der Maaten, and K. Q. Weinberger. CondenseNet: An efficient DenseNet using learned group convolutions. arXiv preprint arXiv:1711.09224, 2017.
 [17] I. Hubara, M. Courbariaux, D. Soudry, R. ElYaniv, and Y. Bengio. Binarized neural networks. In Proc. Advances in Neural Information Processing Systems, pages 4107–4115, 2016.
 [18] F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer. SqueezeNet: AlexNetlevel accuracy with 50x fewer parameters and 0.5 MB model size. arXiv preprint arXiv:1602.07360, 2016.
 [19] C. Liu, B. Zoph, J. Shlens, W. Hua, L.J. Li, L. FeiFei, A. Yuille, J. Huang, and K. Murphy. Progressive neural architecture search. arXiv preprint arXiv:1712.00559, 2017.
 [20] H. Liu, K. Simonyan, and Y. Yang. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055, 2018.
 [21] N. Ma, X. Zhang, H.T. Zheng, and J. Sun. ShuffleNet V2: Practical guidelines for efficient CNN architecture design. arXiv preprint arXiv:1807.11164, 2018.
 [22] D. Marculescu, D. Stamoulis, and E. Cai. Hardwareaware machine learning: Modeling and optimization. arXiv preprint arXiv:1809.05476, 2018.
 [23] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in PyTorch. In Proc. Neural Information Processing Systems Workshop on Autodiff, 2017.
 [24] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.C. Chen. Inverted residuals and linear bottlenecks: Mobile networks for classification, detection and segmentation. arXiv preprint arXiv:1801.04381, 2018.
 [25] K. Simonyan and A. Zisserman. Very deep convolutional networks for largescale image recognition. arXiv preprint arXiv:1409.1556, 2014.

[26]
M. Srinivas and L. M. Patnaik.
Adaptive probabilities of crossover and mutation in genetic algorithms.
IEEE Trans. Systems, Man, and Cybernetics, 24(4):656–667, 1994.  [27] D. Stamoulis, E. Cai, D.C. Juan, and D. Marculescu. Hyperpower: Powerand memoryconstrained hyperparameter optimization for neural networks. In Proc. IEEE Europe Conf. & Exihibition on Design, Automation & Test, pages 19–24, 2018.
 [28] M. Tan, B. Chen, R. Pang, V. Vasudevan, and Q. V. Le. MnasNet: Platformaware neural architecture search for mobile. arXiv preprint arXiv:1807.11626, 2018.
 [29] W. Wen, C. Wu, Y. Wang, Y. Chen, and H. Li. Learning structured sparsity in deep neural networks. In Proc. Advances in Neural Information Processing Systems, pages 2074–2082, 2016.
 [30] B. Wu, A. Wan, X. Yue, P. Jin, S. Zhao, N. Golmant, A. Gholaminejad, J. Gonzalez, and K. Keutzer. Shift: A zero FLOP, zero parameter alternative to spatial convolutions. arXiv preprint arXiv:1711.08141, 2017.

[31]
Z. Xiangyu, Z. Xinyu, L. Mengxiao, and S. Jian.
ShuffleNet: An extremely efficient convolutional neural network for mobile devices.
In Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2017.  [32] T.J. Yang, Y.H. Chen, and V. Sze. Designing energyefficient convolutional neural networks using energyaware pruning. arXiv preprint arXiv:1611.05128, 2016.
 [33] T.J. Yang, A. Howard, B. Chen, X. Zhang, A. Go, M. Sandler, V. Sze, and H. Adam. NetAdapt: Platformaware neural network adaptation for mobile applications. In Proc. European Conf. Computer Vision, volume 41, page 46, 2018.
 [34] T. Zhang, K. Zhang, S. Ye, J. Li, J. Tang, W. Wen, X. Lin, M. Fardad, and Y. Wang. ADAMADMM: A unified, systematic framework of structured weight pruning for DNNs. arXiv preprint arXiv:1807.11091, 2018.
 [35] Y. Zhou, S. Ebrahimi, S. Ö. Arık, H. Yu, H. Liu, and G. Diamos. Resourceefficient neural architect. arXiv preprint arXiv:1806.07912, 2018.
 [36] C. Zhu, S. Han, H. Mao, and W. J. Dally. Trained ternary quantization. arXiv preprint arXiv:1612.01064, 2016.
 [37] B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le. Learning transferable architectures for scalable image recognition. arXiv preprint arXiv:1707.07012, 2(6), 2017.