Towards Evaluating Driver Fatigue with Robust Deep Learning Models

07/16/2020 ∙ by Ken Alparslan, et al. ∙ Drexel University 0

In this paper, we explore different deep learning based approaches to detect driver fatigue. Drowsy driving results in approximately 72,000 crashes and 44,000 injuries every year in the US and detecting drowsiness and alerting the driver can save many lives. There have been many approaches to detect fatigue, of which eye closedness detection is one. We propose a framework to detect eye closedness in a captured camera frame as a gateway for detecting drowsiness. We explore two different datasets to detect eye closedness. We develop an eye model by using new Eye-blink dataset and a face model by using the Closed Eyes in the Wild (CEW). We also explore different techniques to make the models more robust by adding noise. We achieve 95.84 detects eye blinking and 80.01 blinking. We also see that we can improve our accuracy on the face model by 6 when we add noise to our training data and apply data augmentation. We hope that our work will be useful to the field of driver fatigue detection to avoid potential vehicle accidents related to drowsy driving.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

page 5

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Drowsy driving results in approximately 72,000 crashes and 44,000 injuries every year in the US [1]. Driving after going more than 20 hours without sleep is the equivalent of driving with a blood-alcohol concentration of 0.08% – the U.S. legal limit. A driver is three times more likely to be in a car crash if fatigued compared to a non-fatigued driver [2].The problem of drowsy driving is especially important for night-drivers and long-haul drivers. Some car manufacturers have started to implement pressure sensors on the steering wheel to detect whether the driver is engaged with the wheel or not, but they remained limited and such sensors are usually sold as luxury in a high-end sports car. We aim to develop a more robust, more cost effective and ways of detecting driver fatigue for everyone who drives, but not just for luxurious car owners.

In this study, we explore building a deep learning model. While our main goal is to build a robust deep learning model with less false positives and negatives, we also consider the size aspect of the models. We also aim to develop a model that can potentially be used in a very constrained environment such as a mobile application because we foresee that a mobile application can be used by everyone for fatigue detection to alert drowsy drivers. Luxurious high-end car companies are implementing driver fatigue warning systems[29] and providing built-in features to detect driver distractions, but not everyone can afford those cars due to their high price. By developing a model that can run in a mobile application, we are also making this alert system available to everyone with a mobile phone. A robust model is needed to reduce the false negatives as well as the false positives. False negatives are the cases where the driver is sleeping when driving but the model doesn’t alert the driver. False positives are the cases where the model is alerting the driver even though the driver is alert and focused. Fatigue detection for drivers problem becomes non-trivial because peoples lives are involved and the model should be robust enough to eliminate false negatives as well as false positives. We also need to consider the fact that alerting drivers when not needed is as dangerous as failing to alert when needed. Alerting drivers when not needed can create unnecessary distraction that otherwise would have not existed. Avoiding false negatives and false positives requires us to build robust deep learning models for this task. Even though we are not directly aiming to create a mobile application, we should take into consideration the limited processing power and memory available at a mobile phone. Even though the Moore’s empirical studies show that the memory and CPU get more cost effective each year, we still should bear in mind that our robust model should be able to fit into a mobile phone and run as a mobile application.

The task of eye closedness detection is to decide whether the eyes are closed [2]. Detecting eyes’ closedness has a variety of applications. Driver fatigue detection is one of them. This task is a challenging problem since the degree of eye closedness may be different for each face and there are many ambient factors that may significantly change the appearance of the eyes, such as lighting, pose, scales and imaging conditions [14] In addition, inaccurate eye localization may introduce a great difficulty to this problem. A recent push in the literature is to use deep learning based models in order to detect eyes closedness. One problem with the mentioned approach is that deep learning models can be also easily fooled as demonstrated by [3][4][5]. Our goal in this paper is to create a robust fatigue detection framework with high accuracy. In this study, we also consider those perturbations such as bad lightning, pose, and shadows as adversarial attacks and try to provide solutions for the problem of robust eye closedness detection in real life via adversarial training.

The task of eye localization is made more difficult by occlusions, problematic lighting, pose, and motion blur which can cause an adversarial setting for the model. Adversarial attacks are defined as perturbations to the sample image and they can be added on purpose in order to result in misclassification [12][13] under a white box setting [6] or they can be added as a noise under black box setting [7] where the noise is not added on purpose but added because of the environment. For a machine learning model inside a Tesla 3 car model, a sticker on a stop sign can be defined as an black-box adversarial attack. In literature, those perturbations are considered black-box attacks [3][4]. So, when we are training our models to detect eye closedness, we employ defenses in order to mitigate those attacks. Current literature has been focusing on adversarial training [8][9] as a defense mechanism. We explore data augmentation methods and adversarial training in order to make our model more generalizable than what it would have been otherwise. One method that we use adversarial training [10][11]. We define eight aspects on a driving setting that can be considered as a adversarial attack.

Adversarial attacks to the fatigue detection model can be summarized here:

  1. Rotation: Image can be rotated randomly depending on the driver’s position and the camera angle

  2. Width Shift: Image width might depend on the camera angle. The model needs to mitigate this randomness

  3. Height Shift: Image height might depend on the camera angle. The model needs to mitigate this randomness

  4. Shear angle Shift: Drivers plane intersects with the plane in which camera is mounted on a car. This creates additional randomness and the model needs to mitigate this randomness

  5. Zoom: The camera can be close or far to the driver and the model needs to mitigate this randomness.

  6. Horizontal Flip: This doesn’t correlate to real life setting, but the idea is that the driver’s window will be always at its left side, which means the lightning conditions from the left side of the camera will be always poor compared to the right side. The model needs to detect fatigue regardless of the lightning hence the flip.

  7. Image Fill: Image can be scaled down or up. The model needs to detect fatigue regardless.

  8. Scaling: Image can be scaled down or up. The model needs to detect fatigue regardless.

We develop two models, eye model and the face model where the first one is trained on eye patch dataset and the second one is trained on facial image dataset. For each model, we train a baseline model and a adversarially trained model.

Ii Related Work

Recent years have been focusing on convolutional neural network based deep learning methods in order to detect eye closedness [19][20][21]. Before then, the focus was on identification of key points in face via edge detection [17], based on these properties can be applied in face recognition [14][15]. Among these key points, the center of eyes is the most important than others [17][18][19], because with the use of eye centers one can remove in-plane rotation of face and with calculating the distance between eye centers, we can approximately find the size of face in an image [20][21][22]. This is a helpful clue to find the location of other key points [23]. As a result, detection of eye centers facilitates detection of other key points [24][25][26]. Some of the facial point extraction algorithms in literature are: Template matching [8], Integral projection [10], Snakes [9], Deformable template [8], Hough transform [10], Elastic bunch graph matching [10], Region growing search [11], Active Shape Models (ASMs), and Active Appearance Models (AAMs) [11].

The recent years saw an shift on focus to convolutional neural network based deep learning methods. The focus on convolutional neural networks in order to detect the eye closedness has seen increase because of the improved accuracy as well as cheaper memory and faster CPU. In a convolutional neural network, each pixel is treated as a feature. A kernel is applied to an image that to perform a convolutional operation. Resulting matrix, then goes through max pooling with where each kernel-sized overlap produces one value, that is the maximum pixel value in that overlap. Several kernels can be applied and the depth of the model can be very large like ResNet which has about 172 hidden layers. The goal is to extract features from a top-down setting where each layer produces the input for the following layer. After the Convolutional layers have been applied, the output is flattened and then given to a dense layer. The number of dense layers can be also arbitrary large, where each layer produces the input for the following layer. Both our models employ a 4 layer convolutional layer neural network. For a better understanding please refer to the Figures

[3] and [11]

Iii Basic Approach

We discussed that our goal is to reduce car accidents related to drowsy driving with an alert system.

Neural network in the mobile application will alert the driver with sound when drivers are
i) closing their eyes frequently and long
ii) falling asleep on the wheel
iii) heads start to fall down
iv) drivers start to doze off
A robust model is needed to reduce the false positives. Model should not trigger when the driver might be
i) just blinking frequently
ii) leaning towards the radio
iii) checking mirrors/sideways

In order to achieve these goals, we build two convolutional neural network models, one trained on Eye-blink dataset, and one trained on CEW dataset. For easiness, we call the model trained on Eye-blink dataset, eye model because the Eye-blink dataset consists of only images of eye patches. For easiness, we call the model trained on the CEW dataset face model because the CEW dataset consists of facial images. We perform data augmentation and for both models.

Iv Data

We use ’Eye-Blink Dataset [3] data set to craft train our eye model. Eye Blink dataset consisting of 2100 closed and open eye images that are black and white. They are 24x24 pixels and only show eye patches. We use Closed Eyes In The Wild (CEW) [4] dataset to train our face model. CEW dataset contains 1192 subjects with both eyes closed and 1231 subjects with eyes open. Some challenges of this set include amateur photography, occlusions, problematic lighting, pose, and motion blur.

Fig. 1: Closed Eyes in the Wild Dataset(CEW) contains 1192 subjects with both eyes closed and 1231 subjects with eyes open. CEW dataset contains 1192 subjects with both eyes closed and 1231 subjects with eyes open. Some challenges of this set include amateur photography, occlusions, problematic lighting, pose, and motion blur. Two models have been trained on this model: Face baseline model and adversarially trained model.
Fig. 2: Eye Blink dataset consisting of 2100 closed and open 24x24 pixel grayscale eye patch images. Two models have been trained on this model: Eye baseline model and adversarially trained model.

V Evaluation

All four models have been trained and classification reports for the training and the testing datasets for both models can be seen in Table I and Table II.

Model Types Test Accuracy Test Precision Test Recall Test F1 Score Test Loss
Eye Baseline 0.9584 0.9577 0.9606 0.9575 0.1079
Eye Adversarial 0.9153 0.9184 0.9190 0.9186 0.2224
Face Baseline 0.9664 0.9682 0.9645 0.9646 0.0923
Face Adversarial 0.8001 0.7945 0.8347 0.8042 0.4393
TABLE I: All four models have been trained and classification reports for the TRAINING dataset can be seen below. Overall the eye model has performed better than the face model and it can be seen in the comparison of the train and test accuracies of the eye baseline model to the train and test accuracies of the face baseline model.
Model Types Train Accuracy Train Precision Train Recall Train F1 Score Train Loss
Eye Baseline 0.9476 0.9717 0.9097 0.9386 0.2358
Eye Adversarial 0.9126 0.8932 0.9499 0.9206 0.4725
Face Baseline 0.9271 0.9583 0.8863 0.9187 0.2561
Face Adversarial 0.7943 0.8197 0.7603 0.7831 0.4072
TABLE II: All four models have been tested and classification reports for the TESTING dataset can be seen below. Overall the eye model has performed better than the face model and it can be seen in the comparison of the train and test accuracies of the eye baseline model to the train and test accuracies of the face baseline model. Also, adversarial face model performed the worst during testing.
Layer type Output Shape Param #
Conv2D (98, 98,6) 60
Average Pooling (49,49,6) 0
Conv2D (47,47,16) 880
Average Pooling (23,23,16) 0
Flatten 8464 0
Dense 120 1015800
Dense 84 10164
Dense 1 85
TABLE III: Model Architecture for both models. We used the same architecture on the Eye-Blink and CWE dataset. The eye model and the face model both use binary cross entropy with Adam’s optimizer on iterative gradient descent.

V-a Eye Model Results Approach 1: Noisified Training Data

In our first approach, we added noise to the eye training dataset.

  1. rotation range=40,

  2. width shift range=0.2,

  3. height shift range=0.2,

  4. shear range=0.2,

  5. zoom range=0.2,

  6. horizontal flip=True,

  7. fill mode=’nearest’,

  8. rescale = 1./255

Fig. 3: Eye Model Noise Added Samples from the Training Dataset. Randomness emulate real life scenarios and makes it easier for the models to generalize better to those scenarios. the perturbation added to the dataset creates adversarial attacks and using those adversarial training dataset to train a model is called adversarial training.
Fig. 4:

Eye Model Accuracy for training and testing dataset vs Epoch Count when

noise is added. This model is referred as adversarially trained eye model.
Fig. 5: Eye Model Binary Cross Entropy Loss for training and testing datasets vs Epoch Count when noise is added.
Class Label Accuracy Precision Recall F1 Score Sample #
0 0.93 0.95 0.90 0.93 470
1 0.93 0.91 0.96 0.93 492
TABLE IV: Eye Model Accuracy, F1 Score, Recall and Precision when noise is added to the Eye-blink dataset. Class label 0 is for when the eyes are closed. Class label 1 is for when the eyes are open.
TruePredicted Class 0 Class 1
Class 0 424 46
Class 1 21 471
TABLE V: Confusion Matrix for the Eye Model when noise is added to the Eye-blink dataset. Class label 0 is for when the eyes are closed. Class label 1 is for when the eyes are open.

We trained for 40 epochs with 3108 images belonging to 2 classes, eye closed and open(same number on both classes). We then tested the model on 776 images.

  • Testing data for the adversarial eye model had higher accuracy than training data of the same adversarial eye model.

  • Potential cause for underfitting.

  • Training data had several arduous cases to learn due to noise

  • Test data contained easier cases to predict

  • Training loss is the average of the losses over each batch of training data.

Because the model gradients are changing over time, the loss over the first batches of an epoch is generally higher than over the last batches. On the other hand, the testing loss for an epoch is computed using the model as it is at the end of the epoch, resulting in a lower loss.

V-B Eye Model Results Approach 2: No noise addition

We trained for 40 epochs with 3108 images belonging to 2 classes, eye closed and open(same number on both classes). We then tested the model on 776 images.

  • Training data had higher accuracy than training data.(see red highlights)

  • This might be due to the fact that the model is overfitting.

  • Removing data augmentation in this run yielded 6% improvement in accuracy, precision, recall and the f1-score.

Fig. 6: Eye Model Accuracy for training and testing dataset vs Epoch Count when no noise is added. This model is referred as the baseline eye model.
Fig. 7: Eye Model Binary Cross Entropy Loss for training and testing datasets vs Epoch Count when no noise is added.
Class Label Accuracy Precision Recall F1 Score Sample #
0 0.95 0.93 0.97 0.95 470
1 0.95 0.97 0.93 0.95 492
TABLE VI: Eye Model Accuracy, F1 Score, Recall and Precision when no noise is added to the Eye-blink dataset. Class label 0 is for when the eyes are closed. Class label 1 is for when the eyes are open.
TruePredicted Class 0 Class 1
Class 0 455 15
Class 1 33 459
TABLE VII: Confusion Matrix for the Eye Model when no noise is added to the Eye-blink dataset. Class label 0 is for when the eyes are closed. Class label 1 is for when the eyes are open.

V-C Face Model

We developed a 4 layer convolutional neural network. We refer to this model as the "face model".

V-D Face Model Results Approach 1: Noisified Training Data

In our first approach, we added noise to the face training dataset.

  • rotation range=40,

  • width shift range=0.2,

  • height shift range=0.2,

  • shear range=0.2,

  • zoom range=0.2,

  • horizontal flip=True,

  • fill mode=’nearest’,

  • rescale = 1./255

Fig. 8: Face Model Noise Added Samples from the Training Dataset
Fig. 9: Face Model Accuracy for training and testing dataset vs Epoch Count when noise is added. This model is referred as the adversarially trained face model.
Fig. 10: Adversarially trained Face Model Binary Cross Entropy Loss for training and testing datasets vs Epoch Count when noise is added.
Class Label Accuracy Precision Recall F1 Score Sample #
0 0.93 0.92 0.93 0.93 239
1 0.93 0.93 0.92 0.93 246
TABLE VIII: Face Model Accuracy, F1 Score, Recall and Precision when noise is added to the CWE dataset. Class label 0 is for when the eyes are closed. Class label 1 is for when the eyes are open.
TruePredicted Class 0 Class 1
Class 0 223 16
Class 1 19 227
TABLE IX: Confusion Matrix for the Face Model when noise is added to the CWE dataset. Class label 0 is for when the eyes are closed. Class label 1 is for when the eyes are open.

We overall got poor results compared to baseline model. Especially, training accuracy for the adversarial face model is approximately 15% lower than training accuracy of the baseline face model. We trained the face model for 40 epochs to be consistent with the eye model over 1558 images belonging to 2 classes, eye closed or open. We then tested the model on 485 images.

  • Testing data had higher accuracy than training data.(see red highlights)

  • This might be due to the fact that the model is underfitting.

  • The training data had several arduous cases to learn due to noise

  • The test data contained easier cases to predict.

  • high accuracy can be attributed to the fact that not memorizing a complex training data is useful for real life models.

V-E Face Model Results Approach 2: No noise addition

We trained the model for 40 epochs to be consistent with the eye model over 1558 images belonging to 2 classes, eye closed or open. We then tested the model on 485 images.

  • Better results compared to noisy training data.

  • Training accuracy is higher than testing accuracy.

  • Training data had higher accuracy than training data.(see red highlights)

  • The baseline model observes overfitting.

  • Removing data augmentation in this run yielded 12% improvement in accuracy, precision, recall and the f1-score.

Fig. 11: Baseline Face Model Accuracy for training dataset vs Epoch Count when no noise is added. This model is referred as baseline face model because there is no adversarial attacks in the training dataset.
Fig. 12: Baseline Face Model Binary Cross Entropy Loss for testing datasets vs Epoch Count when no noise is added.
Class Label Accuracy Precision Recall F1 Score Sample #
0 0.93 0.92 0.93 0.93 239
1 0.93 0.93 0.92 0.93 246
TABLE X: Face Model Accuracy, F1 Score, Recall and Precision when no noise is added to the CWE dataset. Class label 0 is for when the eyes are closed. Class label 1 is for when the eyes are open.
TruePredicted Class 0 Class 1
Class 0 223 16
Class 1 19 227
TABLE XI: Confusion Matrix for the Face Model when no noise is added to the CWE dataset. Class label 0 is for when the eyes are closed. Class label 1 is for when the eyes are open.

Vi Conclusion

In this study, we explored two different approaches for driver fatigue detection. First, we trained a neural network on eye patch images to detect eyes closedness. Second, we trained a neural network on facial images to detect eyes closedness. This was a more relaxed version of the facial fatigue detection problem. The eye model performed better than the face model by achieving 94% testing accuracy. This testing accuracy was 2% higher compared to that of the face model. We also studied facial fatigue detection under an adversarial detection by applying data augmentation in order to increase robustness. Some of the perturbations that we added to the dataset for data augmentation were shear angle shift, zoom, rescaling. We considered these perturbations as black-box adversarial attacks and performed adversarial training to emulate real life driving settings. Adversarially trained models performed 4% and 16% worse on the training data and 3% and 13% worse on the testing data. This is due to the fact that we introduced arduous cases to the training images and made it harder to memorize the training data hence poor training accuracies. But they generalized well so we were able to see less performance decreases in the testing accuracies compared to the testing accuracies of the non-adversarial models. One important thing to notice is that we were able to increase recall for the eye closedness detection by about 5%. Recall, which quantifies the number of positive class predictions made out of all positive examples in the dataset suggests that if the task is small enough, introducing adversarial training can reduce the false negatives and increase the true positives, which is very crucial for drowsy driver detection because not being able to detect fatigue when it is most needed is very crucial to prevent car accidents related to drowsiness. We were also able to freeze the weights in our all models and convert them to tflite files that can be used in a mobile application. When we converted the models to tflite models[30], the model size was approximately  19 MB, which is a size that can be put into a mobile application. Since we used the same architecture for all four models, we achieved the same file size for all four models. We hope that these remarks will help future researchers.

Vii Future Work

In the future, we would like to explore ensemble learning and combine the two models that we had. We can produce a hybrid model where the eyes closedness is classified by votes coming from the eye model as well as the face model. We also would like to explore the relative head position in order to better detect the driver’s fatigue instead of just focusing on the eye closedness detection. We also think it would be interesting to train an eye detector by using the face dataset, then use the eye dataset eyes to train an eye closedness detector and compare that to just the eye detector. This comparison would highlight the differences between using only eye images and eye + face images for eye detection.

Viii Acknowledgments

We offer our most sincere appreciation to Dr. Matthew Burlick, who gave valuable feedback during several phases of our research.

Ix References

[1] National Safety Council - Road Safety, https://www.nsc.org/road-safety/safety-topics/fatigued-driving
[2] Wheaton AG, Chapman DP, Presley-Cantrell LR, Croft JB, Roehler DR. Drowsy driving – 19 states and the District of Columbia, 2009-2010., MMWR Morb Mortal Wkly Rep. 2013; 61:1033.
[3]

Nicholas Carlini and David Wagner. Defensive distillation is not robust to adversarial examples. arXiv preprint arXiv:1607.04311, 2016.


[4] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. arXiv preprint arXiv:1608.04644, 2016.
[5] Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. arXiv preprint arXiv:1705.07263, 2017.
[6] N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami. Distillation as a defense to adversarial perturbations against deep neural networks. IEEE Symposium on Security and Privacy, 2015
[7]

Moosavi-Dezfooli, Seyed-Mohsen et al. “Universal Adversarial Perturbations.” 2017 IEEE Conference on Computer Vision and Pattern Recognition,2017, 86-94.


[8] R. Brunelli, T. Poggio. “Face recognition: features vs. templates”, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 15, no. 10,1993, pp. 1042-1052
[9] M. Kass, A. Witkin, and D. Terzopoulos. “Snakes: Active contour models”, Int. Journal of Computer Vision, vol. 1, no. 4, 1987, pp. 321-331.
[10]

L. Yuille, D. S. Cohen, and P. W. Hallinan, "Feature Extraction from Faces using Deformable Templates", Int. Journal of Computer Vision, vol. 8, no. 2, 1992, pp. 99-111.


[11] K. M. Lam, and H. Yam, "Locating and Extracting the Eye in Human Face Images", IEEE Int. Conf. on Pattern Recognition, vol. 29, no. 5, 1996, pp. 771-779.
[12] Certified Adversarial Robustness via Randomized Smoothing, ICML, 2019 Jeremy M. CohenElan RosenfeldJ. Zico Kolter
[13] Efficient Adversarial Training with Transferable Adversarial Examples, ArXiv 2019, Haiz Hong, Zheng Ziqi, Zhang Junchengm, Gu Honglak, Lee Atul Prakash
[14] M. Pantic and L.J.M "Automatic analysis of facial expressions: the state of the art, IEEE Trans. Pattern Analysis and Machine Learning, vol. 22 no. 12, pp 1424-1445, Dec 1996.
[15] V. Bruce, “What the Human Face Tells the Human Mind: Some Challenges for the Robot-Human Interface,” Proc. Int’l Workshop Robot and Human Comm., pp. 44-51, 1992.Google ScholarCross Ref
[16] J. Buhmann J. Lange and C. von der Malsburg, “Distortion Invariant Object Recognition—Matching Hierarchically Labelled Graphs,” Proc. Int’l Joint Conf. Neural Networks, pp. 155-159, 1989.
[17] F.W. Campbell, “How Much of the Information Falling on the Retina Reaches the Visual Cortex and How Much is Stored in the Memory?” Seminar at the Pontificae Academiae Scientiarium Scripta Varia, 1983.
[18] L.S. Chen T.S. Huang T. Miyasato and R. Nakatsu, “Multimodal Human Emotion/Expression Recognition,” Proc. Int’l Conf. Automatic Face and Gesture Recognition, pp. 366-371, 1998.
[19] J.F. Cohn A.J. Zlochower J.J. Lien and T. Kanade, “Feature-Point Tracking by Optical Flow Discriminates Subtle Differences in Facial Expression,” Proc. Int’l Conf. Automatic Face and Gesture Recognition, pp. 396-401, 1998.
[20] T.F. Cootes C.J. Taylor D.H. Cooper and J. Graham, “Active Shape Models—Training and Application,” Computer Vision Image Understanding, vol. 61, no. 1, pp. 38-59, 1995.
[21] T.F. Cootes G.J. Edwards and C.J. Taylor, “Active Appearance Models,” Proc. European Conf. Computer Vision, vol. 2, pp. 484-498, 1998.
[22] G.W. Cottrell and J. Metcalfe, “EMPATH: Fface, Emotion, Gender Recognition Using Holons,” Advances in Neural Information Processing Systems 3, R.P. Lippman, ed., pp. 564-571, 1991.
[23] D. DeCarlo D. Metaxas and M. Stone, “An Anthropometric Face Model Using Variational Techniques,” Proc. SIGGRAPH, pp. 67-74, 1998.
[24] L.C. De Silva T. Miyasato and R. Nakatsu, “Facial Emotion Recognition Using Multimodal Information,” Proc. Information, Comm., and Signal Processing Conf., pp. 397-401, 1997.Google Scholar [25] G. Donato M.S. Bartlett J.C. Hager P. Ekman and T.J. Sejnowski, “Classifying Facial Actions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 10, pp. 974-989, Oct. 1999.
[26] M.S. Bartlett J.C. Hager P. Ekman and T.J. Sejnowski, “Measuring Facial Expressions by Computer Image Analysis,” Psychophysiology, vol. 36, pp. 253-263, 1999.
[27] G. Pan, L. Sun, Z. Wu, S. Lao, Eyeblink-based anti-spoofing in face recognition from a generic webcamera, in: Proceedings of IEEE International Conference on Computer Vision, 2007, pp. 1–8. [28] F.Song, X.Tan, X.Liu and S.Chen, Eyes Closeness Detection from Still Images with Multi-scale Histograms of Principal Oriented Gradients, Pattern Recognition, 2014.
[29] Mercedes Benz, Attention Assistant, https://www.mercedesbenzofhonolulu.com/how-does-mercedes-benz-attention-assist-work/
[30]

Freezing TensorFlow models’ weights to be used for mobile applications via The TensorFlow Lite Converter API,

https://www.tensorflow.org/lite/convert/cmdline