Capsule Deep Neural Network for Recognition of Historical Graffiti Handwriting

by   Nikita Gordienko, et al.

Automatic recognition of the historical letters (XI-XVIII centuries) carved on the stoned walls of St.Sophia cathedral in Kyiv (Ukraine) was demonstrated by means of capsule deep learning neural network. It was applied to the image dataset of the carved Glagolitic and Cyrillic letters (CGCL), which was assembled and pre-processed recently for recognition and prediction by machine learning methods ( CGCL dataset contains >4000 images for glyphs of 34 letters which are hardly recognized by experts even in contrast to notMNIST dataset with the better images of 10 letters taken from different fonts. Despite the much worse quality of CGCL dataset and extremely low number of samples (in comparison to notMNIST dataset) the capsule network model demonstrated much better results than the previously used convolutional neural network (CNN). The validation accuracy (and validation loss) was higher (lower) for capsule network model than for CNN without data augmentation even. The area under curve (AUC) values for receiver operating characteristic (ROC) were also higher for the capsule network model than for CNN model: 0.88-0.93 (capsule network) and 0.50 (CNN) without data augmentation, 0.91-0.95 (capsule network) and 0.51 (CNN) with lossless data augmentation, and similar results of 0.91-0.93 (capsule network) and 0.9 (CNN) in the regime of lossless data augmentation only. The confusion matrixes were much better for capsule network than for CNN model and gave the much lower type I (false positive) and type II (false negative) values in all three regimes of data augmentation. These results supports the previous claims that capsule-like networks allow to reduce error rates not only on MNIST digit dataset, but on the other notMNIST letter dataset and the more complex CGCL handwriting graffiti letter dataset also.


page 1

page 2

page 5


Open Source Dataset and Machine Learning Techniques for Automatic Recognition of Historical Graffiti

Machine learning techniques are presented for automatic recognition of t...

Neural network with data augmentation in multi-objective prediction of multi-stage pump

A multi-objective prediction method of multi-stage pump method based on ...

HitNet: a neural network with capsules embedded in a Hit-or-Miss layer, extended with hybrid data augmentation and ghost capsules

Neural networks designed for the task of classification have become a co...

Continuous sign language recognition from wearable IMUs using deep capsule networks and game theory

Sign Language is used by the deaf community all over world. The work pre...

Pushing the Limits of Capsule Networks

Convolutional neural networks use pooling and other downscaling operatio...

Capsule Network Performance on Complex Data

In recent years, convolutional neural networks (CNN) have played an impo...

Segmentation of Bleeding Regions in Wireless Capsule Endoscopy for Detection of Informative Frames

Wireless capsule endoscopy (WCE) is an effective mean for diagnosis of g...