TFS Recognition: Investigating MPH]Thai Finger Spelling Recognition: Investigating MediaPipe Hands Potentials
share
Thai Finger Spelling (TFS) sign recognition could benefit a community of hearing-difficulty people in bridging to a major hearing population. With a relatively large number of alphabets, TFS employs multiple signing schemes. Two schemes of more common signing – static and dynamic single-hand signing, widely used in other sign languages – have been addressed in several previous works. To complete the TFS sign recognition, the remaining two of quite distinct signing schemes – static and dynamic point-on-hand signing – need to be sufficiently addressed. With the advent of many off-the-shelf hand skeleton prediction models and that training a model to recognize a sign language from scratch is expensive, we explore an approach building upon recently launched MediaPipe Hands (MPH). MPH is a high-precision well-trained model for hand-keypoint detection. We have investigated MPH on three TFS schemes: static-single-hand (S1), simplified dynamic-single-hand (S2) and static-point-on-hand (P1) schemes. Our results show that MPH can satisfactorily address single-hand schemes with accuracy of 84.57 However, our finding reveals a shortcoming of MPH in addressing a point-on-hand scheme, whose accuracy is 23.66 obtained from conventional classification trained from scratch. This shortcoming has been investigated and attributed to self occlusion and handedness.
READ FULL TEXT