Publication detail
MediaPipe and Its Suitability for Sign Language Recognition
ŠNAJDER, J. KREJSA, J.
English title
MediaPipe and Its Suitability for Sign Language Recognition
Type
article in a collection out of WoS and Scopus
Language
en
Original abstract
The paper presents the framework MediaPipe as a tool to extract pose features for the task of word-level isolated sign language recognition. It tests the framework’s suitability on the state-of-the-art sign language dataset AUTSL. Extracted sequences of pose features are classified by the Long Short-Term Memory recurrent neural network constructed with the TensorFlow computational library. The paper describes the proposed method flow, preprocessing of the extracted features, and training. Obtained results are then validated on test datasets, and the impact of face landmarks is evaluated. The top-1 accuracy with face landmarks is 49.89 %, while 53.21 % without them.
English abstract
The paper presents the framework MediaPipe as a tool to extract pose features for the task of word-level isolated sign language recognition. It tests the framework’s suitability on the state-of-the-art sign language dataset AUTSL. Extracted sequences of pose features are classified by the Long Short-Term Memory recurrent neural network constructed with the TensorFlow computational library. The paper describes the proposed method flow, preprocessing of the extracted features, and training. Obtained results are then validated on test datasets, and the impact of face landmarks is evaluated. The top-1 accuracy with face landmarks is 49.89 %, while 53.21 % without them.
Keywords in English
Sign language recognition; MediaPipe; Long Short-Term Memory; neural network; classification
Released
10.05.2023
Publisher
Institute of Thermomechanics of the Czech Academy of Sciences
Location
Prague
ISBN
ISBN 978-80-87012-84
ISSN
1805-8256
Book
ENGINEERING MECHANICS 2023
Edition number
1
Pages from–to
251–254
Pages count
4
BIBTEX
@inproceedings{BUT184379,
author="Jan {Šnajder} and Jiří {Krejsa},
title="MediaPipe and Its Suitability for Sign Language Recognition",
booktitle="ENGINEERING MECHANICS 2023",
year="2023",
month="May",
pages="251--254",
publisher="Institute of Thermomechanics of the Czech Academy of Sciences",
address="Prague",
isbn="ISBN 978-80-87012-84",
issn="1805-8256"
}