Publication detail

MediaPipe and Its Suitability for Sign Language Recognition

ŠNAJDER, J. KREJSA, J.

English title

MediaPipe and Its Suitability for Sign Language Recognition

Type

article in a collection out of WoS and Scopus

Language

en

Original abstract

The paper presents the framework MediaPipe as a tool to extract pose features for the task of word-level isolated sign language recognition. It tests the framework’s suitability on the state-of-the-art sign language dataset AUTSL. Extracted sequences of pose features are classified by the Long Short-Term Memory recurrent neural network constructed with the TensorFlow computational library. The paper describes the proposed method flow, preprocessing of the extracted features, and training. Obtained results are then validated on test datasets, and the impact of face landmarks is evaluated. The top-1 accuracy with face landmarks is 49.89 %, while 53.21 % without them.

English abstract

The paper presents the framework MediaPipe as a tool to extract pose features for the task of word-level isolated sign language recognition. It tests the framework’s suitability on the state-of-the-art sign language dataset AUTSL. Extracted sequences of pose features are classified by the Long Short-Term Memory recurrent neural network constructed with the TensorFlow computational library. The paper describes the proposed method flow, preprocessing of the extracted features, and training. Obtained results are then validated on test datasets, and the impact of face landmarks is evaluated. The top-1 accuracy with face landmarks is 49.89 %, while 53.21 % without them.

Keywords in English

Sign language recognition; MediaPipe; Long Short-Term Memory; neural network; classification

Released

10.05.2023

Publisher

Institute of Thermomechanics of the Czech Academy of Sciences

Location

Prague

ISBN

ISBN 978-80-87012-84

ISSN

1805-8256

Book

ENGINEERING MECHANICS 2023

Edition number

1

Pages from–to

251–254

Pages count

4

BIBTEX


@inproceedings{BUT184379,
  author="Jan {Šnajder} and Jiří {Krejsa},
  title="MediaPipe and Its Suitability for Sign Language Recognition",
  booktitle="ENGINEERING MECHANICS 2023",
  year="2023",
  month="May",
  pages="251--254",
  publisher="Institute of Thermomechanics of the Czech Academy of Sciences",
  address="Prague",
  isbn="ISBN 978-80-87012-84",
  issn="1805-8256"
}