Sign For Everyjuan: An Interactive Android Application That Teaches Filipino Sign Language Using Hand Gesture Recognition

DOI

10.2991/978-2-38476-273-6_55How to use a DOI?

Keywords

Android; Sign Language; Hand Gesture; Recognition; Interactive

Abstract

Sign for EveryJUAN is an Android application for Filipinos who want to learn alphabet, numbers, WH questions, greetings, phrases, days, family members, adjectives, and verbs in Filipino Sign Language. The application will display images and play videos of signs in Filipino Sign Language which includes facial expression and hand gestures. Users can use their front facing camera to perform and practice the learned sign and the application will detect if the user performed the correct sign. MediaPipe is used to track and extract key point coordinates from the user’s hands and face. The extracted key point coordinates are used as an input for the Long Short-Term Memory neural network to identify what sign was performed. The LSTM network achieved accuracy ratings of above 90%. The application received an average rating of 4.53 from end users, which is considered to be highly acceptable, and an average rating of 4.18 from IT professionals, which is considered to be very acceptable. Keywords: Filipino Sign Language, Long Short-Term Memory Network, MediaPipe 1.0 Introduction Communication helps humans to interact and understand one another. People may not be able to communicate verbally in another culture’s language, but they can utilize gestures, visual cues, body language, and other nonverbal communication strategies (Naar, 2021). Sign language is a mode of communication that uses hand motions, body movements, and facial expressions. It is used by people around the globe who face difficulty with hearing and speaking to express and communicate. But there.

Copyright

© 2024 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Londren U. Velasco
AU  - Raymond S. Macatangga
AU  - Copernicus P. Pepito
AU  - Marissa G. Chua
AU  - Emmanuel G. Galupo Jr
AU  - Mary Joyce R. Del Mundo
AU  - Ericson L. Dilla
PY  - 2024
DA  - 2024/08/02
TI  - Sign For Everyjuan: An Interactive Android Application That Teaches Filipino Sign Language Using Hand Gesture Recognition
BT  - Proceedings of 5th Borobudur International Symposium on Humanities and Social Science 2023
PB  - Atlantis Press
SP  - 504
EP  - 513
SN  - 2352-5398
UR  - https://doi.org/10.2991/978-2-38476-273-6_55
DO  - 10.2991/978-2-38476-273-6_55
ID  - Velasco2024
ER  -