Skip to content
@TeamAllways-AllEars

TeamAllways-AllEars

Google Solution Challenge 2024

👂 All Ears: A Voice Visualizing WearOS App for Kids with Hearing Impairments in Inclusive Educational Environments

All Ears

🪧 Project Overview

When you lose your vision, you lose contact with things.
When you lose your hearing, you lose contact with people.
Helen Keler

Project Overview of All Ears

In South Korea, changes to the health insurance criteria in 2009 expanded cochlear implant surgery and enabled more kids with hearing impairments to study in inclusive education environments.[1] However, despite many receiving inclusive education, there aren't enough services for kids with hearing impairments in inclusive education classrooms in South Korea. These kids mostly communicate with kids without disabilities through speech.

This environment causes a lot of discomfort for children with hearing impairments. According to Ms. A, a person who has a hearing impairment, one of the difficulties in regular school was not being able to check her own voice's volume and content, which made her continuously ask friends to check them. Therefore, we need a service for kids with hearing impairments who need to engage in everyday communication in inclusive educational settings.

Team Allways build "All Ears", an WearOS app that provides real-time Voice-Noise Ratio Visualization, Live Captioning, and Past Captioning Rewatch for kids with hearing impairment. We aim to build an additional ear through visualization of speech and help kids with disabilities build camaraderie with their friends. With "All Ears" we can achieve 3 UN SDGs goals—the quality of education, promote peace, and reduce inequalities.

⚙️ Architecture & Tech

Wear OS Tensorflow Lite Spring Boot GCP STT GCP VM GCP Cloud SQL
Architecture of All Ears

To achieve Voice Ratio Visualization, we utilized TensorFlow Lite's AudioClassifier model, an on-device AI for edge devices. We also implemented Live Captioning using GCP's STT API. For Past Captioning Rewatch, we created a Spring Boot (Java 17) backend server that uses GCP Cloud SQL (MySQL 8.0 engine) and hosted it on a GCP VM instance.

These features are visualized through an app built on WearOS (Kotlin 1.7 & Java 8), making it possible for users to access it in everyday situations at school.

🪧 Guidance

This project has two repositories: All-Ears_Android for WearOS client application and All-Ears_Server for SpringBoot backend server. The guidance on how to run the code is in each repository's GitHub README.md file. If you want to run the code, please follow the instructions of each repositories.

📽 Demo Video Link

AllEars Demo Video

✅ How to use

All Ears how to use

⚙️ Release Note

2023-02-26

v1.0.0 release

  • Added Voice-Noise Ratio Feature(Tenserflow lite)
  • Added Speech-to-Text Featrue(GCP STT API)
  • Added Past Captioning Rewatch Feature(GCP Cloud SQL & VM instance)

👩‍💻 Contributors

Android Android Backend Design
노원희/Wonhui Roh 정예지/Yeji Jeong 전혜승/Haeseung Jeon 김수연/Suyeon Kim

📄 Related study

[1]Cho, Soo Jin, Mi Sook Lee, and Yu Ri Lee. "Analysis of the Current Status and Characteristics of the Hearing Impaired Using Data from the Actual Survey for the Disabled." Audiology and Speech Research 18.1 (2022): 9-19.

📧 Contact us

allways

Pinned Loading

  1. All-Ears_Android All-Ears_Android Public

    Android Client Codes for 'All Ears'

    Kotlin

  2. All-Ears_Server All-Ears_Server Public

    Springboot Backend Codes for 'All Ears'

    Java

Repositories

Showing 3 of 3 repositories
  • .github Public
    TeamAllways-AllEars/.github’s past year of commit activity
    0 0 0 0 Updated Feb 26, 2024
  • All-Ears_Android Public

    Android Client Codes for 'All Ears'

    TeamAllways-AllEars/All-Ears_Android’s past year of commit activity
    Kotlin 0 0 0 0 Updated Feb 26, 2024
  • All-Ears_Server Public

    Springboot Backend Codes for 'All Ears'

    TeamAllways-AllEars/All-Ears_Server’s past year of commit activity
    Java 0 0 0 0 Updated Feb 25, 2024

Top languages

Loading…

Most used topics

Loading…