Book2Braille

Transcription of Scientific literature to braille format using OAK-D

This project was done as part of Opencv AI Competition 2021 (https://opencv.org/opencv-ai-competition-2021/)

What is OAK-D?

OAK—D is a spatial AI powerhouse, capable of simultaneously running advanced neural networks while providing depth from two stereo cameras and color information from a single 4K camera in the center.

To buy visit page - https://store.opencv.ai/products/oak-d

What is Book2Braille?

Book2Braille is a solution that is built to aid people with visual impairments in the field of science. The aim of the solution is to convert equations and texts from scientific books into Braille thereby making it accessible for visually impaired people.

Follow the below link for in-depth details of the implementation and our motivation behind the project

👉 Detailed report

Setup

---

How it works?

Step1: Image capturing and preprocessing

Capture image ➡️ Filter backgroud ➡️ Split pages ➡️ Uncurve the page ➡️ Adaptive threshold

Step2: Split image into three halves

Step3: Feed each image to custom trained yoloV3 model to locate mathematical expressions and text blocks

Step4: Extract content from text blocks and mathematical expressions using tesseract-OCR and MathPix respectively

Convert text to braille as per Universal English Braille standard and latex to braille using Latex2Nemeth software

To run the program

---

cd Book2Braille
python main.py

Dependencies (python packages):

---

• opencv
• depthai
• pytesseract
• scipy
• skimage
• speech_recognition
• pyttsx3

Tools used:

---

• Custom trained yoloV3 model
• Tesseract
• Mathpix
• Latex2Nemeth