A plug-and-play MATLAB based tool to use NES games for multimodal biofeedback
Explore the docs »
- About the Project
- Getting started
- Usage
- Documentation
- Roadmap
- Contributing
- License
- Contact
- Acknowledgment
- Citation
BioNES (Biofeedback Nintendo Entertainment System) is an open-source plug-and-play MATLAB-based tool to use NES games for multimodal biofeedback. It can be used to deliver the HRV biofeedback via any game designed for the NES system. It can receive real-time heartbeat interval (RR) values from Arduino + ear-clip PPG sensor (or any sensor capable to send real-time heart rate pulses to Arduino). After the acquisition, it computes real-time heart rate and heart rate variability (HRV), biofeedback parameters and then sends feedback to the FCEUX emulator which is used to play the NES game. Besides in-game biofeedback, it also offers real-time data visualization in time-series plots and local saving of data for offline analysis. This tool can benefit the researchers (especially from biofeedback or physiological measurement domain) or hobbyist, who want to quickly deploy a biofeedback system, want to explore the NES games for biofeedback, or just want to record the physiological signals.
- GUI is built using the
Guide
tool in MATLAB-v2017b. - Arduino Mega board for hardware.
- Grove ear-clip sensor for photoplethysmograph (PPG) data acquisition.
- FCEUX emulator to play NES game.
- Super Mario Bros. game for feedback.
- Plug and play biofeedback system for SMB game (using other NES games may need minor changes).
- Data acquisition sampling rate of 10 Hz to acquire RR intervals from Arduino.
- Feedback is displayed at a fixed refresh rate at 60 Hz.
- Real-time data visualization (in time-series plot).
- Local saving of data in
.mat
format for offline analysis. - Screenshot of current GUI for reference purpose.
- 3 independent modes of working.
- Data acquisition only
- Gameplay without biofeedback
- Gameplay with biofeedback
Biofeedback therapy has potential benefits for managing stress. However, traditional biofeedback is an expensive and monotonous process. To increase the engagement in biofeedback sessions, video games can be used. But the problem is the limited availability of affordable video-game-based biofeedback solutions. BioNES is developed keeping in mind the cost-effectiveness and ease of use to deliver biofeedback while simultaneously providing reliability as a research tool and flexibility to allow researchers/developers to modify as per their needs. The efficacy of BioNES is validated using observational study and the paper is currently under review in a reputed journal.
MATLAB-ver2017b
was used to develop the BioNES. It was also tested withver2015b
andver2018b
for backward and forward compatibility respectively.
- Arduino hardware board.
- Grove ear-clip sensor.
- FCEUX.
- NES game ROM file. This version of BioNES uses the Super Mario Bros. game without any modification, for other NES games you have to make minor changes to the
BioNES.lua
file.
- Download the latest stable release and extract contents into your MATLAB working directory. Alternatively, you can also download the latest code from the repository.
- Open
BioNES.m
in MATLAB. - Connect Arduino board to PC with USB cable. Upload
BioNES.ino
sketch to Arduino board. - Connect the ear-clip sensor to the Arduino board.
- Run
BioNES.m
and wait for the GUI to appear. - In the GUI select the
COM
port for the Arduino and press theConnect
button. - After a successful connection, appropriate settings can be selected.
- Select the biofeedback checkbook (if biofeedback gameplay is needed) and press the
Connect
button in theGame
panel. TheFCEUX
window will open with the pre-configured settings. - Start the acquisition from the
Start/Stop
button in theControl
panel. - The acquisition runs till a set time or can be stopped manually.
- After successful completion, the information panel shows various statistics related to the acquisition.
- Finally, the data can be saved for offline processing.
Use this space to show useful examples of how a project can be used. Additional screenshots, code examples, and demos work well in this space. You may also link to more resources.
For more examples, please refer to the Documentation.
See usage section in documentation.
The documentation is available at https://kulbhushanchand.github.io/BioNES/
See the open issues for a list of proposed features (and known issues).
Any contributions you make are greatly appreciated. You can contribute to this project in the following ways :
- Add new functionality
- Review code
- Raise issues about bugs/features/doubts
- Proof-read the documentation
- Cite if used in a publication
- Star on GitHub
- Share with others
Please note that this project is released with a Contributor Code of Conduct. By contributing to this project you agree to abide by its terms.
This project is distributed under the GPLv3
License. See LICENSE for more information.
If you are using this software, please cite the following publications
Chand, K., & Khosla, A. (2022a). BioNES: A plug-and-play MATLAB-based tool to use NES games for multimodal biofeedback. SoftwareX, 19, 101184. https://doi.org/10.1016/j.softx.2022.101184
Chand, K., & Khosla, A. (2022b). Efficacy of Using Retro Games in Multimodal Biofeedback Systems for Mental Relaxation. International Journal of Gaming and Computer-Mediated Simulations (IJGCMS), 14(1), 1–23. https://doi.org/10.4018/IJGCMS.295874
OSF | Efficacy of a multimodal biofeedback system for mental relaxation using NES games. (n.d.). Retrieved September 15, 2022, from https://osf.io/q5ez3/
Chand, K. (2021). Kulbhushanchand/BioNES [MATLAB]. https://github.com/kulbhushanchand/BioNES (Original work published 2021)
I would like to thank my PhD supervisor Prof. Arun Khosla at Dr B R Ambedkar National Institute of Technology for his guidance and kind support. I also want to acknowledge the open-source tools used in some parts of this project.
- FCEUX is used as an NES emulator.
- drawio-desktop is used to create the logo and diagrams.