In this project, students will develop and train a deep learning program that can detect whether the user’s eyes are open or closed and other facial expressions. This platform will be integrated into a desktop AI therapist called Nora, to create a special therapy session focused on relaxing and reducing user heart rate. Nora has been previously developed in our lab, which uses machine learning to predict user stress level and can guide people through a relaxation session. This project also includes the implementation of an application on a smartwatch, that uses heart rate detection sensor to communicate with Nora. This UROP project is meant to augment Nora with additional sensory input and guidance.
- Train a deep learning algorithm find out if eyes are open or closed in a frame.
- Train other deep learning algorithms for facial expressions.
- Choose suitable smartwatch with heart rate sensor. (e.g Android 2.0).
- Option A: Integrate Nora into smartwatch. Option B: integrate sensor with desktop version.
- Design suitable watch interface (front end).
- Integrate heart rate sensor.
- Create new session with heart rate detection integrated.
- Junior research assistant
- Mobile application developer
- some preliminary knowledge of machine learning/deep learning
- programming experience
Applicant's Learning Objectives:
- learn about intelligent dialogue systems
- learn about smartwatch data platform
- learn how to train deep learning alogrithms