Scene Understanding with Raw Data
Project Description

Scene understanding in low-light or direct-sunlight environments is challenging because the captured images are dark, noisy, or overexposed. To address this problem, we can design a machine learning algorithm for scene understanding using a camera which provides raw high-dynamic-range (HDR) data. SONY IMX390 is a next-generation automotive grade camera that offers full-HD images 12-bit raw data at 30 fps. Furthermore, unlike traditional HDR techniques which capture multiple images with different exposure times, this camera has the built-in hardware to generate HDR images with the dynamic range of 120 dB. In low light conditions, it can capture high-quality color images at 0.1 lux. We can perform better scene understanding with raw HDR data than with 8-bit color images since raw HDR data contain richer information. We can train a deep neural network that takes the raw HDR data as input and outputs the desirable scene understanding. The tasks for scene understanding may include semantic segmentation, image restoration, object detection, optical flow estimation, and so on.

Supervisor
CHEN Qifeng
Quota
5
Course type
UROP1100
Applicant's Roles

The students are expected to propose new ideas, run experiments, and collect data for the project.

Applicant's Learning Objectives

The students can get involved in a research project that may be published in a top research conference.

Complexity of the project
Moderate