Master's student capstone spotlight: A Remote Sensing Framework for Rail Incident Situational Awareness Drones

Data science and computational science and engineering master’s students at the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) take “AC297R: Computational Science and Engineering Capstone Project.” Taught by Weiwei Pan, Assistant Director for Graduate Studies in Data Science, the course groups students together for semester-long research projects in which they work with “client” organizations to tackle real-world challenges.

A Remote Sensing Framework for Rail Incident Situational Awareness Drones

Vlad Ivanchuk (A.B./S.M. ‘24), Mitch Miller (S.M. ‘25), Samantha Nahari (S.M. ‘24), Rama Edlabadkar (S.M. ‘25)

Client: Unspace

What real-world challenge does this project address?

Rail incidents can have disastrous outcomes, devastate local communities, and can be difficult to contain. Our project addresses the severe impacts of rail incidents, illustrated by the catastrophic 2023 train derailment in East Palestine, Ohio. This event resulted in the release of hazardous materials, leading to extensive environmental and public health crises. The derailment caused the evacuation of thousands, lasting environmental damage, and ongoing health concerns in the community. The incident underscored the critical need for rapid and effective incident assessment and management, especially in resource-limited areas where the local infrastructure and emergency response capabilities may be overwhelmed.

How does this research attempt to solve that real-world challenge?

Our capstone project focuses on developing a drone-based computer vision system that can be rapidly deployed to assess such disaster sites. We developed a prototype for a lightweight, real-time object detection system that can enhance situational awareness immediately following rail incidents, providing real-time data crucial for coordinating emergency response efforts.

How did you apply the skills you learned at SEAS to your project?

We practiced framing a real-world problem in such a way that it could be addressed computationally with computer vision algorithms, evaluated multiple approaches to designing a computer vision system, and chose the one that prioritizes real-time inference to provide information to emergency responders quickly while meeting the limited computational resources available on the drone.

What part of the project proved the most challenging?

We did not have a dataset of railway accident images available to train our models on, since the project was in the early stages and our client company was still collecting and labeling data. This prompted us to think creatively and find suitable replacement data that simulates the challenges we expect to see in real aerial drone railway images. We maintained a positive outlook and still achieved progress on the project despite challenges and restrictions outside of our control.