Rutgers Researchers Create Artificial Intelligence-Aided Railroad Trespassing Detection Tool

The system analyzes video footage to detect illegal pedestrian activity and deliver data to help reduce preventable train strikes

A pair of Rutgers engineers have developed a tool aided by artificial intelligence to detect trespassing on railroad crossings and curb fatalities that have been increasing over the past decade.

Asim Zaman, a Rutgers project engineer, and Xiang Liu, an associate professor in transportation engineering at the Rutgers School of Engineering, created an AI-aided framework that automatically detects railroad trespassing events, differentiates types of violators and generates video clips of infractions. The system uses an object detection algorithm to process video data into a single dataset.

“With this information we can answer numerous questions, like what time of day do people trespass the most, and do people go around the gates when they are coming down or going up?” said Zaman.

Every year, hundreds of people in the U.S. are killed in trespassing accidents at the country’s 210,000 rail crossings, according to the Federal Railroad Administration. Despite concerted efforts to reduce fatalities, deaths by train strike continue to rise. In 2008, the FRA estimated about 500 people were killed annually trespassing on railroad rights-of-way. Ten years later, the number inclusive of suicides had climbed to 855, the FRA reported.

In their research, Zaman and Liu define trespassers as unauthorized people or vehicles in an area of railroad or transit property not intended for public use, or those who enter a signalized grade crossing after it has been activated.

Until now, most research into railroad trespassing was derived from casualty information. But the research overlooked near-misses – occasions Zaman and Liu said can provide valuable insights into trespassing behaviors, which in turn can help with the design of more effective control measures.

To test their theory, the researchers accessed video footage captured at one crossing in urban New Jersey. The study location had cameras in place installed following the 2015 Fixing America’s Surface Transportation Act (FAST). But most video systems at crossings today are either not reviewed or reviewed manually, which is labor-intensive and expensive.

Zaman and Liu trained their AI and deep-learning tool to analyze 1,632 hours of archival video footage from the study site. What they discovered was during 68 days of monitoring, 3,004 instances of trespassing occurred – an average of 44 a day. The researchers also found that nearly 70 percent of trespassers were men, roughly a third trespassed before the train passed and most violations occurred on Saturdays around 5 p.m. The results are published in the journal Accident Analysis & Prevention.

Zaman said granular data like this could be used by local authorities to position police officers near crossings during periods of peak violations or to inform railway owners and decision makers of more effective crossing solutions – such as grade crossing elimination systems or advanced gates and signals.

“Everyone loves data, and that’s what we are providing,” said Zaman.

Added Liu: “We want to give the railroad industry and decision makers tools to harness the untapped potential of video surveillance infrastructure through the risk analysis of their data feeds in specific locations.”

In addition to the New Jersey location, the researchers are conducting studies in Virginia and North Carolina and were recently awarded a $583,000 grant from the U.S. Department of Transportation to expand to other states including Connecticut, Massachusetts, and Louisiana. Their work is funded by grants from the FRA and the Federal Transit Administration, and research collaborators include rail companies, transit agencies and state transportation departments.