BMCC Led Research on Virtual Guide Dog for Visually Impaired People Receives $500,000 NSF Grant

A research project—Training a Virtual Guide Dog for Visually Impaired People to Learn Safe Routes Using Crowdsourcing Multimodal Data— led by Borough of Manhattan Community (BMCC/CUNY) College Computer Information Systems (CIS) Professor Hao Tang, has received a $500,000, three-year grant from the National Science Foundation (NSF).

The grant is part of the NSF’s Computer and Information Science and Engineering Minority-Serving Institutions Research Expansion Program (CISE-MSI). BMCC is the only two-year institution to receive the CISE-MSI grant so far during this year’s cycle.

The research is a collaboration between BMCC and City College of New York (CCNY/CUNY) as well as the Lighthouse Guild, a nonprofit serving BVI.  Tang is the lead principal investigator (PI) and Zhigang Zhu, Herbert G. Kayser Professor of Computer Science, CCNY and William Seiple, Chief Research Officer, Lighthouse Guild are co-PI’s. Don Wei, chair of the BMCC CIS department is senior personnel on the project.

The grant will fund further research on mobile assistive apps and augmented reality research to assist people who are blind or visually impaired (BVI) so they can move around independently.

The World Health Organization (WHO) estimates that nearly 285 million people are visually impaired of which, 39 million are totally blind. Among the obstacles they face each day are difficulty accessing cultural activities, visiting new places and using public transportation.

Although mobile navigation apps with GPS and map services such as Google maps are currently available when BVI individuals travel outdoors independently, the existing map services do not offer enough information to ensure a safe and comfortable travel experience even in large cities.  Users of navigation apps often sweep canes on other pedestrians, are trapped on uneven pavement or bumping into billboards or other objects.

Tang and his team propose to collect crowdsourcing data and multimodal sidewalk data which is then fed to deep learning neural network models to detect the sidewalk conditions and understand BVI users walking behavior. Tang says the project will lead to an eDog system embedded with a semantic sidewalk map including geo-referenced travel challenges for BVI users.

“Our hypothesis is that multiple benefits will be provided to BVI users if the sidewalk and street crossing are surveyed by BVI users themselves, including the conditions and materials of pavement and their travel challenges, and the corresponding travel decisions and behaviors of BVI users are obtained,” said Tang.

“The project will significantly strengthen the research capacity of BMCC,” said Professor Tang.

He said the partnership with CCNY will provide BMCC students an opportunity to conduct research alongside students in more advanced degree programs—undergraduate, masters and doctoral— at CCNY.  Those interactions will further enhance BMCC student’s experience, help facilitate their transfer from community college to a senior college as well as widen their career choices.

BMCC is dedicated to promoting research and scholarly activity among its faculty and students. As part of its strategic plan, BMCC is expanding efforts in training and support of faculty and student researchers.