Penn drones navigate on their own, could save people from peril

The flying robots called drones were used in a dramatic light show for the Olympics opening ceremonies in South Korea, executing elaborate routines that humans had programmed in advance. Other drones are piloted by remote control, ranging from low-cost toy versions to the sophisticated devices used in the military.

In a University of Pennsylvania lab, engineers now have produced something else entirely: "swarms" of drones that can navigate on their own.

Picture a crew of firefighters outside a burning building, unable to tell whether  any floors have collapsed. Or imagine a nuclear accident that is too dangerous for humans to examine up close. In the not-too-distant future, a group of these drones could handle the job instead, the Penn engineers say.

Such devices can work as a group to canvass a wide area, capturing images and other data that would help emergency responders plot the next step — from a safe distance — said team leader Vijay Kumar, who is also dean of Penn's engineering school.

"The robots basically talk to each other," he said. "They each know where they're going. They can use high-level algorithms to distribute themselves in complex ways to solve tasks."

The devices perceive their surroundings by means of onboard cameras and "inertial measurement units" — the same technology used in smartphones to tell when the screen is tilted this way or that. The computer brain mounted on each drone also came from a smartphone — made by Qualcomm, which funded the research along with the Pentagon and the National Science Foundation.

"It's a smartphone without the case," said Penn research scientist Giuseppe Loianno.

Loianno said he could not disclose the exact funding total for the project, but it is safe to say it is in the millions. Penn's robotics lab has received other grants for related projects with the drones, including a $27 million outlay from the U.S. Army Research Laboratory.

Loianno and colleague Aaron Weinstein, who earned an undergraduate engineering degree from Penn in 2017 and remains part of the team, showed off four of the flying robots recently at the university's Pennovation Center.

Using a laptop computer, Weinstein issued a series of general commands to the drones — form a straight line, a diamond, a diagonal — then let the electronic hive mind figure out the rest.

The four robots rose as one, their tiny propellers whirling so fast they produced a musical hum, a note close to an F-sharp.

Collectively, they determined which among them would occupy a given spot in each formation, using an algorithm to find the most efficient route from where the devices had been hovering immediately before.

The research, scheduled to be presented at a conference later this year, is drawing notice. One fan is Larry Matthies, a senior research scientist at NASA's Jet Propulsion Laboratory at the California Institute of Technology.

"Vijay's group has been one of the world leaders in this area," said Matthies, a computer vision expert who worked on the Mars Exploration Rover and Pathfinder missions.

The drones were made by San Diego-based Qualcomm and customized by the Penn team. A big selling point of the devices is their pinpoint precision.

The Intel drones used for the Olympics ceremony relied on GPS signals to fix their position — a technology that is good to within a few yards in any direction. But the Penn drones can tell where they are within an inch or so, the engineers said.  Weinstein, Loianno, and colleague Adam Cho had no hesitation in posing for a photo as the devices hovered in formation about their heads.

And unlike drones that rely on satellite signals, the Penn drones can navigate indoors.

The key is the onboard cameras combined with some slick software. Given a known starting point, a drone maintains a running fix on its position based on how the camera's view of the ground below changes from one split-second to the next.

It works even when the camera's frame of reference is the chaotic design on the Pennovation Center's floor tiles — a jumble of beige squiggles against a dark-gray background, with no apparent pattern.

The building, tucked in a bend of the Schuylkill in a Philadelphia neighborhood called Forgotten Bottom, straddles the line between research and commerce, hosting both academic labs and start-up ventures. Qualcomm, the wireless company that helped fund Kumar's drone research, also has projects underway there.

Some features of the Penn drones still are in development, such as how they would store images and create a map of an unfamiliar environment. In the event of a nuclear accident, the drones also would need to cope with radiation, which can interfere with wireless communication.

In addition to navigating on their own, the drones are interchangeable, able to adapt and take on new responsibilities in case any members break down during a mission.

In that respect, Kumar said, their performance was like that of the Super Bowl champion Eagles, who successfully thrust a series of backup players into starring roles.

"We're looking at something that is inherently resilient," he said of the drones earlier this month, still hoarse from celebrating on Broad Street the night of the big win. "That term also is used to describe the Eagles."

After all, the football team was lauded this season for its "swarming" defense.

Yet unlike the Eagles, the drones could be produced for about $1,000 apiece, Kumar said. So far, the Penn team has demonstrated swarms with up to 12 drones, and Loianno said there is no reason they cannot do 100.