A CAMERA THAT CAN SEE AROUND CORNERS
01.07.20
WHAT'S IT ABOUT?
Overview from TED
To work safely, self-driving cars must avoid obstacles -- including those just out of sight. And for this to happen, we need technology that sees better than humans can, says electrical engineer David Lindell.
Buckle up for a quick, groundbreaking tech demo as Lindell explains the significant and versatile potential of a high-speed camera that can detect objects hidden around corners.
"These waves of light illuminating the wall are like fireworks that last for just trillionths of a second."
David Lindell
Electrical Engineer
David Lindell develops next-generation algorithms and systems for imaging the world in 3D.
MY TAKE...
Fully self-driving cars are intrinsically futuristic. For most, our only experience of them is from sci-fi movies like Minority Report and Total Recall. And whilst we’re familiar with the ongoing efforts of Tesla and Google’s sister company Waymo (amongst others) to bring automated vehicles to market, the prospect of being chauffeured around remains a distant one.
Perhaps films are to blame for our unrealistic expectations. After all, if Back to the Future Part II was to be believed, we’d have been tootling around in flying cars FIVE years ago. Not to mention the hoverboards, self-tying shoelaces and robot dog walkers (what a film!). But on watching David Lindell’s short TED Talk about cameras that can see around corners, it’s clear to see why progress is taking a little longer than anticipated.
Lindell opens his talk with a bullish statement - that self-driving cars will be safer and more reliable than humans in the future - before caveating:
"But for this to happen, we need technologies that allow cars to respond faster than humans, we need algorithms that can drive better than humans and we need cameras that can see more than humans can see."
And the latter is what he intends to deliver…
"For the past few years as a PhD student in the Stanford Computational Imaging Lab, I've been working on a camera that can do just this - a camera that can image objects hidden around corners or blocked from direct line of sight."
Now, I can’t be the only one who pictures the “Car Periscope” episode from Series 8 of Curb Your Enthusiasm (a concept foreshadowed in Seinfeld – Larry David’s “other” show - some 15 years earlier). The scene in which Larry, Jeff and Susie test out the contraption for the first time depicts the benefits of “seeing more than humans can see” – in their case, avoiding traffic by spotting a garbage truck ahead. But the cameras that Lindell is referring to are *slightly* more advanced than that...
INNOVATION BREEDS INNOVATION
One of my all-time favourite quotes belongs to Isaac Newton: “If I have seen further it is by standing on the shoulders of giants.”
In the fields of science and technology, it is abundantly clear that innovation breeds more innovation. Lindell himself states that the technology he is showcasing during his TED Talk would have been impossible “just two years ago”.
As such, it’s still early days and there are substantial challenges for Lindell and his team to overcome: how to utilise low-power lasers that are eye-safe, and how to miniaturise what is currently a large and bulky prototype system into a useful tool for driverless cars, biomedical imaging or, perhaps, home security.
Nevertheless, the groundwork has been laid for an immensely exciting innovation that could change driving as we know it.
The key is speed.
The prototype system presented by Lindell essentially functions as a high-speed camera:
"Not one that operates at 1,000 frames per second, or even a million frames per second, but a trillion frames per second. So fast that it can actually capture the movement of light itself."
Now that sounds super speedy, but it’s hard to visualise. Thankfully Lindell explains it in layman’s terms, with a little help from DC Comic’s the Flash:
"Let's compare it to the speed of a fast-running comic book superhero who can move at up to three times the speed of sound. It takes a pulse of light about 3.3 billionths of a second, or 3.3 nanoseconds, to travel the distance of a metre.
"Well, in that same time, our superhero has moved less than the width of a human hair. That's pretty fast. But actually, we need to image much faster if we want to capture light moving at sub-centimetre scales. So, our camera system can capture photons at time frames of just 50 trillionths of a second, or 50 picoseconds."
Channelling Phoebe from Friends – “That’s math I can’t even do!” No wonder we’re five years late to the flying car party when engineers are working at this level of granularity!
Lindell continues:
"We take this ultra-high-speed camera and we pair it with a laser that sends out short pulses of light. Each pulse travels to this visible wall and some light scatters back to our camera, but we also use the wall to scatter light around the corner to the hidden object and back. We repeat this measurement many times to capture the arrival times of many photons from different locations on the wall. And after we capture these measurements, we can create a trillion-frame-per-second video of the wall."
Working at a trillion frames per seconds, it is possible to literally see waves of light scattered back from the hidden object and splashing against the wall. With each wave comes new information about the hidden object that sent it. From there, the team pass the measurements into a reconstruction algorithm to recover the 3D geometry of the hidden scene. And I thought algebra was tricky…
DRAGONS AND DISCO BALLS
Did you know glossy dragon statues and mirror disco balls reflect light differently?
It’s not something I’ve given an iota of thought to before today – glossy dragon statues aren’t really my thing, nor disco balls come to think of it! – but it’s fascinating to watch.
Frame by frame, Lindell talks through the different patterns of scattered light cast by each object:
"These waves of light illuminating the wall are like fireworks that last for just trillionths of a second. And even though these objects reflect light differently, we can still reconstruct their shapes. And this is what you can see from around the corner."
With honing, it’s clear to see how this camera / laser combo could be applied to day-to-day life. There’s a world of difference between a static street sign and a toddler running into the road, and it’s reassuring to know that future technology should be sophisticated enough to detect that.
Lindell’s final example sees him sport a rather fetching reflective suit whilst the camera system scans the wall at a rate of four times every second. The reflective suit enables the team to see where he is and what he’s doing, without directly imaging him – capturing an indirect video in real time. Lindell adds:
"We think that this type of practical non-line-of-sight imaging could be useful for applications including for self-driving cars, but also for biomedical imaging, where we need to see into the tiny structures of the body. And perhaps we could also put similar camera systems on the robots that we send to explore other planets."
Who knows what the future holds… Maybe these cameras will discover alien life forms? Or shed light on the deepest depths of our oceans? One thing’s for sure, the cars of the future will be a lot safer with this tech on board - compared to Larry’s periscope, anyway…
For more entertaining, insightful and thought-provoking talks, visit TED.com