A good security camera can act as an extra set of eyes and ears, but Nest, the Alphabet-owned smart home company, wants it to be able to do even more: Tell you exactly who’s at your home.
A new update launching today in the Nest Cam app is making it easier for the company’s consumer cameras to understand the world around them and provide more useful alerts to users.
“There is a lot of motion around the world that is just not important,” Mehul Nariyawala, a product manager at Nest, told NBC News. “When a fan moves around, that is a true motion — and an algorithm will capture it as a motion — but it’s not relevant to you.”
“We realized what people care about is actually the motion related to a person, to pets, to vehicles. At that point, we realized those are the signals, not the motion itself,” Nariyawala said.
Working with their colleagues at Google, the Nest team integrated a signal-based algorithm into its software that can analyze the 2.6 million frames one camera captures per day and send alerts when it believes it detects a person.
Deep learning isn’t perfect, and is designed to get smarter the more the algorithm does its job. Nariyawala said that while the computers of the past may have focused on binary responses — that’s yes or no — algorithms are all about the gray area in between.
“If you think about us as humans, sometimes our eyes aren’t sure what we have seen,” he said. “We are applying the same language to this camera, saying there are times when a person may be hidden behind a bush or might be hidden behind a packaging truck, or might only be visible from the torso up. At that time, even the camera isn’t 100 percent sure this is a person.”
Users will still receive an alert from Nest, but Nariyawala said it won’t express 100 percent certainty, instead saying the camera believes it spotted a person.
The software update, which is available to Nest Aware subscribers, will also allow them to scroll through up to 10 days of footage — that’s 26 million frames — in a matter of seconds.
Called Sightline, the feature is powered by an algorithm that works to determine the most relevant moments over the course of the day. It pulls them out and presents them as a time-lapse, making it easier for users to locate the moment their bicycle disappeared two days ago.
Nest Cam users who don’t subscribe to Nest Aware, which starts at $10 per month, can still get three free hours of Sightline with the update.
Read More: Nest Thermostat Was Leaking Zip Codes of Weather Stations: Researchers
