Lately I’ve been thinking about the relationship between embedded vision and privacy.

Surveillance cameras are nothing new, of course. For decades, they’ve been ubiquitous in and around restaurants, stores, banks, offices, airports, train stations, etc. In the course of a typical week, I’d guess that my image is captured by dozens of these cameras.

Ten years ago, no one would have expected that neural networks would deliver such impressive results on computer vision problems. Since 2012, when AlexNet, a deep convolutional neural network, produced a significantly lower error rate on the ImageNet Large Scale Visual Recognition Challenge than traditional feature extraction approaches, investigation of neural network approaches for visual understanding has intensified. These efforts culminated in 2015 when a deep residual network (ResNet) bested typical human performance by achieving an ImageNet error rate of 3.6%.

Xilinx, like many companies, sees a significant opportunity in burgeoning deep neural network applications, as well as those that leverage computer vision...often times, both at the same time.

Microsoft's Cognitive Toolkit (formerly CNTK) is a relatively recent entrant to the open-source deep learning framework market.

At the recent Embedded Vision Summit, I was struck by the number of companies talking up their new processors for deep neural network applications. Whether they’re sold as chips, modules, systems, or IP cores, by my count there are roughly 50 companies offering processors for deep learning applications. That’s a staggering figure, considering that there were none just a few years ago.

As system designers race to make IoT and edge devices more capable, they are incorporating increasingly complex and demanding algorithms. Cameras and microphones are now the eyes and ears of systems that help us drive our cars, maintain the safety of our homes, diagnose health issues, and much more. Processor vendors, seeking to meet escalating requirements of processing sensor data at the edge, are designing new heterogeneous devices that integrate CPU cores, DSPs, GPUs, and other specialized processing engines for tasks such as image processing and deep neural networks.

ARM's latest image signal processor (ISP), the Mali-C71, marks the first fruit of the company's year-ago acquisition of Apical.

At last year's Embedded Vision Summit, Cadence unveiled the Tensilica Vision P6 DSP, which augmented the imaging and vision processing capabilities of its predecessors with the ability to efficiently execute deep neural network (DNN) inference functions.

Remember when mobile phones were for making phone calls? Given today’s reality, it can be difficult to recall the time – not so long ago – when mobile phones had one purpose: making phone calls. Today, the situation is very different; most people use their phones mainly for sending texts, reading email and news, social networking, navigating, shopping and watching videos. And maybe – rarely – making a phone call.

As the number of IoT devices increases, so does the need for intelligence at the edge—intelligence that will enable a device to acquire insights from its surroundings and make decisions in real-time. Particularly for devices such as drones, personal robots, and autonomous vehicles, real-time decision-making capability is a must. Machine learning approaches, such as deep and convolutional neural networks (DNNs and CNNs) are proving to be the most accurate means for object detection and recognition for these devices.