Jeff Bier’s Impulse Response—Special-purpose Processors Focus on Computer Vision

It's now very clear that computer vision is becoming a mass-market technology. Modern, practical computer vision (or, "embedded vision," as I prefer to call it) is rapidly becoming essential in cars, for example, where it enables a host of valuable safety features. In smartphones, computer vision enables better photographs and image-based search. And new smart-home devices use vision to perform functions such as messaging you when your kid gets home, or when an unknown person arrives.

This is a huge change from typical computer vision systems of five or ten years ago, which mainly performed tasks like automated inspection in factories. These systems typically cost tens of thousands of dollars and consumed hundreds of watts of power. In contrast, mass-market product prices are generally in the hundreds of dollars (or less) and power budgets are often a few watts (or less).

Delivering the processing performance required by computer vision applications – typically tens of billions of operations per second – with cost and power consumption appropriate for mass-market products is a tough challenge. A technique that's worked well in many other application domains (such as video compression for set-top boxes) is to create fixed-function hardware that implements the heavy-lifting algorithms. By foregoing programmability and tailoring the hardware design precisely to the needs of the algorithm, chip designers can achieve optimal cost- and energy-efficiency. But in most computer vision applications, algorithms are changing fast, so programmability is essential.

In contrast, specialized programmable processors often achieve an attractive combination of performance, efficiency and programmability. Broadly speaking, DSPs are perhaps the best-known and most widely used type of specialized processor – billions of them are shipped each year in mobile phones, headsets, cars, and thousands of other products. Now that embedded vision applications are becoming mainstream, a new class of vision-specific processors has emerged. And recently, there's been a surge of new entrants.

Just in the past few months, new vision-specific co-processors –offered as licensable cores for incorporation into chips – have been announced by CEVA, CogniVue and Synopsys. These join previously introduced vision-specific co-processors from Apical, Cadence, Movidius (a start-up chip company that just raised $40 million in venture capital) and videantis. In addition to these vision-specific processors, many vision system developers use other types of parallel co-processors, such as GPUs, DSPs and FPGAs.

Embedded vision is an attractive market for processor suppliers, not only because of the growing size of the market, but also because of its insatiable appetite for processor performance. There's a vast amount of information available via image sensors. As more processing performance becomes available, more useful information can be extracted from images.  For example, increasing the resolution of the image sensor used in an automotive safety system increases the distance at which the system can detect objects like pedestrians and speed-limit signs – but requires more processing power. Similarly, adding a second camera (such as one to track where the driver is looking) can enable valuable new capabilities, but also requires more processing power. And running multiple vision functions at once (such as simultaneous tracking vehicles, lane markings and road signs) also has clear value, but again requires more processing power.

The wealth of vision processor options is great news for chip and system designers, because a range of processor choices optimized for your application domain makes it more likely that you'll be able to find a processor that fits your needs.  At the same time, the large number of diverse processor options – and the rapid pace of new options being introduced – can make it difficult to identify your options and make the best selection decision. The Embedded Vision Summit conference is a great resource for learning about vision processor options. The Summit is a unique conference for product developers who are creating more intelligent products and applications by integrating computer vision into their designs. It will take place on May 12 in Santa Clara, California. I'll be presenting a talk entitled "Choosing a Processor for Embedded Vision: Options and Trends." I'll be joined by experts like Chris Rowen of Cadence (presenting on vision processor instruction set design) and Roberto Mijat of ARM (presenting on the role of integrated GPUs in vision applications). In addition, many of the processor suppliers targeting vision applications will be demonstrating their latest offerings at the Summit. I hope to see you there!

Jeff Bier is president of BDTI and founder of the Embedded Vision Alliance Post a comment here or send him your feedback at http://www.BDTI.com/Contact.