Image-sensor maker Himax, processor core vendor CEVA, and algorithm provider emza Visual Sense have partnered to develop a low-power "always on" vision sensor module with integrated visual analytics. The three companies offered a conceptual demonstration of the product, dubbed the WiseEye IoT vision sensor, at last month's Consumer Electronics Show. Assuming that WiseEye IoT is able to perform with sufficient accuracy and frame rates, with its target power consumption of 5-10 mW for common visual functions (e.g., human detection), it should find application in a variety of low-power IoT (Internet of Things) applications.
Taiwan-based Himax Technologies created the Himax Imaging subsidiary of in 2007, explained Antonio Tsai, Deputy Director of the company's Marketing Division in a recent briefing. Himax Imaging initially focused on high pixel count CMOS image sensor applications such as standalone and display-integrated webcams for computers, as well as front and rear cameras for smartphones and tablets. In recent times, however, these markets have become "mature", in Tsai's words, and his company therefore began searching for other business opportunities with higher growth potential, specifically those with more modest resolution needs but having stringent power consumption requirements.
The resulting initial product, the HM01B0 that the company introduced one year ago, is a 320x320 pixel 8-bit 1/11" CMOS image sensor targeting "Always On" applications, reflective of its sub-2mW operating power consumption at QVGA resolution and a 30 fps frame rate (both higher frame rate and lower power consumption mode options are also available) (Figure 1).
Figure 1. Himax's ultra low power HM01B0 sensor, available in both bare die and packaged forms, is the image capture foundation of the WiseEye IoT vision processing subsystem.
As Himax began promoting the HM01B0 to potential customers, Tsai recounted, it found that many of them were looking for a complete solution: not just the sensor or a sensor-plus-lens-module but also an associated vision processor and algorithms to run on it. Addressing the latter algorithm need, Himax last summer struck up a relationship with Israel-located emza Visual Sense via a potential customer introduction. And through emza, Himax cultivated a connection with another Israeli technology supplier, CEVA.
The initial outcome of the collaboration, the WiseEye IoT vision sensor demonstrated in proof—of-concept form at CES, is based on CEVA's TeakLite-4 DSP core architecture, covered in InsideDSP in 2012 and moderately upgraded by CEVA two years later (Figure 2). At first glance, the core choice might seem curious, since TeakLite-4 is mostly known as a modest DSP for voice and other audio processing applications, versus an upscale vision-focused DSP such as CEVA's MM3101 and more recent XM4 and XM6. However, as Tsai explained, the objective for this project was to first nail down the necessary processor cost and power consumption targets, and then identify the most robust DSP that met these objectives.
Figure 2. CEVA's TeakLite-4 DSP architecture, while best known today for its voice and other audio processing capabilities, also delivered on the cost and power consumption objectives for WiseEye IoT's basic vision processing needs.
At CES, the three companies' proof-of-concept demonstration mated the Himax HM01B0 sensor module to CEVA's TeakLite-4 Development Platform, running the TL410 variant of the processor core:
Under development by Himax at the present time is a WiseEye IoT mini-PC board prototype with target dimensions of 2 cm x 2 cm. The prototype will contain the HM01B0 sensor module, a discrete TeakLite-4-based processor, and both volatile and nonvolatile memory, the latter housing the board firmware including the emza-developed vision algorithms. Himax plans to demonstrate this prototype at the upcoming April 2017 ICS West security show in Las Vegas, Nevada.
Preliminary product specifications postulate a number of possible applications for WiseEye IoT:
- Intruder detection
- Human classification (full or upper body)
- Occupancy identification (moving or static people)
- Face recognition
- People counting
- Breathing rate measurements
- Gesture recognition
Tsai estimates that an elementary motion detection function running on a 200 MHz unspecified member of the TeakLite-4 core family could be implemented on WiseEye IoT at less than 5 mW of total power consumption; the companies' target is 1-2 mW (Figure 3). A more elaborate human classification function, operating at a sensor resolution- and lens focal length/viewing angle-determined distance of up to 25 meters, is estimated to draw between 5-10 mW. Frame rates for either function are also unspecified at the present time.
Figure 3. CEVA's TeakLite-4 DSP family's four members offer varying amounts of on-core MACs, along with other resources, enabling a variety of WiseEye IoT capabilities at various cost and power consumption points.
While the WiseEye IoT documentation proposes even more elaborate operations, such as face detection (up to 8 meters) and face recognition (up to 2 meters), Tsai admits that their implementation will likely require at minimum a faster-than-200 MHz TeakLite-4 clock speed; perhaps even a different, more vision-optimized DSP architecture. Note, too, that the quoted power consumption estimates do not include the additional power draw of any wired or wireless communication subsystem that might be needed to transfer the analytics results elsewhere for alert, archive and/or further analysis.
Operating distance is also dependent on accuracy aspirations, of course; Tsai declined to quote specific accuracy targets for various functions, but noted that at minimum, security systems based on WiseEye IoT should exceed the reliability of today's widespread and rudimentary PIS (passive infrared sensor) alternatives with ease. To that point, too, Tsai points out WiseEye IoT's target operation across a wide 1-20,000 lux ambient light range, enabling it to closely mimic PIS systems' nighttime monitoring capabilities.
While WiseEye IoT is little more than a proof of concept at the moment, it's based on proven production silicon and software building blocks, and the three companies are highly motivated to deliver both standardized and customer-optimized versions of it as appropriate for any particular situation. And as the PIS case study points out, if WiseEye IoT meets cost, accuracy, frame rate, power consumption and other metrics, there are already established markets for it to target, not to mention plenty of additional emerging opportunities.
Add new comment