Investment in a particular technology segment, not only by small startups but also by established suppliers, tends to be a dependable indication that the application has large business potential and lengthy staying power. Consider embedded vision, the use of computer vision techniques to extract meaning from visual inputs in embedded systems, mobile devices, PCs and the cloud. BDTI, accurately predicting that embedded vision would rapidly become an important market, founded the Embedded Vision Alliance in May 2011.
Since then, embedded vision processor announcements have come fast and furious, whether integrated within SoCs sold by the same companies (such as Analog Devices and Texas Instruments), available for licensing and provided by core suppliers (i.e., Apical, CEVA, Tensilica (now Cadence) and videantis) or both delivered in SoC form for certain markets and licensed to others, as in the case of companies like CogniVue. Imagination Technologies, a recent addition to the Embedded Vision Alliance member roster, becomes the latest supplier with its Raptor product line of image processing cores, which are also capable of implementing vision processing functions both standalone and in heterogeneous partnership with CPU and GPU cores.
Imagination Technologies is a name that may be most familiar to you from the company's PowerVR graphics core line, which is a significant player in ARM-based mobile application processors (where it competes against ARM's own Mali line, for example, along with Qualcomm's proprietary Adreno GPUs, among others). PowerVR graphics cores have also been leveraged by Intel in several generations of Atom CPU-based SoCs, and can be paired with the MIPS processor cores that Imagination Technologies acquired in February 2013. And the company also develops other function-specific core offerings, such as video encoders and decoders, as well as its Ensigma communications processors (Figure 1).
Figure 1. Sourcing multiple processing cores from the same supplier delivers numerous advantages, according to Imagination Technologies.
Imagination Technologies cites a number of advantages to migrating the imaging and vision processor core onto the main application processor, and away from either a standalone ISP (image signal processor) chip or alternative core integration within the CMOS image sensor:
- No functional constraints that might be caused by the image sensor's or standalone chip's comparatively conservative manufacturing process
- Higher performance due to the application processor's same process advantages
- Access to main system memory for enhanced functionality, and
- Reduced cost and increased flexibility in image sensor choice
One other advantage cited by company representatives, "leverages other Imagination IP to create optimized subsystems," has obvious "one-stop shopping" fiscal upsides to Imagination Technologies. But it also leads to some intriguing potential implementation benefits for core licensees as well. Consider, for example, the relative ease of partitioning and optimizing various imaging and vision processing functions across multiple cores from the same supplier, in a heterogeneous SoC arrangement (Figure 2):
Figure 2. Partitioning algorithms (and portions of algorithms) among various cores in an SoC's heterogeneous processor array enables each core to tackle tasks for which it is optimally suited.
Specifically consider, for example, the added quality-versus-bitrate benefit that might be accrued by a video encoder core if it not only had access to the "raw" RGB data coming out of an image sensor (Figure 3), but also to histogram and other supplemental information supplied by the imaging core ahead of it in the processing chain (Figure 4).
Figure 3. Leveraging an ISP on-board the application's processor delivers numerous advantages, according to Imagination Technologies, versus alternative discrete or image sensor-embedded ISP approaches. These include application processor access to "raw" RGB data versus pre-processed YUV information.
Figure 4. Access to data beyond the base pixel information coming out of an image- or vision-processing core can enable a downstream video encoder to deliver superior single-pass compression results.
Speaking of image processing, a notable portion of Raptor's development know-how comes from Nethra Imaging, an ISP supplier that Imagination Technologies acquired in June 2012. In this respect, the company has entered the vision processing market from an image processing foundation, following a similar evolutionary path to that of Apical.
To date, Imagination Technologies has publicly revealed little detailed information about Raptor's implementation, aside from noting that the core's pipeline pixel width can be flexibly matched to that of the source image sensor, conserving power versus an alternative implementation using a DSP with a fixed data path width. Four-bit (entry-level), eight-bit (mainstream), ten-bit (Ultra HDTV) and sixteen-bit (high-end, and computer vision) image sensors are all supported, as are various sensor-to-SoC interface schemes. Raptor can also support dual-sensor (i.e., stereo) 3D camera arrangements, along with front-and-back dual cameras in smartphones and tablets; explicit support for alternative depth sensor schemes such as structured light and time of flight is not currently in place.
Multimedia Technology Marketing Director Peter McGuinness also made a point of noting Raptor's flexibility in terms of the ability to access, extract and re-insert data partway through the processing pipeline. Multiple "tap points" in the architecture mean, for example, that custom hardware or software can process the data before or after the de-mosaic or other filtering steps, prior to or subsequent to monochrome-to-full color Bayer conversion, and so forth, as suggested in Figure 1. McGuinness also explicitly showcased the "zero memory" direct pass-through of data from Raptor to a video encoder core (for example) without intermediary system memory transfers, thereby reducing both power consumption and latency.
The first family member of Imagination Technologies' Raptor core line is now implemented in RTL (register transfer level) form and available for licensing. Additional Raptor implementations will follow later this quarter. Imagination Technologies' endorsement of embedded vision by virtue of its Raptor announcement provides valuable evidence of the viability of this burgeoning, ultimately widespread application category. A blossoming list of supplier options might complicate your sourcing decisions somewhat, but all in all it makes for an overwhelmingly positive shift in situation.
Add new comment