In the last issue of Inside DSP, we explained why good benchmarks are important both for processor users and for processor designers. We also discussed why optimized benchmarks are critical when assessing a processor’s performance for digital signal processing tasks. Given that optimized benchmarks are required, if you’re a processor vendor, how do you ensure that your benchmark implementations are fully optimized, and therefore show your processor’s full potential?
Optimizing digital signal processing software is a multi-faceted process requiring specialized skills. Often, the largest performance gains come not from tweaking code, but from reordering computations or rearranging data in memory in ways that enable more efficient use of the processor. When BDTI certifies implementations of BDTI benchmark suites, expert BDTI engineers painstakingly ensure that the implementations are thoroughly optimized.
In some cases, processor vendors contract with BDTI to implement one or more BDTI benchmark suites on a processor. In most cases, though, the processor vendor’s engineers implement the benchmarks and then submit their implementations to BDTI for certification. In those cases, as part of the certification process BDTI engineers review the benchmark implementations in detail to ensure not only that they comply with the benchmark specifications, but also that they are fully optimized. Even in cases where the vendor’s engineers are experts in optimizing code for their processor, a second set of experienced eyes often results in significant improvements in benchmark performance.
For example, in one benchmark certification project, BDTI found that a vendor’s implementation of BDTI’s single-sample FIR filter benchmark used a circular buffer to update the delay line. Use of a circular buffer is the classical way to maintain a delay line. However, in this case, using a circular buffer resulted in the processor only utilizing half of the available memory bandwidth—thereby starving the MAC units of data. BDTI suggested an alternative approach, leveraging maximum memory bandwidth and keeping the MAC units busy. The result was a 33% increase in throughput.
In other cases, BDTI’s expertise in performance analysis helps ensure that a processor achieves the best performance scores possible. For example, in another benchmark certification project BDTI engineers analyzed the processor’s instruction set, pipeline, and memory architecture and determined that the processor was capable of significantly higher performance on four benchmarks from the BDTI DSP Kernel Benchmarks suite. BDTI provided feedback to the vendor’s engineers indicating that higher performance was possible on these benchmarks and suggested optimizations that might help achieve that performance. As a result, the vendor was able to achieve an overall BDTIMark2000 performance score that was 11% higher than the score resulting from its initial benchmark implementations.
The benefits of fully optimized benchmarks go beyond showing a processor in the best light. Optimizations discovered through the intensive benchmarking effort can also be used in application software. They can be used in software component libraries provided by the processor vendor, and they can be incorporated into customer training to ensure that customers get the most out of the processor.
In summary, BDTI’s benchmark certification process serves dual purposes. First, certification is key to having truly useful benchmarks, because it ensures that benchmark results reflect the true performance of the processor, and that the results enable fair, apples-to-apples comparisons. Second, certification facilitates new insights into how to get the most out of a processor—insights that can pay dividends not only for marketing purposes, but also in processor users’ application development efforts.
If you’re a processor designer or marketer, contact Jeremy Giddings of BDTI (giddings@BDTI.com) to find out how BDTI can help you credibly demonstrate the full advantages of your product, whether they are in the realm of performance, cost-performance, energy efficiency, or ease-of-use. For sample BDTI Benchmark results, see BDTI’s web site.
Add new comment