Processor vendors who use BDTI's benchmarks can contract with BDTI to implement the benchmarks, or the vendor can implement the benchmarks themselves. If they choose the latter, they can submit their implementations to BDTI for certification. As part of the certification process BDTI engineers review the benchmark implementations in detail to ensure not only that they comply with the benchmark specifications, but also that they are fully optimized. Even in cases where the vendor’s engineers are experts in optimizing code for their processor, a second set of experienced eyes often results in significant improvements in benchmark performance.
For example, in one benchmark certification project, BDTI found that a vendor’s implementation of BDTI’s single-sample FIR filter benchmark used a circular buffer to update the delay line. Use of a circular buffer is the classic way to maintain a delay line. However, in this case, using a circular buffer resulted in the processor only utilizing half of the available memory bandwidth—thereby starving the MAC units of data. BDTI suggested an alternative approach that leveraged the processor's maximum memory bandwidth and kept the MAC units busy. The result was a 33% increase in throughput.
In other cases, BDTI’s expertise in performance analysis helps ensure that a processor achieves the best performance scores possible. For example, in another benchmark certification project BDTI engineers analyzed the processor’s instruction set, pipeline, and memory architecture and determined that the processor was capable of significantly higher performance on four benchmarks from the BDTI DSP Kernel Benchmarks suite. BDTI provided feedback to the vendor’s engineers indicating that higher performance was possible on these benchmarks and suggested optimizations that might help achieve that performance. As a result, the vendor was able to achieve an overall BDTIMark2000 performance score that was 11% higher than the score resulting from its initial benchmark implementations.
The benefits of fully optimized benchmarks go beyond showing a processor in the best light. Optimizations discovered through the intensive benchmarking effort can also be used in application software. They can be used in software component libraries provided by the processor vendor, and they can be incorporated into customer training to ensure that customers get the most out of the processor.
In summary, BDTI’s benchmark certification process serves two purposes. First, certification is key to having truly useful benchmarks, because it ensures that benchmark results reflect the true performance of the processor, and that the results enable fair, apples-to-apples comparisons. Second, certification facilitates new insights into how to get the most out of a processor—insights that can pay dividends not only for marketing purposes, but also in processor users’ application development efforts.
If you’re a processor designer or marketer, contact Jeremy Giddings of BDTI (giddings@BDTI.com) to find out how BDTI can help you credibly demonstrate the full advantages of your product, whether they are in the realm of performance, cost-performance, energy efficiency, or ease-of-use. For sample BDTI Benchmark results, see BDTI’s web site.