Benchmarks are used by processor vendors for a variety of purposes. They are used internally for competitive analysis and new product development. Externally they are used in presentations, articles, and marketing materials to credibly demonstrate the advantages of a processor. In fact, sophisticated processor users increasing demand independently verified benchmark for use in their processor selection analyses. Given the importance placed on benchmarks, processor vendors are well advised to invest sufficient resources to obtain benchmark results that show the true capacity of their processor. For digital signal processing benchmarks, this means thoroughly optimizing the code to extract maximum performance.
Typically, processor vendors use their in-house engineering resources to optimize benchmark implementations. This can yield good benchmark results, particularly when the benchmarks are implemented by engineers with intimate knowledge of a processor’s architecture and sufficient time to do a thorough job. In some cases, though, the processor vendor may prefer to outsource benchmark implementation work. For example, the processor vendor’s own engineering resources may be better used for developing the company’s products. In other cases – for example, if the vendor is a start-up, or new to DSP applications – the vendor may not have the right skills in-house for creating highly efficient digital signal processing code. In such situations, vendors often turn to BDTI for benchmark implementation services.
With twelve-plus years of experience developing and implementing benchmarks, BDTI has the skills and infrastructure to implement benchmarks quickly, with highest quality results. BDTI ensures confidence in benchmark results through rigorous optimization, drawing on BDTI’s experience with many different types of architectures and programming tools. In addition to world-class coding skills, BDTI applies knowledge of algorithm-level optimizations to adapt benchmark implementations to make best use of architectural features.
Recently, BDTI implemented the BDTI DSP Kernel Benchmarks™ on a new DSP processor from a major vendor. Having reviewed and certified the vendor’s implementation of the BDTI DSP Kernel Benchmarks on the vendor’s previous generation DSP, BDTI was already familiar with the basic architectural features of the new processor. Enhancements in the new architecture, such as a significantly deeper pipeline, rendered the benchmark implementations created for the previous generation processor far from optimal. Running the previous generation benchmark code on the new processor, BDTI found that while two of the benchmarks were unaffected by architectural changes, the others showed an average 20% cycle count increase. By leveraging the existing benchmark code from the previous processor, BDTI’s experience optimizing code for processors with similar features, and BDTI’s familiarity with the benchmarking process, BDTI was able to reduce these cycle counts significantly and produce benchmark implementations that showed the new processor’s true capacity. The vendor’s own experts reviewed BDTI’s implementations of the benchmarks to look for further optimization opportunities and did not find much room for improvement. The vendor found improvements that yielded an increase in the processor’s overall BDTIsimMark2000™ score of 3%. Compared to the score obtained by using highly optimized code for the previous generation processor, BDTI was able to improve the BDTIsimMark2000 score by 30%. The work was completed on a tight schedule, allowing the vendor to provide a major customer with the benchmark results needed to move forward with processor selection.
To learn more about how BDTI can help you create high quality benchmark implementations quickly and efficiently, please contact Jeremy Giddings at BDTI (giddings@BDTI.com).
Add new comment