Benchmarks

Jeff Bier’s Impulse Response—Making Benchmarks Useful

A few weeks ago I participated in a panel discussion on benchmarking. The theme of the panel was how to benchmark multi-threaded and multi-core processors. In my view, this theme highlights a key problem with many benchmark approaches: too many benchmarks are designed to exercise hardware features, rather than to provide information that system developers need. In most embedded applications, system developers care about high-level system attributes such as low cost, long battery life, Read more...

Case Study—Benchmarks for Tools, Software Libraries, and More

System developers often rely on processor benchmarks to gauge system performance. However, the processor is just one of many components that determines overall performance. Fully understanding system performance requires careful analysis of many other elements, such as code-generation tools and third-party software libraries. Unfortunately, a host of factors can confound attempts to analyze these components. For example, it is difficult to prevent variations in programmer skill and style Read more...

Case Study: Measuring Multimedia Performance

At first glance, measuring processor performance on multimedia applications can seem straightforward. Many multimedia applications are based on published standards and widely available software. For example, MPEG-4 video decompression software is available for most popular processors. Because such software is often readily available, measuring multimedia performance may seem to be a simple matter of checking the processor vendor's published performance data for the relevant software modules Read more...

Case Study—Benchmarks for Tools, Software Libraries, and More

System developers often rely on processor benchmarks to gauge system performance. However, the processor is just one of many components that determines overall performance. Fully understanding system performance requires careful analysis of many other elements, such as code-generation tools and third-party software libraries.   Unfortunately, a host of factors can confound attempts to analyze these components. For example, it is difficult to prevent variations in programmer skill and style Read more...

Jeff Bier’s Impulse Response—Stuck in the Past

Vendors announcing new signal-processing chips tend to brag about the clock speed of the processor core, just as they did ten years ago. “Look at our hot new processor!” they proclaim. “It’s got lots and lots of Hertz!” But in embedded applications—just as in PC applications—comparing chip performance solely on the basis of core clock speeds never tells the whole story. In fact, it can be downright misleading. The performance of today’s highly integrated chips isn’t just a function of the Read more...

Inside DSP: A New Slant—Welcome to InsideDSP

CMP Media and BDTI are pleased to introduce Inside[DSP], an innovative new series of periodic supplements to EE Times. Each Inside[DSP] supplement will focus on the digital signal-processing technology behind a particular end-equipment market. These will include product categories such as consumer audio and video, mobile multimedia devices, automotive signal-processing applications, and communications equipment. “What?” you say, “Another trade publication? Don't these guys understand that I' Read more...

New BDTIsimMark2000™ Measures Pre-Silicon DSP Speed

Processor users often want a simplified way of comparing processors' speeds in DSP applications. To this end, BDTI publishes the BDTImark2000™, a single-number DSP speed metric based on BDTI's suite of DSP benchmarks, the BDTI Benchmarks™. Because it is based on realistic DSP algorithm kernel benchmarks, the BDTImark2000 characterizes a processor's signal processing speed far more accurately than traditional simplified measures such as MIPS or MFLOPS. BDTI's policy is to verify its Read more...