A few weeks ago I participated in a panel discussion on benchmarking. The theme of the panel was how to benchmark multi-threaded and multi-core processors. In my view, this theme highlights a key problem with many benchmark approaches: too many benchmarks are designed to exercise hardware features, rather than to provide information that system developers need.