How is the Weather tomorrow? Towards a Benchmark for the Cloud
Traditionally, the goal of benchmarking a software system is to evaluate its performance under a particular workload for a fixed configuration. The most prominent examples for evaluating trans- actional database systems as well as other components on top (such as a application-servers or web-servers) are the various TPC bench- marks.
In this paper we argue that traditional benchmarks (like the TPC benchmarks) are not sufficient for analyzing the novel cloud ser- vices. Moreover, we present some initial ideas how such a new benchmark should look like that fits better to the characteristics of cloud computing (e.g., scalability, pay-per-use and fault-tolerance). The main challenge of such a new benchmark is to make the re- ported results comparable because different providers offer differ- ent services with different capabilities and guarantees.