Testing inference at all system scales.
Independent benchmarks of system inference performance, covering hardware & inference software across all scales: datacenter nodes/racks, laptops and workstations, and mobile phones.
Choose an inference scale
We benchmark systems of all sizes on the throughput, latency, cost and power use metrics that matter for decision-making. We customize our inference benchmarks for each of the three size tiers below.