Benchmark shows cloud warehouse costs can vary wildly

CorrDyn and Estuary have published an independent benchmark comparing seven cloud data warehouses. The study found wide differences in annual costs for the same workload.

The benchmark covered Snowflake, Databricks, BigQuery, Redshift, Microsoft Fabric, ClickHouse and MotherDuck. CorrDyn ran 244 queries on data volumes from 1 GB to 1 TB, testing each system with a single user and with bursts of up to 100 concurrent sessions.

The headline finding was the cost gap for serving 50 concurrent users on 100 GB of data. According to the benchmark, that workload cost about USD $150,000 a year on one engine and about USD $420,000 on another, or roughly USD $17 an hour versus USD $48 an hour.

The authors said no system topped every measure. Each engine led at least one test, and some results challenged established market views.

Two products often seen as less competitive in current warehouse discussions, Redshift and MotherDuck, ranked first and second on reporting queries for a single user across all tested data volumes. The study also found that adding more compute did not always improve performance. Some engines ran slower on small datasets when larger configurations were used.

Cost pressure

The work focused heavily on concurrency, which the authors described as a bigger cost driver than single-query speed. As more staff and software agents run direct queries, they argued, data teams are being forced to buy more compute to keep response times within acceptable limits.

At 1 TB, no single compute unit handled more than 10 concurrent users while keeping typical response times under five seconds, the study found. That suggests teams with large dashboard audiences may need several compute units running together, with costs shaped by how those units are combined and how long they stay active.

Another finding centred on idle time. The same thousand queries cost USD $0.31 on one engine and USD $7.14 on another, but the benchmark said the bigger issue for many users was whether billing stopped between query bursts.

David Yaffe, Chief Executive Officer of Estuary, said the study was designed to give buyers a clearer picture of actual spending. "Most teams choose a warehouse once and live with it for years. They pick on reputation because the price list is no help: the hourly rate says little about what you'll actually pay."

"The study measured what each engine costs on real workloads. When one pipeline can feed several engines from the same data, the warehouse stops being a ten-year decision. A team can run each workload on the engine that suits it and change its mind when the numbers change," Yaffe said.

Public data

CorrDyn published the benchmark code, query files and raw results database on GitHub, including failed query runs. The study used data supplied to all seven systems through a single Estuary pipeline, so each engine was tested on the same underlying dataset.

Automatic scaling was switched off during the tests. In total, the load testing produced 191,176 query runs across 42 compute sizes.

The benchmark drew on three public benchmark suites and examined how systems behaved under different query patterns. Query structure proved a major factor in the rankings, particularly where table joins were involved.

On that point, the study said join-heavy reporting workloads changed the performance order more than compute-size changes did. It also said one engine that led on scan-style queries ran out of memory on join-heavy reporting queries at 1 TB.

James York-Winegar, Chief Executive Officer of CorrDyn, said the exercise was intended to address what he described as a lack of neutral evidence in the market. "We kept telling clients that the right warehouse depends on their workload and budget, and we could not point to anything that proved it."

"Every vendor publishes a benchmark it wins. So we measured all of it ourselves and published the code, so anyone who thinks we got it wrong can check," York-Winegar said.

The findings add to a broader debate in cloud data infrastructure over whether procurement decisions should focus less on headline compute prices and more on sustained usage patterns, concurrency demands and billing during periods of low activity. For buyers comparing warehouse products, the benchmark offers a rare public dataset showing not just successes but failures, with every raw result included.