# Hardware measurement discipline: what makes a number trustworthy

> Fix the build, driver, power plan and temperature before comparing anything, report percentiles instead of averages, and keep the raw data.

**Tested on**

- Build: 26300.9457 / 26100.33650
- Edition: Pro (26H2) / Server
- Architecture: x64
- Evidence: E1
- Verified: 2026-10-29

## The short answer

A hardware number is only meaningful with **five things fixed** and **three things reported**:

Fixed: build, driver version, power plan, ambient/thermal state, workload.

Reported: the **distribution** (not the mean), the **method** (how it was captured), and the **environment** (what machine, what settings).

Most "this is faster" claims fail on at least three of those.

## The five variables people forget

| Variable | Why it changes the result |
|---|---|
| **Windows build** | scheduler, power behaviour, driver model change between builds |
| **Driver version** | GPU drivers change clocks and frame pacing |
| **Power plan / mode** | minimum processor state and boost change sustained clocks |
| **Temperature** | a cold machine boosts higher than a warm one |
| **Background load** | Windows Update, indexing, sync clients |

If two runs differ in any of these, you measured two different machines.

## Report distributions, not averages

For interactive workloads (games, UI, audio), the average hides the problem:

- report **1% low / 99th percentile** frame time, not just average FPS;
- report **worst frame time**, because one 200 ms hitch is what the user notices;
- for throughput workloads, report **median and spread** across runs, not the best run.

"Average 120 fps" and "median 120 fps with 1% lows at 40 fps" describe the same average and completely different experiences.

## A minimal reproducible protocol

1. **Snapshot the environment**: build (`winver`), driver version, power plan, room temperature, background apps disabled.
2. **Warm up** for a fixed period so clocks and temperatures stabilise.
3. **Run the same workload 3 times**; keep every run, not the best one.
4. **Capture with a real tool** — WPT (wpr/wpa), performance counters, or the workload's own frame-time log. Task Manager is an indicator, not a measurement.
5. **Report the median of the runs plus the spread**, and the worst case.
6. **Change one variable** and repeat the whole protocol.

```powershell
# a reproducible counter capture
Get-Counter '\Processor Information(_Total)\% Processor Performance',
            '\Processor Information(_Total)\% of Maximum Frequency',
            '\Memory\Available MBytes' -MaxSamples 60 -SampleInterval 1 |
  Export-Counter -Path "$env:TEMP\run1.blg" -Force
```

`Export-Counter` gives you a `.blg` you can reopen later, which is the difference between data and a memory of a number.

## Where measurements go wrong

| Failure | Symptom in the report |
|---|---|
| Comparing driver versions | "the new driver is 15% faster" with no version pinned |
| Comparing a cold run to a warm run | first run fastest, later runs slower — called "throttling" |
| Best-of-N reporting | numbers nobody can reproduce |
| Different scenes/levels | huge unexplained variance |
| VM result presented as bare metal | plausible numbers, wrong claim |
| No raw data kept | cannot re-analyse or check the method |

> **Caution: A VM measurement is not a bare-metal measurement**
>
> Virtualised GPU, CPU and storage results describe the VM. They are useful for relative comparisons inside the same VM, and misleading as absolute performance. Label the environment every time.

## What this site does

- **E1** pages cite documentation and make no performance claim.
- **E2** pages report what was observed in a fixed PVE template, with the host/guest configuration named.
- No page publishes a frame rate, temperature or power number without the hardware on the bench.

See [lab method](https://winbook.fyi/en/lab/method/) for the evidence levels (E1/E2/E3, M-V/M-P) and how snapshots are kept.

## Related

- [WDDM, HAGS and MPO](https://winbook.fyi/en/hardware/gpu/wddm-hags-mpo/)
- [Throttling: thermal or power](https://winbook.fyi/en/hardware/laptop/throttling/)
- [WHEA and bugchecks](https://winbook.fyi/en/hardware/diag/whea-bugcheck/)

## Lab boundary

E1. The tooling and counter behaviour are documented by Microsoft. This page describes method; it contains no measurement of its own.

## Sources

- [Microsoft Learn — Windows Performance Toolkit](https://learn.microsoft.com/en-us/windows-hardware/test/wpt/)
- [Microsoft Learn — Windows Engineering Guide](https://learn.microsoft.com/en-us/windows-hardware/test/weg/)
- [Microsoft Learn — System power states](https://learn.microsoft.com/en-us/windows/win32/power/system-power-states)

---

Source: https://winbook.fyi/en/hardware/diag/measurement-method/