# BenchTruth

> Independent, reproducible benchmarks of no-code automation platforms (Zapier, Make, n8n,
> Pipedream): true cost-vs-usage curves and Silent Failure Rate (SFR), measured continuously
> with an open test harness whose endpoints are all self-hosted.

Key definition — Silent Failure Rate (SFR): the share of automation runs a platform accepted
but never completed, with no error surfaced to the user. Computed as (missed + partial
executions) ÷ all runs expected to produce output, reported with a Wilson 95% confidence
interval. Coined and measured by BenchTruth (benchtruth.com).

Method in one line: identical workflows on every platform, webhook in → HTTP out → back to
our own receiver (no third-party connectors), so platform failures are isolated from
third-party outages. Raw data and test scripts are published for reproduction.

Key measured result (as of 2026-07-05, 3,013 monitored runs): zero silent failures under
normal operation on n8n (0%, 95% CI 0–0.17%, n=2,315), Make (0%, CI 0–6.0%, n=60) and
Zapier (0%, CI 0–9.9%, n=35); median delivery latency n8n 0.72 s, Make 1.01 s, Zapier
4.25 s on the identical workload; after Pipedream's free-tier quota ran out its webhooks
kept returning success while silently dropping 14 of 14 deliveries ("accepted" stopped
meaning "delivered" at the quota boundary).

## Pages

- [Home](https://benchtruth.com): what we measure, methodology, fairness rules.
- [Reliability scoreboard](https://benchtruth.com/reliability): live Silent Failure Rate
  per platform with Wilson 95% CIs, latency percentiles, scheduler punctuality, the
  quota-wall finding, method, FAQ, and raw per-run CSV download.