Back to home

Numbers you can put your name on

100% from n=2 is the most confident-looking number in any report — and the least supportable. Gotopico is built to refuse to show you that number. This page is the policy, with the real thresholds.

One suppression policy, product-wide
Real significance tests
Guardrails before you field

Thin bases don't get reported

One policy decides whether a percentage may exist, everywhere in the product — the analysis layer and the pre-field checks quote the same thresholds, so a study is never told two different stories about the same base.

Below n=15: suppressed
The percentage is withheld entirely. You see “Base too small to report (n=12)” where the number would go. Internally the value is null, deliberately not 0.0 — anything that forgets to check fails visibly instead of rendering a confident zero.
n=15–29: flagged
The percentage is reported, permanently marked low-base, and carries its margin of error: “62.0% (low base, n=24)”. The reader is never left to guess how soft the figure is.
n=30+: reported
Reported with its base and a 95% margin of error attached. Thirty is the same floor the pre-field guardrails and the runtime sample-size flags already use — one convention, met everywhere.

Operational rates vs. reportable proportions

Whole-sample rates — survey completion, per-question drop-off — are a different animal: hiding them would obscure the fielding status of a freshly launched survey without preventing any misreading. So they are annotated instead of suppressed: the rate is always shown next to its base, tagged when it sits below the reporting minimum, with its margin of error making a thin base look thin.

No precision for withheld figures

A margin of error is never quoted for a suppressed percentage. Quoting a precision for a figure withheld as unreliable invites reporting the precision instead — so the policy withholds both.

Would we publish this percentage?

Drag the sample size and watch the verdict change. These are the exact thresholds the product applies — suppress below n=15, flag low-base below n=30 — not a simplified demo.

62% ± 8.7 pts

95% interval: 53.3% – 70.7% · n = 120

Reportable

Margin of error is the 95% normal-approximation half-width, the same formula the reporting layer uses. Small bases make wide intervals: at n=30 a 50% result is ±17.9 points.

Every base travels with its chart

Purchase intent by segment
n=412

Illustration of the dashboard's chart header.

Charts in the analytics dashboard carry their base in the header — an n=412 badge next to the title, not a footnote in an appendix. Model-fit banners state their sample the same way.

Significance testing done properly

“Is this difference real?” gets a real answer: the appropriate test for the data, an effect size next to the p-value, and intervals chosen for coverage rather than convenience.

Chi-square with effect size
Group-vs-group proportion tables get a chi-square test of independence — and Cramér's V alongside the p-value, so a "significant" difference that is practically tiny says so.
Two-proportion z-test
Comparing two segments' percentages uses a proper two-proportion z-test, with a Newcombe-Wilson interval on the difference for honest coverage at real-world sample sizes.
t-tests and one-way ANOVA
Mean comparisons run an independent-samples t-test for two groups and one-way ANOVA for three or more, with pairwise follow-ups — the right test is picked from the shape of your data.
Wilson intervals, not naive ones
Confidence intervals on proportions use the Wilson score method, which behaves properly for small samples and extreme percentages where the textbook normal approximation falls apart.
NPS with honest error bars

NPS confidence intervals are built from Wilson intervals on the promoter and detractor shares, combined conservatively — the interval is honest about the score being a difference of two noisy percentages.

Comparing NPS across segments runs a significance test on the promoter rates and states the verdict in plain language, p-value included — whether or not the difference holds up.

Cross-tabs: per-cell marks that earn their stars

Cross-tabulations mark individually significant cells using adjusted residuals — but only inside a table whose overall chi-square test passed first. Cell marks are never inferred from eyeballing percentages.

The output states the exact residual threshold a cell had to clear, and how many cells cleared it. Zero marked cells in a significant table is reported as a real result, not papered over.

Guardrails before you spend a cent on fieldwork

Every planned pricing or choice study is evaluated before it fields — and re-validated when you hit publish. Each check returns a verdict of ok, warning, or blocking, with a plain-language message and a concrete remedy. A design that would produce statistically meaningless results cannot be fielded.

blocking

A Van Westendorp plan under 15 respondents is blocked outright — the curve intersections would be driven by single respondents.

warning

A MaxDiff plan below the Sawtooth/Orme rule of thumb (n ≥ 500 × items ÷ (tasks × shown)) gets a warning with the exact number to aim for.

blocking

A conjoint design whose respondents can't mathematically cover its utility parameters — tasks × (concepts − 1) below the parameter count — cannot be fielded at all.

warning

More than 15 choice tasks per respondent triggers a fatigue warning: past that point, speeding and straight-lining degrade the data you paid for.

blocking

Segment-level estimates with fewer than 100 respondents per segment are blocked; below 200 per segment you get a warning before you commit.

And after fielding, the same rule of thumb is run against what you actually collected: choice-model results carry an achieved-sample verdict — high, medium, or low confidence, with directional-only results labeled as exactly that.

A chart palette that is measured, not asserted

Plenty of tools claim a “colorblind-friendly” palette. Ours is validated by an automated check that parses the actual hex values out of the chart theme source — so the claim can never drift from the colors the app really draws with — and fails the change if any check breaks.

  • Adjacent series stay distinguishable under simulated protanopia, deuteranopia, and tritanopia, with a minimum perceptual-difference floor
  • A separate, higher separation floor for full-color vision
  • Every hue kept in a usable lightness band, and above a chroma floor, against both the light and dark card surfaces
  • 3:1 contrast against the chart surface — or the check demands visible labels instead

The check covers exactly the number of series the app is allowed to draw at once — beyond that, extra categories fold into “Other” rather than dipping into unvalidated colors.

We never paywall honesty

Some analyses are plan features. Honesty about your numbers never is. On every plan, every reported percentage passes the same base-size policy, low bases stay flagged, margins of error stay attached, and the pre-field guardrails stand between every user and a statistically invalid study. Upgrading buys you more analysis — it never buys anyone the ability to hide a caveat.

Keep digging

Methods

Choose the right research method

MaxDiff, conjoint, Van Westendorp, and more — organized by the decision you need to make.

Data quality

Data quality you can defend

Multi-signal fraud scoring, method-aware quality checks, and a flag-don't-delete posture.

Compare

How we stack up

See how Gotopico compares to other survey platforms on methods, analytics, and rigor.

Run research you can defend

Field a study with the guardrails on, and report numbers that carry their own evidence.