You ran a survey. You have sales data. You want to know if a change actually made a difference. But which statistical test do you use? Here's a plain-English guide to the most common tests, when to use them, and how to interpret the results.
Statistical tests answer one question: is what I'm seeing real, or could it have happened by chance? They do this by calculating a p-value. If the p-value is below 0.05 (the standard threshold), the result is "statistically significant" which likely didn't happen by random chance.
Use when: You want to compare the average of two groups.
What you need: One categorical variable (group A vs group B) and one continuous variable (the thing you're measuring).
Result: If p < 0.05, the two groups are significantly different. The test also tells you the effect size (how big the difference is).
Use when: You want to compare averages across three or more groups.
Why not multiple t-tests: Running t-tests on every pair increases the chance of a false positive. ANOVA handles multiple groups in one test.
Result: If p < 0.05, at least one group is different from the others. Post-hoc tests tell you which specific groups differ.
Use when: You want to know if two categorical variables are related.
What you need: Two categorical variables with counts or frequencies.
Result: If p < 0.05, the two variables are not independent. There's a relationship between them.
Use when: You want to know if two continuous variables move together.
What you get: A correlation coefficient from -1 to 1. Close to 1 means strong positive relationship. Close to -1 means strong negative relationship. Close to 0 means no relationship.
Important: Correlation does not mean causation. Two things can be correlated without one causing the other.
Use when: You want to predict one variable from others, or understand how variables influence an outcome.
What you get: An equation that shows how each predictor influences the outcome, with statistical significance for each factor.
You don't need to memorize these rules. Queryly's Smart Test Selector analyzes your data and automatically recommends the right statistical test. Upload your file, describe what you want to compare, and the AI handles the rest.