How Does A/B Testing Turn a Product Change Into an Estimable Effect?
A/B testing turns a product change into an estimate with uncertainty. A useful report covers effect size, confidence, guardrails, randomization, and launch risk.
#statistics
72 postsWhy Not Run Many t-Tests? What Is ANOVA Protecting?
ANOVA first checks whether three or more group means differ overall, so you do not inflate false-positive risk by running many pairwise t tests.
Why Do Large-Sample Approximations Work, and When Do They Fail?
Large-sample normal approximation describes the behavior of estimators, not raw data. It is useful, but dependence, boundaries, and distribution shift can make it unreliable.
How Does Bayesian Inference Connect Prior, Data, and Posterior?
Bayesian inference updates uncertainty about an unknown parameter by combining prior belief with the likelihood from observed data, producing a posterior distribution.
What Do Bias, Variance, and Consistency Check in Point Estimation?
Bias checks whether an estimator is centered correctly, variance checks sampling fluctuation, MSE combines both, and consistency asks whether the estimator approaches truth as sample size grows.
When the Formula Distribution Is Unknown, How Does Bootstrap Estimate Uncertainty?
Bootstrap estimates uncertainty by resampling from the observed sample with replacement, rebuilding many sample-like datasets, and watching the statistic fluctuate.
Causal Inference Basics: Why Prediction Accuracy Does Not Mean Real Effect
Causal inference separates prediction from effect. A model can predict who will buy without proving that an intervention will make them buy.
How Do You Tell Goodness-of-Fit From Independence in Chi-Square Problems?
Chi-square tests compare observed counts with expected counts. First decide whether the problem is goodness-of-fit for one categorical variable or independence for two categorical variables.
When Should Bernoulli, Binomial, Normal, and Poisson Appear?
Distributions are names for data-generating situations, not formula cards. Learn when Bernoulli, Binomial, Poisson, and Normal distributions fit a problem.
How Do You Write Confidence Intervals Without Only Memorizing Bounds?
A confidence interval puts a point estimate back inside sampling fluctuation. Computing bounds is only the first step; you also need to explain standard error, critical values, and coverage.
When You See a Dataset, What Statistics Should You Check First?
Data type determines the statistical tools you can use. Start with categorical, numeric, count, and time-ordered data, then choose summaries that fit the question.
How Does the Delta Method Estimate Uncertainty for F1 and Ratio Metrics?
The delta method transfers uncertainty through a smooth function: the local derivative expands or shrinks the estimator's original standard error.
What Makes an Estimator Good: Bias, Variance, or MSE?
An estimator is a rule for using samples to infer a population parameter. To judge whether it is good, look at bias, variance, and MSE together.
When a Mixed Problem Appears, How Do You Pick the Tool in 30 Seconds?
At the final review stage, train problem recognition: identify data type, unknown quantity, and decision goal before choosing a formula and writing a contextual conclusion.
What Do Expectation and Variance Mean in Exams and Model Evaluation?
Expectation describes long-run center; variance describes fluctuation. This post computes E[X], E[X^2], and Var(X), then connects them to average loss and model stability.
How Does Experimental Design Make Results Interpretable Rather Than Merely Correlated?
Experimental design decides whether a result can be interpreted. Randomization, control, blocking, replication, blinding, and pre-specified outcomes give inference a usable foundation.
How Does Fisher Information Tell You Whether a Parameter Is Stable?
Fisher information uses likelihood curvature to measure how well the data locate a parameter; larger information usually means a smaller standard error for the MLE.
Confidence Intervals Are More Than t-Tables: What Is the General Construction?
A confidence interval is built by defining the target estimate, describing its sampling error, and choosing a rule that turns uncertainty into a range.
How Does a GLM Choose Distributions and Link Functions by Data Type?
A generalized linear model starts from the response type, chooses a suitable distribution, and uses a link function to connect the mean to a linear predictor.
From H0 to p-Values, What Decision Is a Hypothesis Test Making?
A hypothesis test is a decision process under uncertainty: write H0/H1, choose alpha, compute a test statistic and p-value, then decide whether the data is strong enough to challenge H0.
How Do Estimation, Testing, Likelihood, and Bayes Fit on One Inference Map?
The inference map starts with the question type: point estimate, uncertainty interval, decision test, likelihood model comparison, Bayesian update, or resampling.
How Does the Likelihood Ratio Test Compare Nested Models?
The likelihood-ratio test compares the log likelihood of a restricted model with a full model; the usual chi-square reference only makes sense under nested-model and approximation conditions.
When OLS Assumptions Fail, How Can the Regression Line Still Be Used?
OLS is a useful baseline, but coefficient interpretation, inference, prediction, and diagnosis depend on assumptions about linearity, errors, independence, and variance.
How Does Logistic Regression Move From Probability to Thresholds and Error Costs?
Logistic regression estimates probabilities first. Classification decisions come later, when thresholds turn those probabilities into actions under real error costs.
Why Should Classification Start With Log Odds?
Logistic regression connects a linear score to a probability between 0 and 1. Understanding odds, log odds, and odds ratios prevents wrong coefficient interpretations.
Why Does MAP Turn Priors Into Regularization?
MAP maximizes the posterior. After taking logs, the prior becomes a penalty term, which connects Bayesian estimation to L1, L2, and regularized ML objectives.
How Do Matching and Weighting Make Observational Data More Experiment-Like?
Matching and weighting do not turn observational data into a true experiment. They try to make treatment and control comparable on observed variables.
Why Does MLE Ask Which Parameter Most Likely Generated the Data?
MLE fixes the observed data and compares which parameter values make that data most plausible; log likelihood turns products into sums and connects directly to negative log loss.
Why Does the Method of Moments Match Sample Moments to Population Moments?
Method of Moments matches sample moments to theoretical population moments, then solves for parameters. It is not always the most efficient method, but it builds the first intuition for parameter estimation.
Missing Data Is Not Just Blank Cells: How Does It Distort Statistics and Models?
Missing data can change representativeness, bias estimates, and mislead ML systems. The first question is why the data are missing.
How Do You Write an ML/AI Evaluation Report That Is More Than a Leaderboard Score?
A useful ML/AI evaluation report turns statistical evidence into a decision: ship, stage, roll back, or run more experiments.
What Do Residuals, Outliers, and Leverage Reveal About Model Failure?
Model diagnostics turn fitted errors into evidence: residual patterns, outliers, leverage, and influential points reveal how a model fails.
How Does Multivariate Analysis Organize Features That Move Together?
Multivariate analysis looks at features together. Covariance, correlation, and PCA reveal shared directions that univariate summaries miss.
What Kind of Optimal Test Is the Neyman-Pearson View About?
The Neyman-Pearson view treats a test as a decision rule: under a fixed Type I error rate alpha, choose the rejection region with the highest power.
What Assumptions Do Nonparametric Methods Relax, and What Do They Cost?
Nonparametric methods are not assumption-free. They relax fixed distributional forms, often gaining flexibility while paying in efficiency, interpretation, or overfitting risk.
How Should You Analyze NTU IM 114-115 Statistics Papers Without Memorizing Answers?
Past papers train question-analysis discipline, not fortune-telling. Each problem should return to data type, unknown quantity, statistical tool, calculation path, and contextual conclusion.
How Do You Avoid Missing Cells in Joint Distribution and PMF Transformations?
Joint PMF problems require listing every cell. Marginalization, conditional probability, and variable transformations are all sums or regroupings of the original cells.
Conditional Probability, Independence, and Bayes: What Viewpoint Is the Problem Switching?
Probability problems are often hard because the viewpoint changes. Define events first, then distinguish conditioning, independence, mutual exclusivity, and Bayes' rule.
How Do Samples, Statistics, and Sampling Distributions Differ?
A sample is the data, a statistic is a function of the sample, and a sampling distribution is the distribution of that statistic under repeated sampling.
How Do PMF, PDF, and CDF Turn Probability Into Computation?
Random variables turn uncertain outcomes into numbers. PMF, PDF, and CDF then let you compute discrete probabilities, continuous interval probabilities, thresholds, and model-score distributions.
How Should coef, SE, t, F, and R-Squared Be Read Together?
A regression table is not a p-value list: coef, SE, t, F, and R-squared answer effect size, uncertainty, single-coefficient tests, overall model signal, and in-sample explanation.
Why Do Ridge, Lasso, and Weight Decay Make Models More Stable?
Regularization adds a preference against extreme parameters. Ridge, Lasso, and weight decay trade some training fit for a model that generalizes more reliably.
How Can Statistics and ML Evaluation Be Rerun to Reach the Same Conclusion?
A reproducible workflow preserves the evidence chain from data to conclusion. Results need data versions, code, seeds, environment, metrics, and raw outputs.
Why Can a Sample Say Something About a Population or Model?
Sampling makes sample statistics fluctuate, and standard error describes that fluctuation. This post separates SD, SE, sampling distributions, and CLT, then connects them to benchmark uncertainty.
How Do Sampling Distributions Become Exam-Ready Reasoning?
A sampling distribution describes how a statistic fluctuates under repeated sampling. Means, proportions, and variances each connect to common distributions used in intervals and tests.
After 53 Posts, How Do You Connect Statistics to ML, Causality, and Mathematical Statistics?
The series does not finish all of statistics. It gives beginners a working map for exams, ML/AI evaluation, causality, Bayesian thinking, time series, and mathematical statistics.
How Does One Regression Line Become Prediction, Interpretation, and Error?
Simple linear regression uses one X to describe the average change in Y. Slope, intercept, residuals, and squared error form the smallest supervised learning model.
How Does Monte Carlo Use Repeated Simulation to Answer Hard Statistical Questions?
Monte Carlo repeats a data-generating process many times so sampling variation, power, coverage, and evaluation instability become visible.
Where Should You Start Statistics If You Need Exams and ML/AI?
Do not start statistics exam prep by memorizing formulas. Start with the sequence of data, probability, sampling, inference, regression, then connect those ideas to model evaluation, A/B testing, and uncertainty in ML/AI.
Why Should Time-Series Data Not Be Randomly Split?
Time-series data have order. Random splits can leak future information into training and make forecasting or monitoring results look better than they are.
Which Test Fits a Two-Group Mean or Proportion Difference?
Two-group comparisons start by classifying the outcome and the design: numeric or binary, independent or paired. That choice determines the standard error, test statistic, and conclusion.
How Does Variable Selection Avoid Memorizing the Training Data?
Variable selection is not only about choosing predictors. It is about avoiding noisy training-set wins that do not generalize.
Statistics Is Not Formula Memorization: What Is It Deciding?
The core of statistics is judgment: describe data, estimate unknowns, compare differences, inspect associations, and make decisions under uncertainty.
Stanford CS109 Lecture 1 | What is Probability?: List outcomes first; only then assign probabilities to events.
List outcomes first; only then assign probabilities to events.
Stanford CS109 Lecture 2 | Conditional Probability: A condition restricts the sample space to outcomes still compatible with the evidence.
A condition restricts the sample space to outcomes still compatible with the evidence.
Stanford CS109 Lecture 3 | Bayes Theorem: Bayes’ theorem turns an easier generative direction into the inferential direction we need.
Bayes’ theorem turns an easier generative direction into the inferential direction we need.
Stanford CS109 Lecture 4 | Counting and Combinatorics: Decide whether order matters and repetition is allowed before choosing a formula.
Decide whether order matters and repetition is allowed before choosing a formula.
Stanford CS109 Lecture 5 | Random Variables and Expectation: A random variable maps outcomes to numbers; expectation is a weighted average, not necessarily an attainable value.
A random variable maps outcomes to numbers; expectation is a weighted average, not necessarily an attainable value.
Stanford CS109 Lecture 6 | Moments: Expectation, LOTUS, and linearity
Expectation compresses a distribution into a weighted average; LOTUS handles transformed values, while linearity makes sums tractable even without independence.
Stanford CS109 Lecture 7 | Variance and Poisson: From spread to rare-event counts
Variance describes a random variable's spread; Poisson models counts in a fixed interval and approximates a large-n, small-p binomial.
Stanford CS109 Lecture 8 | Continuous Random Variables: PDFs, CDFs, Uniform, and Exponential
A continuous variable assigns zero probability to a point and area to intervals; CDFs, Uniform, Exponential, and memorylessness build on that distinction.
Stanford CS109 Lecture 9 | Normal Distribution: Standardization, Phi, and continuity correction
Standardization maps Normal variables to Z; Phi, linear transforms, and continuity correction turn intervals and large binomials into computable probabilities.
Stanford CS109 Lecture 10 | Probabilistic Models: Joints, marginals, independence, and Bayes
A joint distribution retains the full relationship among variables; marginals, conditionals, independence, and Bayes extract different answers from it.
Stanford CS109 Lecture 11 | Inference: Prior times likelihood, then normalize
Inference multiplies each hidden-variable prior by an observation likelihood and normalizes; the same loop handles repeated evidence and discretized continuous beliefs.
Stanford CS109 Lecture 12 | General Inference: Bayesian networks, sampling, and rare evidence
A Bayesian network factorizes a huge joint through conditional independence; ancestral sampling generates joint samples, and rejection sampling filters them into a conditional.
Stanford CS109 Lecture 13 | Multinomial: Category counts, bag of words, and log probability
The Multinomial extends two-category Binomial counts to many categories; the same PMF models documents as word counts for Bayesian authorship with log-scores.
Stanford CS109 Lecture 14 | Beta: Turn an unknown probability into an updatable random variable
A Beta distribution represents full belief about an unknown success rate; success/failure data updates two parameters for posteriors, smoothing, and Thompson-sampling decisions.
Stanford CS109 Lecture 15 | Adding Random Variables and the Central Limit Theorem
A few independent sums have closed forms; general IID sums become approximately Normal under the CLT, with continuity correction for discrete sums.
Stanford CS109 Lecture 16 | Bootstrapping: Sampling statistics, error bars, and p-values
The bootstrap treats a sample histogram as a population proxy, resampling with replacement to approximate a statistic's sampling distribution, error bar, or null p-value.
Stanford CS109 Lecture 17 | Algorithmic Analysis: Conditional expectation, indicators, and recursion
Expected cost in randomized code can be conditioned on the first random choice; counting problems become indicator sums, often avoiding the full distribution entirely.
Stanford CS109 Lecture 18 | Information Theory: Surprise, entropy, information gain, and KL
Surprise turns rare events into bits; entropy is expected surprise, information gain selects uncertainty-reducing questions, and KL measures excess cost from a model distribution.
Stanford CS109 Lecture 19 | Maximum Likelihood Estimation: Hold data fixed and optimize the parameter
MLE fixes observed data and optimizes parameters; log-likelihood turns products into sums, but a maximum can also lie on a boundary.