BAR · Business Analysis · 6 practice questions
Correlation vs. Causation: Confounding and Selection Bias
A strong correlation or statistically significant regression coefficient does not establish causation. Follow a retailer's coupon-push analysis to check confounding, reverse causality, and selection before recommending action.
Try one first
Hint
Correlation ≠ causation. Strong r doesn't establish causal direction. Alternatives: direct causation, reverse causation, confounding (most common, e.g., season drives both temperature and sales), spurious correlation. Test causality with controls, experiments, domain knowledge.
Answer C. CORRELATION DOES NOT ESTABLISH CAUSATION, a foundational principle in statistical reasoning. While the strong correlation (r = 0.92) suggests a relationship, several alternative explanations exist: (1) DIRECT CAUSATION: temperature → sales (the analyst's hypothesis). (2) REVERSE CAUSATION: sales → temperature (implausible here, but logically possible in other scenarios). (3) CONFOUNDING VARIABLE: a third variable causes both. Most likely here: 'time of year' / 'season' simultaneously drives both temperature and ice cream demand (people consume more ice cream in summer regardless of any specific temperature reading; temperature is a feature of summer, not the only cause of consumption). (4) SPURIOUS / COINCIDENTAL: random correlation in limited data. (5) SELECTION/SAMPLE BIAS: the data may not represent the broader population. PROFESSIONAL APPROACH: before recommending action based on correlation, the analyst should: (a) consider domain knowledge about plausible causal pathways; (b) test for confounders (control for season, holiday effects, marketing spend); (c) perform regression with controls; (d) where possible, design experiments (A/B tests in different markets) to isolate causal effects; (e) look for natural experiments (sudden temperature anomalies, events that shift one variable but not the other). ICE CREAM EXAMPLE: temperature likely IS a cause of ice cream sales (consumers want cold treats when hot), but the marketing-focus recommendation should account for total demand drivers, not just temperature. Targeting hot regions might have diminishing returns if those regions are already saturated. KEY LESSON: 'correlation ≠ causation' is the most fundamental principle of causal inference. Always consider alternative explanations before drawing causal conclusions. Answer A is incorrect: r = 0.92 does NOT establish causation. Correlation alone, no matter how strong, doesn't address causality. Answer B is incorrect: high correlations are not 'typically spurious', high correlations often reflect real relationships, but they don't establish CAUSATION. Discounting high correlations as overfitting artifacts is wrong. Answer D is incorrect: temperature being a 'physical-science variable' doesn't automatically establish a causal role for sales. The causal mechanism must be tested independently.
Step by step
- State the observed association
A retailer finds that stores sending more mobile-app coupon pushes have higher weekly sales, with a positive, statistically significant regression coefficient. This supports an association, not proof that more pushes cause higher sales.
- Look for a common cause
Larger stores can have more app users and higher sales, creating more pushes and more sales without establishing an effect of pushes. Store size is a potential confounder because it can drive both variables.
- Check the causal direction
If strong sales lead managers to schedule more pushes for the following week, sales are helping cause pushes. That is reverse causality, not a common cause.
- Check how stores were selected
If stores chose whether to increase pushes, the groups may differ before the increase. Comparing those groups can reflect selection rather than the effect of additional pushes.
- Do not overread regression controls
Adding store size can address that measured confounder, but it does not automatically resolve reverse causality or selection. A significant push coefficient after controls still does not establish causation.
- Prefer a randomized comparison
Randomly assign similar stores to additional pushes or usual pushes during the same period. This provides a stronger basis for attributing a sales difference to the additional pushes.
- Match the conclusion to the design
For the observational analysis, report that more pushes are associated with higher sales. Do not describe the coefficient as the sales lift management will obtain by requiring more pushes.
Key points
- Remember: More pushes and higher sales together do not show that requiring more pushes will increase sales.
- A common cause explains both variables; reverse causality sends the causal direction backward.
- Controls address measured factors, not every possible explanation.
- Stores that choose more pushes may differ before any pushes occur.
How the exam traps you
- Calling a significant push coefficient a proven sales lift. Describe an association unless the study design supports a causal effect.
- Assuming store-size controls eliminate every alternative explanation. Check reverse causality and selection even after controlling for size.
- Treating stores that chose more pushes as a comparable treatment group. Prefer random assignment to more pushes or usual pushes during the same period.
Question 2
Hint
Focus on incremental performance using data not used to fit the model, rather than simple association.
Answer A. A high correlation and a higher training R-squared show in-sample association, not necessarily incremental forecasting usefulness. Predictive value is supported when model performance improves on data not used to fit the model. Here, the holdout RMSE worsens when clicks are added, so clicks do not add predictive value.
Why not B: This treats simple association as evidence of forecasting value. It fails because a standalone training correlation does not show incremental out-of-sample improvement over the current model.
Why not C: This rejects additional predictors simply because the current model already has predictors. It fails because another variable can add predictive value if validation performance improves.
Why not D: This relies on improved in-sample fit after adding a predictor. It fails because training R-squared can rise even when holdout prediction performance worsens.
Question 3
Hint
Prefer evidence that compares similar groups at the same time while limiting self-selection and timing effects.
Answer C. Random assignment creates comparable groups, and measuring both groups in the same month reduces the effect of timing differences. That gives the strongest support that any sales difference is due to the layout rather than other factors.
Why not A: This answer comes from assuming any side-by-side comparison is sufficient. It fails because stores that choose the new layout may already differ in management style, traffic, or customer mix.
Why not B: This answer comes from treating a before-and-after change as enough by itself. It fails because other changes over time, such as promotions, weather, or seasonality, could explain the difference.
Why not D: This answer comes from using a period with high sales and assuming the layout caused the increase. It fails because holiday demand changes sales even without the new layout, and there is no comparison group.
Question 4
Hint
Compare the population of interest with the records used before interpreting the observed relationship.
Answer B. The analysis is based only on customers who passed an intermediate screen: completing a trial and becoming paid users. Because trial completion varies by age, the observed relationship within paying subscribers may differ from the relationship for all trial users.
Why not A: This treats the issue as a measurement-timing problem, but age at signup is not the reason certain trial users are excluded from the analysis.
Why not C: This treats the issue as aggregation over time, but monthly summarization does not explain why the selected records differ from all trial users.
Why not D: This treats the issue as mixing different product groups, but the stated concern is that entry into the analyzed group varies with age.
Question 5
Hint
Ask whether the design identifies cause, or only estimates a relationship conditional on included variables.
Answer B. The regression coefficient supports an association conditional on the variables included in the model. Because acceptance was voluntary and unmeasured customer interest could affect both acceptance and revenue, the analysis does not establish causation. Statistical significance does not convert an observed relationship into a causal conclusion.
Why not A: This treats the controlled coefficient as an amount attributable to the offer. The regression controls only for measured variables, so voluntary acceptance and omitted factors can still explain the relationship.
Why not C: This assumes the accepted offer explains the higher revenue. The data leave open other explanations, including unmeasured interest in upcoming releases, so explanation by acceptance is stronger than the analysis supports.
Why not D: This mistake equates a statistically significant coefficient with a causal effect. Without random assignment or another causal identification strategy, the model estimates an association rather than proving what caused revenue differences.
Question 6
Hint
Check whether the reported statistic can ever be less than the minimum or greater than the maximum possible value.
Answer C. A Pearson correlation coefficient must fall between -1 and +1, inclusive. Values inside that interval, including -1.00, are possible. A reported correlation of 1.03 exceeds the maximum possible value, so it indicates an error.
Why not A: This choice comes from thinking large negative values are invalid. Correlation can be negative, and any value from -1 through +1 is allowable.
Why not B: This choice comes from assuming correlations must be near 0 or near 1 to be meaningful. The rule tested here is only whether the number falls within the allowable range.
Why not D: This choice comes from thinking the endpoints are excluded. They are included, so a perfect negative correlation of -1.00 is valid.
Common questions
Does a statistically significant regression coefficient prove causation?
No. A significant coefficient linking coupon pushes to sales supports an association, but confounding, reverse causality, or selection may explain it.
What is the difference between confounding and reverse causality?
Store size is a confounder if it drives both coupon pushes and sales. Reverse causality occurs if strong sales lead managers to schedule more pushes.
Why is random assignment better than voluntary participation?
Stores choosing more pushes may already differ from other stores. Random assignment creates comparable groups and provides stronger evidence about the effect of additional pushes.
Practice BAR like the real exam
The free ChatCPA simulator: real exam layout, timed testlets, starting with a question on this topic. No account needed to start.
Open the free simulator →More on Data analytics, regression analysis, and quantitative methods
- Calculate Predicted Cost Using Simple Linear Regression
- Adjusted R-Squared When Adding Regression Predictors
- Time-Series Forecasting: Match the Method to the Pattern
- Sensitivity Analysis vs Scenario Analysis: One Input or Many
All Data analytics, regression analysis, and quantitative methods practice →
Questions from the ChatCPA bank of 17,658 CPA exam questions, each with a written reason for every wrong answer. ChatCPA is built by Nicholas Miller, CPA (Oregon #14907). How these pages are made. Spot an error? Tell us.