There are some interesting examples in the "The Book of Why" [y] by Pearl and Mackenzie. This book is all about causal inference & has examples from things like drug trials where the analysis and inferences drawn from analysing experimental data depend upon what causal model you assume is generating the data, and if you pick an analysis that implicitly assumes the wrong underlying causal structure then your inferences drawn from the data may be wrong. There's a chapter about the historical scientific debate around if smoking causes lung cancer.
Andrew Gelman has an interesting review of the book [g], from the perspective of someone working within the statistical establishment that Pearl's book often critiques.
My own is what I call the framework rule. All discussions of software frameworks eventually degrade to first person pronouns and the word easy. Knowing that you can always predict your audience and their desires well in advance with extremely high confidence/precision. Easiness is an unmeasurable subjective concept whose second order consequences are always measurable and always cost more and which are likewise identifiable and predictable. That makes it similar to a fractal of Conways law but it applies more broadly.
This isn't a "statistics" problem exactly, but rather a data science problem, but... Personality tests are usually framed as psychological instruments, but the core problem is statistical. They assume responses are noisy observations of a stable latent variable, when in practice they’re samples from a context-dependent decision process. Respondents condition on incentives, infer what’s being rewarded, and optimize accordingly, so the data-generating process shifts with each setting. From a statistical perspective this breaks identifiability: multiple latent personalities can produce the same observed responses, and the same personality can produce different responses under different incentive structures. Because those incentives are unobserved or only weakly observed, the model is fundamentally misspecified. More samples don’t converge on truth; they converge on a biased estimate of a strategic equilibrium rather than the latent trait itself.
On paper, I'm a real collaborator, because that is what I think the test should reveal about me. In reality, I just don't wanna deal with collaboration and I'd rather work alone. Can this be measured? Probably not, but it does present an odd paradox for data scientists to solve.
It sounds like you're looking for problems in probability theory (rather than statistics). I don't have anything specific for you but you might have better luck searching for problems, puzzles, and examples in probability. For instance:
There are some interesting examples in the "The Book of Why" [y] by Pearl and Mackenzie. This book is all about causal inference & has examples from things like drug trials where the analysis and inferences drawn from analysing experimental data depend upon what causal model you assume is generating the data, and if you pick an analysis that implicitly assumes the wrong underlying causal structure then your inferences drawn from the data may be wrong. There's a chapter about the historical scientific debate around if smoking causes lung cancer.
Andrew Gelman has an interesting review of the book [g], from the perspective of someone working within the statistical establishment that Pearl's book often critiques.
[y] https://www.hachettebookgroup.com/titles/judea-pearl/the-boo...
[g] https://statmodeling.stat.columbia.edu/2019/01/08/book-pearl...
https://en.wikipedia.org/wiki/Pareto_principle
https://en.wikipedia.org/wiki/The_Mythical_Man-Month
https://en.wikipedia.org/wiki/Conway%27s_law
My own is what I call the framework rule. All discussions of software frameworks eventually degrade to first person pronouns and the word easy. Knowing that you can always predict your audience and their desires well in advance with extremely high confidence/precision. Easiness is an unmeasurable subjective concept whose second order consequences are always measurable and always cost more and which are likewise identifiable and predictable. That makes it similar to a fractal of Conways law but it applies more broadly.
This isn't a "statistics" problem exactly, but rather a data science problem, but... Personality tests are usually framed as psychological instruments, but the core problem is statistical. They assume responses are noisy observations of a stable latent variable, when in practice they’re samples from a context-dependent decision process. Respondents condition on incentives, infer what’s being rewarded, and optimize accordingly, so the data-generating process shifts with each setting. From a statistical perspective this breaks identifiability: multiple latent personalities can produce the same observed responses, and the same personality can produce different responses under different incentive structures. Because those incentives are unobserved or only weakly observed, the model is fundamentally misspecified. More samples don’t converge on truth; they converge on a biased estimate of a strategic equilibrium rather than the latent trait itself.
On paper, I'm a real collaborator, because that is what I think the test should reveal about me. In reality, I just don't wanna deal with collaboration and I'd rather work alone. Can this be measured? Probably not, but it does present an odd paradox for data scientists to solve.
It sounds like you're looking for problems in probability theory (rather than statistics). I don't have anything specific for you but you might have better luck searching for problems, puzzles, and examples in probability. For instance:
https://math.stackexchange.com/questions/2140493/counterintu...
Look into a concept called "The curse of dimensionality". https://en.wikipedia.org/wiki/Curse_of_dimensionality
Here's one I did at A-level: https://en.wikipedia.org/wiki/Bertrand_paradox_(probability)
Simpsons paradox and anscombe quartet are pretty good starters