Link: https://emilyriederer.netlify.app/post/causal-design-patterns/
Graph:
Excerpt:
One antidote to this is true experimentation in which treatment is randomly assigned within the homogenous target population. Experimentation, particularly A/B tests, have become a mainstay of industry data science, so why observational causal inference matters?
Some situations you cannot test due to ethics or reputational risk
Even when you can experiment, understanding observational causal inference can help you better identify biases and design your experiments
Testing can be expensive. There are direct costs (e.g. testing a marketing promotion) of instituting a policy that might not be effective, implementation costs (e.g. having a tech team implement a new display), and opportunity costs (e.g. holding out a control group and not applying what you hope to be a profitable strategy as broadly as possible)2
Randomized experimentation is harder than it sounds! Sometimes experiments may not go as planned, but treating the results as observational data may help salvage some information value
Data collection can take time. We may want to read long-term endpoints like customer retention or attrition after many year. When we long to read an experiment that wasn’t launched three years ago, historical observational data can help us get a preliminary answer sooner
It’s not either-or but both-and. Due to the financial and temporal costs of experimentation, causal inference can also be a tool to help us better prioritize what experiments are worth running
Author: Emily Riederer
Publication Date: 30 January 2021
Publication Site: Emily Riederer