Applied econometrics is concerned with the interpretation of statistical results. The key word here is interpretation - the interpretation you use when you’re walking around an art gallery. Applied econometrics is a lot like walking through an art gallery, but where the various portraits and paintings correspond to statistical results.
I like this metaphor (simile?) for a number of reasons. For one, it makes clear that applied econometrics is not concerned with trying to convince someone of something. The “best” you can do is to explain to your audience how you interpret something, and provide the necessary background details so that the audience can form their own interpretation. Maybe they see it like you. Maybe they don’t. Again, it’s a lot like interpreting a piece of art.
Why are there multiple interpretations? It’s because there’s a general trade off in this type of work between questions that are important and those that can be credibly answered via statistical analysis. And so when you're tackling a somewhat important question, you have to make some decisions about how to do the analysis and how to think about how you’re doing the analysis. And the “results” will depend on these choices, and so people will rightly suggest that there were other reasonable choices that you could have made which would have lead to different results.
Causal Inference is typically a degree of maybe. Or, put another way — how maybe?
<aside>
“Most recent studies have relied predominantly on a quasi-experimental design, which leaves open the issue of selection bias among the treatment and control groups.”
</aside>
We’d like to be more nuanced in our questions than Are you identified?, and Are the results statistically significant?
Let’s start with the second question - Are the results statistically significant? Statistical significance, as we’ll explain later in great detail, is a thought exercise. The idea is the following. You have a population probability space, and a sample probability space. Your analysis can be thought of as the realization of a random variable defined on the sample probability space. The corresponding p-value denotes the probability (defined with respect to the sample probability space) under a null hypothesis that you observe a test-statistic as extreme as the one you did in your data. So the question of whether one’s result is statistically significant depends on how one conceptualizes the sample probability space. Until you’ve explained what that is, the question Are the results statistically significant? is a bit meaningless.
If was ever to write a test on this type of material, I’d ask students to bring a sketch pad to class, and sketch what comes to mind when you look at terms like $\mathbb{E}[\cdot \vert X]$.