I get asked by a lot of clients if their one measure of an event is sufficient, or if they need more. And although the answer is always, “it depends”, as a Bayesian statistician I also like to say, “if you give me more and different ways to measure the same event, you’re actually giving me more evidence, and then I can integrate that and come to a better, more informed conclusion.”
Why, you might ask, would multiple lines of evidence make a conclusion more precise? Well, read on and find out.
Measurement Error and Cookies
To me, cooking is a great way of introducing folks to measurement error. Most of us are familiar, even if we aren’t great chefs, with liquid and dry measuring cups. Even if you exclusively make Hamburger Helper (no judgment here!) you still need to know how to measure ingredients.
In the cup in the picture above I’ve measured about 1 cup of flour. Yeah, about 1 cup. Now, clearly, that’s not actually 1 cup. It’s not 1 cup tapped down. It’s also not 1 cup measured above the rim and leveled with a butter knife. It’s just about 1 cup. Importantly, that was literally me, scooping up a bunch of flour with a 1 cup measuring cup.
It’s really sloppy. It actually made a bit of a mess. And for the batch of cookies I’m making, it probably doesn’t make a big difference (although, honestly, I’d go back and fill it up some more, and level it off, b/c that’s how I was trained in the kitchen).
But could I really call that 1 cup of flour? As a scratch kitchen cook that makes cookies on occasion I could probably get away with it — I personally won’t taste the difference (others in the house might…but they’d only care if I skimped the chocolate chips by that amount).
So, clearly, that’s not 1 cup of flour. We can objectively see that. But I can also tell you that it’s really, really hard getting a perfect 1 cup of all purpose flour. So, I’m good with saying, yeah, it’s about 1 cup — it’s not, but it’s close enough — there’s some degree of measurement error
I want another way to say that I do or don’t have 1 cup of flour. I got it — I’ll weigh it, surely someone knows how much 1 cup of all purpose flour weighs, right? Right?
I found an authoritative source — King Arthur Flour says that 1 cup of all purpose flour weighs 120 grams. Equipped with this knowledge, I pulled out my kitchen balance (or scale). I zeroed it with an empty 1 cup measuring cup, then did my haphazard “close enough to 1 cup” scoop. And in the picture to the right, you can see the result.
109 grams. That’s not 120 grams. King Arthur is an authoritative source, and they have no reason to change what 1 cup of all purpose flour weighs just so that I’m right or wrong (I doubt they even know of me).
So What Exactly Is Measurement Error?
We define measurement error as the difference between what we measure a quantity to be and the true value. In the case of my measuring cup, it’s an observer error — I’m the one who is not accurately measuring 1 cup of flour. And we can see I’m off by about 11 grams.
My measurement technique is clearly not great. If I were to keep doing this random scooping approach, I may be off by 5 grams next time, or maybe 15 grams — who knows. But the bottom line is, even if I were to do this really carefully, and use a butter knife to level the top, I’d still have some differences (although my kitchen balance/scale isn’t sensitive enough to report those).
So here we have the tale of two measuring devices. In one case, I’m saying, “meh, it’s close enough to 1 cup, so for this recipe I’m calling it 1 cup.” So I pour it into the bowl.
Now, my wife who just asked me to measure the 1 cup might look at the bowl and say, “I’ve done this a lot, and that does not look like 1 cup. Are you sure it’s 1 cup?” (and yes, she’s said that to me a lot; I’m much better at grilling).
My wife has probable cause to suspect that I’m skimping on the flour. Based on her knowledge, training, and experience baking, she knows 1 cup of all purpose flour when she sees it in the mixing bowl (and really, when we make cookies, we tend to make at least 4 dozen — so my 1 cup example would actually be multiplied by 4 which is now starting to make a real textural difference for cookies).
So in this case, we have multiple measures of my 1 cup of flour. We have my “meh, close enough” measure (a lot of error), my wife’s eyes which seem to be extremely well calibrated for what a cup of flour looks like, and our kitchen balance, which appears to be highly accurate.
Because my wife and I both believe the balance is highly accurate we give it a greater weight than my eyes. And when I don’t want to pull out the balance, my wife’s eyes have far more weight of evidence than my eyes.
Now, if I say, “meh, it’s close enough”, and my wife says I’m skimping, and the balance says I’m skimping, I have a high degree of confidence that I’m skimping. Because of time (and the balance isn’t always available because it’s buried on some deep counter in a corner of the kitchen), if my wife says, “nope” I generally agree.
What If I Am More Conscientious About Measuring
Another good example of measurement error is to say, let’s say I’m gonna do my best. I really want to get an accurate 1 cup of all purpose flour. But let’s also imagine that the flour container isn’t full. Maybe it’s about a 1/4 full.
I don’t know about you, but I have a really hard time getting 1 cup out of our containers when it’s about 1/4 full. I scoop, and I get 1/2 cup — not good enough.
So, I put the cup down, I try to do the paper funnel trick and pour it into my 1 cup measuring cup, but I’m trying to not waste and spill a bunch of flour.
I pour and stop. Just short of 1 cup (about like above).
So I tip the container again and I get really close — not over the top of the cup; but I want to level it off, so I pour some more in. Now the flour is over the top of the cup.
I could stop here and say, “Yep, it’s great”. I could also level the cup off with my finger (not as great as the butter knife), and my knuckle drags, and I get a huge valley in my flour….ugh. I could stop here and say, “I got 1 cup”. And if I did, I’d have some measurement error.
I could measure this on my kitchen balance and see how far off I am. Let’s say it says 115 grams. That’s not good enough, I’m off by 5 grams. So I pour it all back in and try again.
This time I get 117 grams — close, but I want 120 grams. So I pour it all in again, and try again — 116 grams. Okay, one last time, 122 grams — I’m calling it a day.
At any point I could’ve said any of these are close enough. These are all examples of measurement error.
