If you’re a home brewer, than you should probably be reading the Brülosophy blog. They have great write ups on brewing methods, new ingredients, and experiments (what they call ex-Beer-iments).
Their experiments usually consist of changing one variable in the brewing process, while keeping the rest of the variables of the brewing process fixed. Once the finished beers from the experiment are ready to consume, they perform a triangle test. A triangle test is one in which you pour two of one of the beers and one of the other into opaque cups. Then, a participant is asked to identify the one that is different.
If they can identify the one that is different, they give their perceptions of the two beers. Specifically, what they found different about the two and which one they prefer.
Say a group of 24 people participated. If all of them just guessed at random, then you would expect around 8 of them to guess correctly (1/3 of the 24, since you have a 1/3 chance at guessing correctly). Indeed, one can calculate the probability (using a binomial distribution) of several different outcomes.
Again, say that 24 people just selected at random. The probabilities of
- 8 or fewer people guessing correctly is 0.594,
- 9 or fewer people guessing correctly is 0.746,
- 10 or fewer people guessing correctly is 0.860,
- 11 or fewer people guessing correctly is 0.932, and
- 12 or fewer people guessing correctly is 0.972.
The last bullet point means that the probability of 13 or more people guessing correctly is 0.038. Since this drops below that magical probability of .05, this is where results would become “significant.” That means that the probability of that many people guessing correctly is so low, that we are more apt to believe that people aren’t just guessing, but actually can tell a difference.
Anyway (sorry that got long winded), Brülosophy‘s experiments very often come up insignificant to the surprise of many readers and home brewers. There are a lot of techniques and methods home brewers think of as “best practices” when it comes to home brewing that come up insignificant in the results.
Enter the oboe. Seth Godin wrote the following blog post called Does an orchestra need the oboe? on January 4, 2019:
For most pieces, for most audiences, most of the time, you wouldn’t miss it if it were gone.
But take away one more instrument, and then another, and pretty soon, we’ll stop listening.
The little fillips, the extraneous extras, the dispensable nice bits–they count for more than we know.
So, yes, take away a best practice of your home brewing, while maintaining all of the others, and it will go unmissed (insignificant in a triangle test).
But take away another, and another, and soon your beer won’t be easy drinking anymore. The Brülosophy experiments are very useful and informative, because it is nice to know which “best practices” can be individually ignored from time to time. However, I would take much care in making several changes in your brewing based on these results.