I just finished an intriguing book, Think Twice by Michael Mauboussin. The author is an investment strategist with a large firm in the financial services industry as well as Chairman of the Board of Trustees of the Santa Fe Institute. In the book he makes the case for better decision-making by “harnessing the power of counter intuition.” Sounds like a mouthful, but he raises several good points including one that deals with what is commonly referred to as “the wisdom of the crowd.” He points out that decisions made based on the collective wisdom of a group of individuals are frequently better than the judgment of an expert in the field of interest. When the decisions have probabilistic outcomes (as opposed to rule-based outcomes) and when there are a wide range of choices the wisdom of the crowd typically prevails.
Mauboussin cites the work of Scott Page (the professor, not the saxophonist) and replicates the following equation in his book related to the diversity prediction theorem:
Collective error = average individual error – prediction diversity
The average individual error is a reflection of ability, and the prediction diversity reflects how different the individual answers are. The equation basically states that if you have a group of incentivized, or otherwise motivated individuals, with diverse backgrounds, the crowd will always predict more accurately than the average person. Yes that is always, not sometimes. Of interest, the collective is often better than many of the experts in a particular field. In the words of the author, “This is not good news for experts and it is deeply humbling for all decision-makers.”
As I was mulling this over, a friend informed me he was participating in an office NCAA tournament pool. You are probably familiar with the idea, which is typically frowned upon by senior management, but goes something like this: Participants donate, say their lunch money, and then pick the teams they believe will win each of the 63 single elimination games played during three successive weekends in March and April. One of those teams will win six games in the tournament and be crowned the National Champion, which coincidentally will occur in Atlanta, Georgia, tonight.
My buddy said there were almost 30 people participating in the office where he works. Is there a place for the diversity prediction theorem here? We have an incentive (the lunch money), a diverse group of individuals and, likely, some ability in this group. I asked him to provide me with access to that sports’ authority, ESPN, where the groups’ picks were housed. Then, I walked through the following exercise:
- During the first two days of the tournament there were 32 games. Collectively ,who did this group of almost 30 souls think would win each of those games?
- Using the averages, 21 of 32 were called correctly and, not surprisingly, this 21 out of 32 was right in the middle of the pack for those who had donated their lunch money to the cause.
Over the next two days, another 16 games were played to determine the “Sweet Sixteen” and on the following weekend after 12 more games were played, the field was whittled down to the “Final Four.” Imagine my surprise when I walked through this exercise and found the collective wisdom of this group of participants scored better than 85% of the original participants through the end of the first two weekends. What is perhaps more impressive is how this isolated collective beat the pants off of several experts in the field. (Our good friends at ESPN post a celebrity and expert page for this tournament and, yes, even the President of the United States participates in this annual ritual.)
Granted, this is very small sample, and the results may be more related to chance than the wisdom of the crowd. But for a moment, think about this counter intuitive approach to decision-making.
Crowd sourcing has become prominent in several arenas. One example in the world of informatics is Kaggle. Toss your data analytics problem into the world of Kaggle, and if the incentives are appropriate, you could have several solutions to choose from.
What about closer to home? As nephrologists, we are often considered the experts and are frequently called upon to make a decision. Is there room for considering the wisdom of the crowd in our world?
In Mauboussin’s opinion, we would be wise to consider the collective when we face problems that have a wide range of outcomes and are probabilistic in nature. Clearly, this is not the case when we are considering how to deal with a potassium of 7, but what about the therapeutic regimen in an unusual case of glomerular disease? Or the best course to take in a complicated renal transplant patient? Perhaps, we would do well to “think twice” in circumstances such as these.
Can you think of circumstances in nephrology where the wisdom of the collective might be useful? If so, drop us a comment and join the conversation. Oh, and about that game tonight, the collective wisdom of my buddy’s office pool had Louisville facing Indiana in the Championship. The Hoosiers did not make it to Atlanta, but will the Cardinals prevail? Enjoy the game.
RG says
Terry,
Thanks for the book recommendation. Here is one for you: Quiet. By Susan Cain. The power of introverts in a world that can’t stop talking. She talks about leaders in business that are introverts, and why they are so successful. I wonder what category you would fall into. Another one is called “Extraordinary Popular Delusions and the madness of crowds”, by Charles Mackay. Sometimes crowds are irrational, like in the Tulip Bubble or the South Sea bubble. The collective can be wrong in major ways, but probably not most of the time. In college I did a paper on race track betting. We concluded exactly as you have, that the crowd gets it right most of the time. The money bet on horse A divided by the total amount of money bet on all the horses essentially equals the probability that the horse will win. There is a slight discrepancy in that people like to bet on the long shots as they get an adrenal surge out of the possibility of winning large amounts of money from a small bet. If you look at the index fund industry and Jack Bogles philosophy, indeed the crowd is right almost always, because the unmanaged index beats the experts almost always over long periods of time. Fascinating stuff.
RG
Terry Ketchersid, Vice President and Medical Officer at HITSG says
Thanks for the feedback RG. I have read Cain’s book and very much enjoyed it. I will keep my eyes open for Mackay’s as well.