Don’t confuse statistics with reality

Don’t confuse statistics with reality

Photo by Lukas from Pexels

Originally published 19 July 1993

As I went off to col­lege in 1954 my father hand­ed me a lit­tle orange book and said, “Read this. It will be as use­ful as any­thing you’ll learn in college.”

The book was Dar­rell Huf­f’s How to Lie with Sta­tis­tics, charm­ing­ly illus­trat­ed by Irv­ing Geis. In 1954, the year of pub­li­ca­tion, the book cost $2.95 in hard­cov­er. It is still avail­able in paper­back at $4.95, and as good a buy for the mon­ey as any­thing you’ll get in college.

My father, I should say, was a qual­i­ty-con­trol engi­neer, a pio­neer in that field, and a sta­tis­ti­cian to the soles of his feet. He knew how to make sta­tis­tics say any­thing he want­ed them to say. And he knew that in the sec­ond half of the 20th cen­tu­ry Amer­i­cans would be increas­ing­ly pro­pa­gan­dized by sta­tis­tics. He want­ed his kids to be prepared.

He also knew how to make sta­tis­tics tell the truth. Our house was filled with home­made gad­gets designed to demon­strate every known law of prob­a­bil­i­ty. We kids sat mes­mer­ized as hun­dreds of tiny balls rat­tled down between nails he had knocked into a ver­ti­cal board, arrang­ing them­selves into a bell-curve dis­tri­b­u­tion. As his col­lab­o­ra­tors, we tossed dice for hours, shuf­fled and drew cards, and plucked mul­ti­col­ored ceram­ic chips from box­es. All the while, he count­ed and cal­cu­lat­ed and graphed.

He loved graphs, and he loved to draw our atten­tion to graphs or dia­grams that dis­tort­ed data. The more egre­gious the dis­tor­tion, the more my father delight­ed in the igno­rance or deceit that pro­duced it.

I did not not know then, but I know now (because I just spent a hour in the library), that Dar­rell Huff was my father’s kind of guy. A prac­ti­cal man with wide-rang­ing inter­ests. A jack-of-all-trades who wrote month­ly arti­cles for Pop­u­lar Sci­ence on such things as “How to Build a Two-way Lawn Chair” or “Three Dozen Ways to Use Tape.”

He had a gift for explain­ing com­plex things sim­ply, and his expose of sta­tis­ti­cal lies was clear enough to keep the lit­tle orange book in print for almost 40 years. It is still ref­er­enced even in tech­ni­cal works on sta­tis­tics. Huff, now 80 years old [in 1993], is revis­ing for a new hard­cov­er edition.

I asked him if he ever imag­ined the book would last so long. He said: “I’ve been mar­ried for 50-some­thing years. Did­n’t expect the mar­riage to last that long. Did­n’t expect it not to. Nev­er thought about it at all. It was the same for the book.”

Huf­f’s lit­tle book showed that it is pos­si­ble to prove almost any­thing with sta­tis­tics if you are cre­ative about how you select your data, how you manip­u­late the data math­e­mat­i­cal­ly, and how you dis­play your results graph­i­cal­ly. He called his book a “primer in ways to use sta­tis­tics to deceive.” It was not, he insist­ed, a man­u­al for swindlers. “The crooks already know these tricks,” wrote Huff; “hon­est men must learn them in self-defense.”

Huff told me about a mur­der tri­al in Cal­i­for­nia that hinged on the prob­a­bil­i­ty that a thread found on the accused came from the vic­tim’s coat. The pros­e­cu­tor expound­ed sta­tis­tics. The accused sat qui­et­ly in his chair hold­ing a copy of How to Lie with Sta­tis­tics for judge and jury to see. “Did­n’t save him,” says Huff.

Sci­en­tists use sta­tis­tics all the time as a way of get­ting at the truth, and although they gen­er­al­ly try to be hon­est and objec­tive about what they do, the let­ters columns to sci­en­tif­ic jour­nals are filled with dis­putes about the legit­i­ma­cy of sta­tis­ti­cal inferences.

In fact, not even pro­fes­sion­al sta­tis­ti­cians agree upon the objec­tiv­i­ty of sta­tis­ti­cal infer­ence. The so-called Bayesians (sub­jec­tivists) and anti-Bayesians (objec­tivists) have been bat­tling it out with undi­min­ish­ing fer­vor for as long as I have been alive. And longer. The Rev. Thomas Bayes, who ini­ti­at­ed the debate, lived in the 1700s.

No one in sci­ence, of course, doubts the impor­tance and val­ue of sta­tis­tics as a way of teas­ing knowl­edge from sam­pled data, but there is less agree­ment as to whether sta­tis­tics is a branch of rhetoric or mathematics.

My father was right about Huf­f’s book. Thir­ty-nine years lat­er, I can­not read a sci­en­tif­ic paper with­out a healthy skep­ti­cism regard­ing sta­tis­ti­cal infer­ences, nor can I look at a graph or chart in Time or Newsweek with­out men­tal­ly com­par­ing them to Huf­f’s bag of tricks. Most of all, I am skep­ti­cal of the con­stant bar­rage of opin­ion polls that pur­port to show what Amer­i­cans feel about every­thing from pol­i­tics to sex.

Ross Per­ot set a new stan­dard of sta­tis­ti­cal pro­pa­gan­da with his flash­card graphs on tele­vi­sion dur­ing the last pres­i­den­tial cam­paign. Those graphs, with­out con­text or ade­quate quan­ti­ta­tive cues, proved noth­ing at all.

Which is not to say that Mr. Per­ot was delib­er­ate­ly lying with sta­tis­tics. No doubt every use of sta­tis­tics in sci­ence or pop­u­lar cul­ture con­tains a germ of truth. What I learned from Dar­rell Huf­f’s lit­tle orange book is not to con­fuse the germ with reality.


Dar­rell Huf­f’s clas­sic book on sta­tis­tics is still in print, and as valu­able as ever. ‑Ed.

Share this Musing: