Brian Kernighan on Millions, Billions, Zillions

KernighanNumbers are often intimidating, confusing, and even deliberately deceptive—especially when they are really big. The media loves to report on millions, billions, and trillions, but frequently makes basic mistakes or presents such numbers in misleading ways. And misunderstanding numbers can have serious consequences, since they can deceive us in many of our most important decisions, including how to vote, what to buy, and whether to make a financial investment. In this short, accessible, enlightening, and entertaining book, leading computer scientist Brian Kernighan teaches anyone—even diehard math-phobes—how to demystify the numbers that assault us every day. Giving you the simple tools you need to avoid being fooled by dubious numbers, Millions, Billions, Zillions is an essential survival guide for a world drowning in big—and often bad—data.

Why is it so important to be able to spot “bad statistics?”

We use statistical estimates all the time to decide where to invest, or what to buy, or what politicians to believe. Does a college education pay off financially? Is marijuana safer than alcohol? What brands of cars are most reliable? Do guns make society more dangerous? We make major personal and societal decisions about such topics, based on numbers that might be wrong or biased or cherry-picked. The better the statistics, the more accurately we can make good decisions based on them.

Can you give a recent example of numbers being presented in the media in a misleading way?

“No safe level of alcohol, new study concludes.” There were quite a few variants of this headline in late August. There’s no doubt whatsoever that heavy drinking is bad for you, but this study was actually a meta-analysis that combined the results of nearly 700 studies covering millions of people.  By combining results, it concluded that there was a tiny increase in risk in going from zero drinks a day to one drink, and more risk for higher numbers. But the result is based on correlation, not necessarily causation, and ignores potentially related factors like smoking, occupational hazards, and who knows what else. Fortunately, quite a few news stories pointed out flaws in the study’s conclusion.  To quote from an excellent review at the New York Times, “[The study] found that, over all, harms increased with each additional drink per day, and that the overall harms were lowest at zero. That’s how you get the headlines.”

What is an example of how a person could spot potential errors in big numbers?

One of the most effective techniques for dealing with big numbers is to ask, “How would that affect me personally?” For example, a few months ago a news story said that a proposed bill in California would offer free medical care for every resident, at a cost of $330 million per year. The population of California is nearly 40 million, so each person’s share of the cost would be less than $10. Sounds like a real bargain, doesn’t it? Given what we know about the endlessly rising costs of health care, it can’t possibly be right. In fact, the story was subsequently corrected; the cost of the bill would be $330 *billion* dollars, so each person’s share would be more like $10,000. Asking “What’s my share?” is a good way to assess big numbers.

In your book you talk about Little’s Law. Can you please describe it and explain why it’s useful?

Little’s Law is a kind of conservation law that can help you assess the accuracy of statements like “every week, 10,000 Americans turn 65.” Little’s Law describes the relationship between the time period (every week), the number of things involved (10,000 Americans), and the event (turning 65). Suppose there are 320 million Americans, each of whom is born, lives to age 80, then dies. Then 4 million people are born each year, 4 million die, and in fact there are 4 million at any particular age. Now divide by 365 days in a year, to see that about 11,000 people turn 65 on any particular day. So the original statement can’t be right—it should have said “per day,” not “per week.” Of course this ignores birth rate, life expectancy, and immigration, but Little’s Law is plenty good enough for spotting significant errors, like using weeks instead of days.

Is presenting numbers in ways designed to mislead more prevalent in the era of “alternative facts” than in the past?

I don’t know whether deceptive presentations are more prevalent today than they might have been, say, 20 years ago, but it’s not hard to find presentations that could mislead someone who isn’t paying attention. The technology for producing deceptive graphs and charts is better than it used to be, and social media makes it all too easy to spread them rapidly and widely.

Brian W. Kernighan is professor of computer science at Princeton University. His many books include Understanding the Digital World: What You Need to Know about Computers, the Internet, Privacy, and Security. He lives in Princeton, New Jersey.

Remembering Luigi Cavalli-Sforza, pioneer in population genetics

Luigi Cavalli-Sforza, a pioneer in using genetic information to help trace human evolution, history and patterns of migration, passed away on August 31 at the age of 96. Hailed as a breakthrough in the understanding of human evolution, his book, The History and Geography of Human Genes offers the first full-scale reconstruction of where human populations originated and the paths by which they spread throughout the world. It remains among the most influential of all PUP publications; American Journal of Human Biology called it “A crowning achievement, a compendium of a career’s work, and a sourcebook for years to come. . . . a landmark publication, a standard by which work in this field must be judged in the future.”

From the New York Times:

Millions of people in recent years have sent off samples of their saliva to DNA-testing companies like 23andMe and Ancestry.com hoping to find out where their forebears came from and whether they have mystery relatives in some distant land, or even around the corner.

The trend itself can be traced to an Italian physician and geneticist, Luigi Luca Cavalli-Sforza, who died on Aug. 31 at his home in Belluno, Italy, at 96. He laid the foundation for such testing, having honed his skills more than 60 years ago using blood types and 300 years of church records to study heredity in the villagers of his own country.

Dr. Cavalli-Sforza was a pioneer in using genetic information to help trace human evolution, history and patterns of migration. The founder of a field that he called genetic geography, he was renowned for synthesizing information from diverse disciplines — genetics, archaeology, linguistics, anthropology and statistics — to explain how human populations fanned out over the earth from their original home in Africa.

Stanford Medicine News Center chronicles Cavalli-Sforza’s work creating the field of genetic geography, which, according to Jarad Diamond, “demolish[ed] scientists’ attempts to classify human populations into races in the same way that they classify birds and other species into races.”

He is survived by his sons Matteo, Francesco and Luca Tommaso Cavalli-Sforza, and by his daughter Violetta Cavalli-Sforza.

Kip Thorne & Roger Blandford on Modern Classical Physics

PhysicsThis first-year, graduate-level text and reference book covers the fundamental concepts and twenty-first-century applications of six major areas of classical physics that every masters- or PhD-level physicist should be exposed to, but often isn’t: statistical physics, optics (waves of all sorts), elastodynamics, fluid mechanics, plasma physics, and special and general relativity and cosmology. Growing out of a full-year course that the eminent researchers Kip S. Thorne, winner of the 2017 Nobel Prize in Physics, and Roger D. Blandford taught at Caltech for almost three decades, this book is designed to broaden the training of physicists. Its six main topical sections are also designed so they can be used in separate courses, and the book provides an invaluable reference for researchers.

This book emerged from a course you both began teaching nearly 4 decades ago. What drove you to create the course, and ultimately to write this book?

KST: We were unhappy with the narrowness of physics graduate education in the United States. We believed that every masters-level or PhD physicist should be familiar with the basic concepts of all the major branches of classical physics and should have some experience applying them to real world phenomena. But there was no obvious route to achieve this, so we created our course.

RDB: Of course we had much encouragement from colleagues who helped us teach it and students who gave us invaluable feedback on the content.

The title indicates that the book is a “modern” approach to classical physics (which emphasizes physical phenomena at macroscopic scales). What specifically is “modern” in your book’s approach to this subject?

KST: Classical-physics ideas and tools are used extensively today in research areas as diverse as astrophysics, high-precision experimental physics, optical physics, biophysics, controlled fusion, aerodynamics, computer simulations, etc. Our book draws applications from all these modern topics and many more. Also, these modern applications have led to powerful new viewpoints on the fundamental concepts of classical physics, viewpoints that we elucidate—for example, quantum mechanical viewpoints and language for purely classical mode-mode coupling in nonlinear optics and in nonlinear plasma physics.

Why do you feel that it is so important for readers to become more familiar with classical physics, beyond what they may have been introduced to already?

KST: In their undergraduate and graduate level education, most physicists have been exposed to classical mechanics, electromagnetic theory, elementary thermodynamics, and little classical physics beyond this. But in their subsequent careers, most physicists discover that they need an understanding of other areas of classical physics (and this book is a vehicle for that).

In many cases they may not even be aware of their need. They encounter problems in their research or in R&D where powerful solutions could be imported from other areas of classical physics, if only they were aware of those other areas. An example from my career: in the 1970s, when trying to understand recoil of a binary star as it emits gravitational waves, I, like many relativity physicists before me, got terribly confused. Then my graduate student, Bill Burke—who was more broadly educated than I—said “we can resolve the confusion by adopting techniques that are used to analyze boundary layers in fluid flows around bodies with complicated shapes.” Those techniques (matched asymptotic expansions), indeed, did the job, and through Bill, they were imported from fluid mechanics into relativity.

RDB: Yes. To give a second example, when I was thinking about ways to accelerate cosmic rays, I recalled graduate lectures on stellar dynamics and found just the tools I needed.

You also mention in the book that geometry is a deep theme and important connector of ideas. Could you explain your perspective, and how geometry is used thematically throughout the book?

KST: The essential point is that, although coordinates are a powerful, and sometimes essential, tool in many calculations, the fundamental laws of physics can be expressed without the aid of coordinates; and, indeed, their coordinate-free expressions are generally elegant and exceedingly powerful. By learning to think about the laws in coordinate-free (geometric) language, a physicist acquires great power. For example, when one searches for new physical laws, requiring that they be geometric (coordinate-free) constrains enormously the forms that they may take. And in many practical computations (for example, of the relativistic Doppler shift), a geometric route to the solution can be faster and much more insightful than one that uses coordinates. Our book is infused with this.

RDB: We are especially keen on presenting these fundamental laws in a manner which makes explicit the geometrically formulated conservation laws for mass, momentum, energy, etc. It turns out that this is often a good starting point when one wants to solve these equations numerically. But ultimately, a coordinate system must be introduced to execute the calculations and interpret the output.

One of the areas of application that you cover in the book is cosmology, an area of research that has undergone a revolution over the past few decades. What are some of the most transformative discoveries in the field’s recent history? How does classical physics serve to underpin our modern understanding of how the universe formed and is evolving? What are some of the mysteries that continue to challenge scientists in the field of cosmology?   

RDB: There have indeed been great strides in understanding the large scale structure and evolution of the universe, and there is good observational support for a comparatively simple description. Cosmologists have found that 26 percent of the energy density in the contemporary, smoothed-out universe is in the form of “dark matter,” which only seems to interact through its gravity. Meanwhile, 69 percent is associated with a “cosmological constant,” as first introduced by Einstein and which causes the universe to accelerate. The remaining five percent is the normal baryonic matter which we once thought accounted for essentially all of the universe. The actual structure that we observe appears to be derived from almost scale-free statistically simple, random fluctuations just as expected from an early time known as inflation. Fleshing out the details of this description is almost entirely an exercise in classical physics. Even if this description is validated by future observations, much remains to be understood, including the nature of dark matter and the cosmological constant, what fixes the normal matter density, and the great metaphysical question of what lies beyond the spacetime neighborhood that we can observe directly.

KST: Remarkably, in fleshing out the details in the last chapter of our book, we utilize classical-physics concepts and results from every one of the other chapters. ALL of classical physics feeds into cosmology!

The revolution in cosmology that you describe depends upon many very detailed observations using telescopes operating throughout the entire electromagnetic spectrum and beyond. How do you deal with this in the book?

RDB: We make no attempt to describe the rich observational and experimental evidence, referring the reader to many excellent texts on cosmology that describe these in detail. However, we do describe some of the principles that underlie the design and operation of the radio and optical telescopes that bring us cosmological data.

There is has also been a lot of excitement regarding the recent observation by LIGO of gravitational waves caused by merging black holes. How is this subject covered in the book, and how, briefly, are some of the concepts of classical physics elucidated in your description of this cutting-edge research area?   

KST: LIGO’s gravitational wave detectors rely on an amazingly wide range of classical physics concepts and tools, so time and again we draw on LIGO for illustrations. The theory of random processes, spectral densities, the fluctuation-dissipation theorem, the Fokker-Planck equation; shot noise, thermal noise, thermoelastic noise, optimal filters for extracting weak signals from noise; paraxial optics, Gaussian beams, the theory of coherence, squeezed light, interferometry, laser physics; the interaction of gravitational waves with light and with matter; the subtle issue of the conservation or non conservation of energy in general relativity—all these and more are illustrated by LIGO in our book.

What are some of the classical physics phenomena in every day life that you are surprised more people do not fully understand—whether they are lay people, students, or scientists?

KST: Does water going down a drain really have a strong preference for clockwise in the northern hemisphere and counterclockwise in the south? How strong? What happens as you cross the equator? How are ocean waves produced? Why do stars twinkle in the night sky, and why doesn’t Jupiter twinkle? How does a hologram work? How much can solid objects be stretched before they break, and why are there such huge differences from one type of solid (for example thin wire) to another (a rubber band)?

RDB: I agree and have to add that I am regularly humbled by some every day phenomenon that I cannot explain or for which I have carried around for years a fallacious explanation. There is, rightly, a lot of focus right now on climate change, energy, hurricanes, earthquakes, and so on. We hear about them every day. We physicists need to shore up our understanding and do a better job of communicating this.

Do you believe that some of your intended readers might be surprised to discover the deep relevance of classical physics to certain subject areas?

KST: In subjects that physicists think of as purely quantum, classical ideas and classical computational techniques can often be powerful. Condensed matter physics is an excellent example—and accordingly, our book includes a huge number of condensed-matter topics. Examples are Bose-Einstein condensates, the van der Waals gas, and the Ising model for ferromagnetism.

RDB: Conversely, quantum mechanical techniques are often used to simplify purely classical problems, for example in optics.

Writing a book is always an intellectual journey. In the preparation of this tremendously wide-ranging book, what were some of the most interesting things you learned along the way?

KST: How very rich and fascinating is the world of classical physics—far more so than we thought in 1980 when we embarked on this venture. And then there are the new inventions, discoveries, and phenomena that did not exist in 1980 but were so important or mind-boggling that we could not resist including them in our book. For example, optical-frequency combs and the phase-locked lasers that underlie them, Bose-Einstein condensates, the collapse of the World Trade Center buildings on 9/11/01, the discovery of gravitational waves and the techniques that made it possible, laser fusion, and our view of the universe at large.

Kip S. Thorne is the Feynman Professor Emeritus of Theoretical Physics at Caltech. His books include Gravitation and Black Holes and Time Warps. Roger D. Blandford is the Luke Blossom Professor of Physics and the founding director of the Kavli Institute of Particle Astrophysics and Cosmology at Stanford University. Both are members of the National Academy of Sciences.