Pariah Moonshine Part III: Pariah Groups, Prime Factorizations, and Points on Elliptic Curves

by Joshua Holden

This post originally appeared on The Aperiodical. We republish it here with permission. 

In Part I of this series of posts, I introduced the sporadic groups, finite groups of symmetries which aren’t the symmetries of any obvious categories of shapes. The sporadic groups in turn are classified into the Happy Family, headed by the Monster group, and the Pariahs. In Part II, I discussed Monstrous Moonshine, the connection between the Monster group and a type of function called a modular form. This in turn ties the Monster group, and with it the Happy Family, to elliptic curves, Fermat’s Last Theorem, and string theory, among other things. But until 2017, the Pariah groups remained stubbornly outside these connections.

In September 2017, John Duncan, Michael Mertens, and Ken Ono published a paper announcing a connection between the Pariah group known as the O’Nan group (after Michael O’Nan, who discovered it in 1976) and another modular form. Like Monstrous Moonshine, the new connection is through an infinite-dimensional shape which breaks up into finite-dimensional pieces. Also like Monstrous Moonshine, the modular form in question has a deep connection with elliptic curves. In this case, however, the connection is more subtle and leads through yet another set of important mathematical objects: the quadratic fields.

At play in the fields quadratic

What mathematicians call a field is a set of objects which are closed under addition, subtraction, multiplication, and division (except division by zero). The rational numbers form a field, and so do the real numbers and the complex numbers. The integers don’t form a field because they aren’t closed under division, and the positive real numbers don’t form a field because they aren’t closed under subtraction.  (It’s also possible to have fields of things that aren’t numbers, which are useful in lots of other situations; see Section 4.5 of The Mathematics of Secrets for a cryptographic example.)

A common way to make a new field is to take a known field and enlarge it a bit. For example, if you start with the real numbers and enlarge them by including the number i (the square root of -1), then you also have to include all of the imaginary numbers, which are multiples of i, and then all of the numbers which are real numbers plus imaginary numbers, which gets you the complex numbers. Or you could start with the rational numbers, include the square root of 2, and then you have to include the numbers that are rational multiples of the square root of 2, and then the numbers which are rational numbers plus the multiples of the square root of 2. Then you get to stop, because if you multiply two of those numbers you get

Holden

which is another number of the same form. Likewise, if you divide two numbers of this form, you can rationalize the denominator and get another number of the same form. We call the resulting field the rational numbers “adjoined with” the square root of 2. Fields which are obtained by starting with the rational numbers and adjoining the square root of a rational number (positive or negative) are called quadratic fields.

Identifying a quadratic field is almost, but not quite, as easy as identifying the square root you are adjoining. For instance, consider adjoining the square root of 8. The square root of 8 is twice the square root of 2, so if you adjoin the square root of 2 you get the square root of 8 for free. And since you can also divide by 2, if you adjoin the square root of 8 you get the square root of 2 for free. So these two square roots give you the same field.  For technical reasons, a quadratic field is identified by taking all of the integers whose square roots would give you that field, and picking out the integer D with the smallest absolute value that can be written in the form b2 – 4ac for integers a, b, and c.  (This is the same b2 – 4ac as in the quadratic formula.)  This number D is called the fundamental discriminant of the field. So, for example, 8 is the fundamental discriminant of the quadratic field we’ve been talking about, not 2, because 8 = 42 – 4 × 2 × 1, but 2 can’t be written in that form.

Prime suspects

After addition, subtraction, multiplication, and division, one of the really important things you can do with rational numbers is factor their numerators and denominators into primes. In fact, you can do it uniquely, aside from the order of the factors. If you have number in a quadratic field, you can still factor it into primes, but the primes might not be unique. For example, in the rational numbers adjoined with the square root of negative 5 we have

Holden

where 2, 5, 1 + √–5, and 1 – √–5 are all primes. You’ll have to trust me on that last part, since it’s not always obvious which numbers in a quadratic field are prime. Figures 1 and 2 show some small primes in the rational numbers adjoined with the square roots of negative 1 and negative 3, respectively, plotted as points in the complex plane.

Holden 3

Figure 1. Some small primes in the rational numbers adjoined with the square root of -1 (D = -4), plotted as points in the complex plane. By Wikimedia Commons User Georg-Johann.)

 

Holden

Figure 2. Some small primes in the rational numbers adjoined with the square root of -3 (D = -3), plotted as points in the complex plane. By Wikimedia Commons User Fropuff.)

We express this by saying the rational numbers have unique factorization, but not all quadratic fields do. The question of which quadratic fields have unique factorization is an important open problem in general. For negative fundamental discriminants, we know that D = ‑3, ‑4, ‑7, ‑8, ‑11, ‑19, ‑43, ‑67, ‑163 are the only such quadratic fields; an equivalent form of this was conjectured by Gauss but fully acceptable proofs were not given until 1966 by Alan Baker and 1967 by Harold Stark. For positive fundamental discriminants, Gauss conjectured that there were infinitely many quadratic fields with unique factorization but this is still unproved.

Furthermore, Gauss identified a number, called the class number, which in some sense measures how far from unique factorization a field is. If the class number is 1, the field has unique factorization, otherwise not. The rational numbers adjoined with the square root of negative 5 (D = -20) have class number 2, and therefore do not have unique factorization. Gauss also conjectured that the class number of a quadratic field went to infinity as its discriminant went to negative infinity; this was proved by Hans Heilbronn in 1934.

Moonshine with class (numbers)

What about Moonshine? Duncan, Mertens, and Ono proved that the O’Nan group was associated with the modular form

F(z) = e -8 π i z + 2 + 26752 e 6 π i z + 143376 e 8 π i z  + 8288256 e 14 π i z  + …

which has the property that the coefficient of e 2 |D| π I z  is related to the class number of the field with fundamental discriminant < 0.  Furthermore, looking at elements of the O’Nan group sometimes gives us very specific relationships between the coefficients and the class number.  For example, the O’Nan group includes a symmetry which is like a 180 degree rotation, in that if you do it twice you get back to where you started.  Using that symmetry, Duncan, Mertens, and Ono showed that for even D < -8, 16 always divides a(D)+24h(D), where a(D) is the coefficient of  e 2 |D| π i z  and h(D) is the class number of the field with fundamental discriminant D.  For the example D = -20 from above, a(D) = 798588584512 and h(D) = 2, and 16 does in fact divide 798588584512 + 48.  Similarly, other elements of the O’Nan group show that 9 always divides a(D)+24h(D) if D = 3k+2 for some integer k and that 5 and 7 always divide a(D)+24h(D) under other similar conditions on And 11 and 19 divide a(D)+24h(D) under (much) more complicated conditions related to points on an elliptic curve associated with each D, which brings us back nicely to the connection between Moonshine and elliptic curves.

How much Moonshine is out there?

Monstrous Moonshine showed that the Monster, and therefore the Happy Family, was related to modular forms and elliptic curves, as well as string theory. O’Nan Moonshine brings in two more sporadic groups, the O’Nan group and its subgroup the “first Janko group”. (Figure 3 shows the connections between the sporadic groups. “M” is the Monster group, “O’N” is the O’Nan group, and “J1” is the first Janko group.) It also connects the sporadic groups not just to modular forms and elliptic curves, but also to quadratic fields, primes, and class numbers. Furthermore, the modular form used in Monstrous Moonshine is “weight 0”, meaning that k = 0 in the definition of a modular form given in Part II. That ties this modular form very closely to elliptic curves.

Holden

Figure 3. Connections between the sporadic groups. Lines indicate that the lower group is a subgroup or a quotient of a subgroup of the upper group. “M” is the Monster group and “O’N” is the O’Nan group; the groups connected below the Monster group are the rest of the Happy Family. (By Wikimedia Commons User Drschawrz.)

The modular form in O’Nan Moonshine is “weight 3/2”. Weight 3/2 modular forms are less closely tied to elliptic curves, but are tied to yet more ideas in mathematical physics, like higher-dimensional generalizations of strings called “branes” and functions that might count the number of states that a black hole can be in. That still leaves four more pariah groups, and the smart money predicts that Moonshine connections will be found for them, too. But will they come from weight 0 modular forms, weight 3/2 modular forms, or yet another type of modular form with yet more connections? Stay tuned! Maybe someday soon there will be a Part IV.

Joshua Holden is professor of mathematics at the Rose-Hulman Institute of Technology. He is the author of The Mathematics of Secrets: Cryptography from Caesar Ciphers to Digital Encryption.

Pariah Moonshine Part II: For Whom the Moon Shines

by Joshua Holden

This post originally appeared on The Aperiodical. We republish it here with permission. 

HoldenI ended Part I with the observation that the Monster group was connected with the symmetries of a group sitting in 196883-dimensional space, whereas the number 196884 appeared as part of a function used in number theory, the study of the properties of whole numbers.  In particular, a mathematician named John McKay noticed the number as one of the coefficients of a modular form.  Modular forms also exhibit a type of symmetry, namely if F is a modular form then there is some number k for which

Figure 1

for every set of whole numbers a, b, c, and d such that adbc=1.  (There are also some conditions as the real part of z goes to infinity.)

Modular forms, elliptic curves, and Fermat’s Last Theorem

In 1954, Martin Eichler was studying modular forms and observing patterns in their coefficients.  For example, take the modular form

Figure 2

(I don’t know whether Eichler actually looked at this particular form, but he definitely looked at similar ones.)  The coefficients of this modular form seem to be related to the number of whole number solutions of the equation

y2 = x3 – 4 x2 + 16

This equation is an example of what is known as an elliptic curve, which is a curve given by an equation of the form

y2 = x3 + ax2 + bx + c

Note that elliptic curves are not ellipses!  Elliptic curves have one line of symmetry, two open ends, and either one or two pieces, as shown in Figures 1 and 2. They are called elliptic curves because the equations came up in the seventeenth century when mathematicians started studying the arc length of an ellipse.  These curves are considered the next most complicated type of curve after lines and conic sections, both of which have been understood pretty well since at least the ancient Greeks.   They are useful for a lot of things, including cryptography, as I describe in Section 8.3 of The Mathematics of Secrets.

Figure 1

Figure 1. The elliptic curve y2= x3 + x has one line of symmetry, two open ends, and one piece.

Figure 2

Figure 2. The elliptic curve y2 = x3 – x has one line of symmetry, two open ends, and two pieces.

 

In the late 1950’s it was conjectured that every elliptic curve was related to a modular form in the way that the example above is.  Proving this “Modularity Conjecture” took on more urgency in the 1980’s, when it was shown that showing the conjecture was true would also prove Fermat’s famous Last Theorem.  In 1995 Andrew Wiles, with help from Richard Taylor, proved enough of the Modularity Conjecture to show that Fermat’s Last Theorem was true, and the rest of the Modularity Conjecture was filled in over the next six years by Taylor and several of his associates.

Modular forms, the Monster, and Moonshine

Modular forms are also related to other shapes besides elliptic curves, and in the 1970’s John McKay and John Thompson became convinced that the modular form

J(z) = e -2 π i z + 196884 e 2 π i z + 21493760 e 4 π i z  + 864299970 e 6 π i z  + …

was related to the Monster.  Not only was 196884 equal to 196883 + 1, but 21493760 was equal to 21296876 + 196883 + 1, and 21296876 was also a number that came up in the study of the Monster.  Thompson suggested that there should be a natural way of associating the Monster with an infinite-dimensional shape, where the infinite-dimensional shape broke up into finite-dimensional pieces with each piece having a dimension corresponding to one of the coefficients of J(z).   This shape was (later) given the name V♮, using the natural sign from musical notation in a typically mathematical pun.  (Terry Gannon points out that there is also a hint that the conjectures “distill information illegally” from the Monster.) John Conway and Simon Norton formulated some guesses about the exact connection between the Monster and V♮, and gave them the name “Moonshine Conjectures” to reflect their speculative and rather unlikely-seeming nature. A plausible candidate for V♮ was constructed in the 1980’s and Richard Borcherds proved in 1992 that the candidate satisfied the Moonshine Conjectures.  This work was specifically cited when Borcherds was awarded the Fields medal in 1998.

The construction of V♮ turned out also to have a close connection with mathematical physics.  The reconciliation of gravity with quantum mechanics is one of the central problems of modern physics, and most physicists think that string theory is likely to be key to this resolution.  In string theory, the objects we traditionally think of as particles, like electrons and quarks, are really tiny strings curled up in many dimensions, most of which are two small for us to see.  An important question about this theory is exactly what shape these dimensions curl into.  One possibility is a 24-dimensional shape where the possible configurations of strings in the shape are described by V♮.  However, there are many other possible shapes and it is not known how to determine which one really corresponds to our world.

More Moonshine?

Since Borcherds’ proof, many variations of the original “Monstrous Moonshine” have been explored.  The other members of the Happy Family can be shown to have Moonshine relationships similar to those of the Monster.  “Modular Moonshine” says that certain elements of the Monster group should have their own infinite dimensional shapes, related to but not the same as V♮.  (The “modular” in “Modular Moonshine” is related to the one in “modular form” because they are both related to modular arithmetic, although the chain of connections is kind of long. )  “Mathieu Moonshine” shows that one particular group in the Happy Family has its own shape, entirely different from V♮, and “Umbral Moonshine” extends this to 23 other related groups which are not simple groups.  But the Pariah groups remained outsiders, rejected by both the Happy Family and by Moonshine — until September 2017.

Joshua Holden is professor of mathematics at the Rose-Hulman Institute of Technology. He is the author of The Mathematics of Secrets: Cryptography from Caesar Ciphers to Digital Encryption.

Pariah Moonshine Part I: The Happy Family and the Pariah Groups

by Joshua Holden

This post originally appeared on The Aperiodical. We republish it here with permission. 

HoldenBeing a mathematician, I often get asked if I’m good at calculating tips. I’m not. In fact, mathematicians study lots of other things besides numbers. As most people know, if they stop to think about it, one of the other things mathematicians study is shapes. Some of us are especially interested in the symmetries of those shapes, and a few of us are interested in both numbers and symmetries. The recent announcement of “Pariah Moonshine” has been one of the most exciting developments in the relationship between numbers and symmetries in quite some time. In this blog post I hope to explain the “Pariah” part, which deals mostly with symmetries. The “Moonshine”, which connects the symmetries to numbers, will follow in the next post.

What is a symmetry?

First I want to give a little more detail about what I mean by the symmetries of shapes. If you have a square made out of paper, there are basically eight ways you can pick it up, turn it, and put it down in exactly the same place. You can rotate it 90 degrees clockwise or counterclockwise. You can rotate it 180 degrees. You can turn it over, so the front becomes the back and vice versa. You can turn in over and then rotate it 90 degrees either way, or 180 degrees. And you can rotate it 360 degrees, which basically does nothing. We call these the eight symmetries of the square, and they are shown in Figure 1.

Figure1

Figure 1. The square can be rotated into four different positions, without or without being flipped over, for eight symmetries total.

If you have an equilateral triangle, there are six symmetries. If you have a pentagon, there are ten. If you have a pinwheel with four arms, there are only four symmetries, as shown in Figure 2, because now you can rotate it but if you turn it over it looks different. If you have a pinwheel with six arms, there are six ways. If you have a cube, there are 24 if the cube is solid, as shown in Figure 3. If the cube is just a wire frame and you are allowed to turn it inside out, then you get 24 more, for a total of 48.

Figure 2

Figure 2. The pinwheel can be rotated but not flipped, for four symmetries total.

Figure 3

Figure 3. The cube can be rotated along three different axes, resulting in 24 different symmetries.

These symmetries don’t just come with a count, they also come with a structure. If you turn a square over and then rotate it 90 degrees, it’s not the same thing as if you rotate it first and then flip it over. (Try it and see.) In this way, symmetries of shapes are like the permutations I discuss in Chapter 3 of my book, The Mathematics of Secrets: you can take products, which obey some of the same rules as products of numbers but not all of them. These sets of symmetries, which their structures, are called groups.

Groups are sets of symmetries with structure

Some sets of symmetries can be placed inside other sets. For example, the symmetries of the four-armed pinwheel are the same as the four rotations in the symmetries of the square. We say the symmetries of the pinwheel are a subgroup of the symmetries of the square. Likewise, the symmetries of the square are a subgroup of the symmetries of the solid cube, if you allow yourself to turn the cube over but not tip it 90 degrees, as shown in Figure 4.

Figure 4

Figure 4. The symmetries of the square are contained inside the symmetries of the cube if you are allowed to rotate and flip the cube but not tip it 90 degrees.

In some cases, ignoring a subgroup of the symmetries of a shape gets us another group, which we call the quotient group. If you ignore the subgroup of how the square is rotated, you get the quotient group where the square is flipped over or not, and that’s it. Those are the same as the symmetries of the capital letter A, so the quotient group is really a group. In other cases, for technical reasons, you can’t get a quotient group. If you ignore the symmetries of a square inside the symmetries of a cube, what’s left turns out not to be the symmetries of any shape.

You can always ignore all the symmetries of a shape and get just the do nothing (or trivial) symmetry, which is the symmetries of the capital letter P, in the quotient group. And you can always ignore none of the nontrivial symmetries, and get all of the original symmetries still in the quotient group. If these are the only two possible quotient groups, we say that the group is simple. The group of symmetries of a pinwheel with a prime number of arms is simple. So is the group of symmetries of a solid icosahedron, like a twenty-sided die in Dungeons and Dragons. The group of symmetries of a square is not simple, because of the subgroup of rotations. The group of symmetries of a solid cube is not simple, not because of the symmetries of the square, but because of the smaller subgroup of symmetries of a square with a line through it, as shown in Figures 5 and 6. The quotient group there is the same as the symmetries of the equilateral triangle created by cutting diagonally through a cube near a corner.

Figure 5

Figure 5. The symmetries of a square with line through it. We can turn the square 180 degrees and/or flip it, but not rotate it 90 degrees, so there are four.

Figure 6

Figure 6. The symmetries of the square with a line through it inside of the symmetries of the cube.

Categorizing the Pariah groups

As early as 1892, Otto Hölder asked if we could categorize all of the finite simple groups. (There are also shapes, like the circle, which have an infinite number of symmetries. We won’t worry about them now.)  It wasn’t until 1972 that Daniel Gorenstein made a concrete proposal for how to make a complete categorization, and the project wasn’t finished until 2002, producing along the way thousands of pages of proofs. The end result was that almost all of the finite simple groups fell into a few infinitely large categories: the cyclic groups, which are the groups of symmetries of pinwheels with a prime number of arms, the alternating groups, which are the groups of symmetries of solid hypertetrahedra in 5 or more dimensions, and the “groups of Lie type”, which are related to matrix multiplication over finite fields and describe certain symmetries of objects known as finite projective planes and finite projective spaces. (Finite fields are used in the AES cipher and I talk about them in Section 4.5 of The Mathematics of Secrets.)

Even before 1892, a few finite simple groups were discovered that didn’t seem to fit into any of these categories. Eventually it was proved that there were 26 “sporadic” groups, which didn’t fit into any of the categories and didn’t describe the symmetries of anything obvious — basically, you had to construct the shape to fit the group of symmetries that you knew existed, instead of starting with the shape and finding the symmetries. The smallest of the sporadic groups has 7920 symmetries in it, and the largest, known as the Monster, has over 800 sexdecillion symmetries. (That’s an 8 with 53 zeros after it!) Nineteen of the other sporadic groups turn out to be subgroups or quotient groups of subgroups of the Monster. These 20 became known as the Happy Family. The other 6 sporadic groups became known as the ‘Pariahs’.

The shape that was constructed to fit the Monster lives in 196883-dimensional space. In the late 1970’s a mathematician named John McKay noticed the number 196884 turning up in a different area of mathematics. It appeared as part of a function used in number theory, the study of the properties of whole numbers. Was there a connection between the Monster and number theory? Or was the idea of a connection just … moonshine?

Joshua Holden is professor of mathematics at the Rose-Hulman Institute of Technology. He is the author of The Mathematics of Secrets: Cryptography from Caesar Ciphers to Digital Encryption.

Joshua Holden: Quantum cryptography is unbreakable. So is human ingenuity

Two basic types of encryption schemes are used on the internet today. One, known as symmetric-key cryptography, follows the same pattern that people have been using to send secret messages for thousands of years. If Alice wants to send Bob a secret message, they start by getting together somewhere they can’t be overheard and agree on a secret key; later, when they are separated, they can use this key to send messages that Eve the eavesdropper can’t understand even if she overhears them. This is the sort of encryption used when you set up an online account with your neighbourhood bank; you and your bank already know private information about each other, and use that information to set up a secret password to protect your messages.

The second scheme is called public-key cryptography, and it was invented only in the 1970s. As the name suggests, these are systems where Alice and Bob agree on their key, or part of it, by exchanging only public information. This is incredibly useful in modern electronic commerce: if you want to send your credit card number safely over the internet to Amazon, for instance, you don’t want to have to drive to their headquarters to have a secret meeting first. Public-key systems rely on the fact that some mathematical processes seem to be easy to do, but difficult to undo. For example, for Alice to take two large whole numbers and multiply them is relatively easy; for Eve to take the result and recover the original numbers seems much harder.

Public-key cryptography was invented by researchers at the Government Communications Headquarters (GCHQ) – the British equivalent (more or less) of the US National Security Agency (NSA) – who wanted to protect communications between a large number of people in a security organisation. Their work was classified, and the British government neither used it nor allowed it to be released to the public. The idea of electronic commerce apparently never occurred to them. A few years later, academic researchers at Stanford and MIT rediscovered public-key systems. This time they were thinking about the benefits that widespread cryptography could bring to everyday people, not least the ability to do business over computers.

Now cryptographers think that a new kind of computer based on quantum physics could make public-key cryptography insecure. Bits in a normal computer are either 0 or 1. Quantum physics allows bits to be in a superposition of 0 and 1, in the same way that Schrödinger’s cat can be in a superposition of alive and dead states. This sometimes lets quantum computers explore possibilities more quickly than normal computers. While no one has yet built a quantum computer capable of solving problems of nontrivial size (unless they kept it secret), over the past 20 years, researchers have started figuring out how to write programs for such computers and predict that, once built, quantum computers will quickly solve ‘hidden subgroup problems’. Since all public-key systems currently rely on variations of these problems, they could, in theory, be broken by a quantum computer.

Cryptographers aren’t just giving up, however. They’re exploring replacements for the current systems, in two principal ways. One deploys quantum-resistant ciphers, which are ways to encrypt messages using current computers but without involving hidden subgroup problems. Thus they seem to be safe against code-breakers using quantum computers. The other idea is to make truly quantum ciphers. These would ‘fight quantum with quantum’, using the same quantum physics that could allow us to build quantum computers to protect against quantum-computational attacks. Progress is being made in both areas, but both require more research, which is currently being done at universities and other institutions around the world.

Yet some government agencies still want to restrict or control research into cryptographic security. They argue that if everyone in the world has strong cryptography, then terrorists, kidnappers and child pornographers will be able to make plans that law enforcement and national security personnel can’t penetrate.

But that’s not really true. What is true is that pretty much anyone can get hold of software that, when used properly, is secure against any publicly known attacks. The key here is ‘when used properly’. In reality, hardly any system is always used properly. And when terrorists or criminals use a system incorrectly even once, that can allow an experienced codebreaker working for the government to read all the messages sent with that system. Law enforcement and national security personnel can put those messages together with information gathered in other ways – surveillance, confidential informants, analysis of metadata and transmission characteristics, etc – and still have a potent tool against wrongdoers.

In his essay ‘A Few Words on Secret Writing’ (1841), Edgar Allan Poe wrote: ‘[I]t may be roundly asserted that human ingenuity cannot concoct a cipher which human ingenuity cannot resolve.’ In theory, he has been proven wrong: when executed properly under the proper conditions, techniques such as quantum cryptography are secure against any possible attack by Eve. In real-life situations, however, Poe was undoubtedly right. Every time an ‘unbreakable’ system has been put into actual use, some sort of unexpected mischance eventually has given Eve an opportunity to break it. Conversely, whenever it has seemed that Eve has irretrievably gained the upper hand, Alice and Bob have found a clever way to get back in the game. I am convinced of one thing: if society does not give ‘human ingenuity’ as much room to flourish as we can manage, we will all be poorer for it.Aeon counter – do not remove

Joshua Holden is professor of mathematics at the Rose-Hulman Institute of Technology and the author of The Mathematics of Secrets.

This article was originally published at Aeon and has been republished under Creative Commons.

Cipher challenge #3 from Joshua Holden: Binary ciphers

The Mathematics of Secrets by Joshua Holden takes readers on a tour of the mathematics behind cryptography. Most books about cryptography are organized historically, or around how codes and ciphers have been used in government and military intelligence or bank transactions. Holden instead focuses on how mathematical principles underpin the ways that different codes and ciphers operate. Discussing the majority of ancient and modern ciphers currently known, The Mathematics of Secrets sheds light on both code making and code breaking. Over the next few weeks, we’ll be running a series of cipher challenges from Joshua Holden. The last post was on subliminal channels. Today’s is on binary ciphers:

Binary numerals, as most people know, represent numbers using only the digits 0 and 1.  They are very common in modern ciphers due to their use in computers, and they frequently represent letters of the alphabet.  A numeral like 10010 could represent the (1 · 24 + 0 · 23 + 0 · 22 + 1 · 2 + 0)th = 18th letter of the alphabet, or r.  So the entire alphabet would be:

 plaintext:   a     b     c     d     e     f     g     h     i     j
ciphertext: 00001 00010 00011 00100 00101 00110 00111 01000 01001 01010

 plaintext:   k     l     m     n     o     p     q     r     s     t
ciphertext: 01011 01100 01101 01110 01111 10000 10001 10010 10011 10100

 plaintext:   u     v     w     x     y     z
ciphertext: 10101 10110 10111 11000 11001 11010

The first use of a binary numeral system in cryptography, however, was well before the advent of digital computers. Sir Francis Bacon alluded to this cipher in 1605 in his work Of the Proficience and Advancement of Learning, Divine and Humane and published it in 1623 in the enlarged Latin version De Augmentis Scientarum. In this system not only the meaning but the very existence of the message is hidden in an innocuous “covertext.” We will give a modern English example.

Suppose we want to encrypt the word “not” into the covertext “I wrote Shakespeare.” First convert the plaintext into binary numerals:

   plaintext:   n      o     t
  ciphertext: 01110  01111 10100

Then stick the digits together into a string:

    011100111110100

Now we need what Bacon called a “biformed alphabet,” that is, one where each letter can have a “0-form” and a “1-form.”We will use roman letters for our 0-form and italic for our 1-form. Then for each letter of the covertext, if the corresponding digit in the ciphertext is 0, use the 0-form, and if the digit is 1 use the 1-form:

    0 11100 111110100xx
    I wrote Shakespeare.

Any leftover letters can be ignored, and we leave in spaces and punctuation to make the covertext look more realistic. Of course, it still looks odd with two different typefaces—Bacon’s examples were more subtle, although it’s a tricky business to get two alphabets that are similar enough to fool the casual observer but distinct enough to allow for accurate decryption.

Ciphers with binary numerals were reinvented many years later for use with the telegraph and then the printing telegraph, or teletypewriter. The first of these were technically not cryptographic since they were intended for convenience rather than secrecy. We could call them nonsecret ciphers, although for historical reasons they are usually called codes or sometimes encodings. The most well-known nonsecret encoding is probably the Morse code used for telegraphs and early radio, although Morse code does not use binary numerals. In 1833, Gauss, whom we met in Chapter 1, and the physicist Wilhelm Weber invented probably the first telegraph code, using essentially the same system of 5 binary digits as Bacon. Jean-Maurice-Émile Baudot used the same idea for his Baudot code when he invented his teletypewriter system in 1874. And the Baudot code is the one that Gilbert S. Vernam had in front of him in 1917 when his team at AT&T was asked to investigate the security of teletypewriter communications.

Vernam realized that he could take the string of binary digits produced by the Baudot code and encrypt it by combining each digit from the plaintext with a corresponding digit from the key according to the rules:

0 ⊕ 0 = 0
0 ⊕ 1 = 1
1 ⊕ 0 = 1
1 ⊕ 1 = 0

For example, the digits 10010, which ordinarily represent 18, and the digits 01110, which ordinarily represent 14, would be combined to get:

1 0 0 1 0
0 1 1 1 0


1 1 1 0 0

This gives 11100, which ordinarily represents 28—not the usual sum of 18 and 14.

Some of the systems that AT&T was using were equipped to automatically send messages using a paper tape, which could be punched with holes in 5 columns—a hole indicated a 1 in the Baudot code and no hole indicated a 0. Vernam configured the teletypewriter to combine each digit represented by the plaintext tape to the corresponding digit from a second tape punched with key characters. The resulting ciphertext is sent over the telegraph lines as usual.

At the other end, Bob feeds an identical copy of the tape through the same circuitry. Notice that doing the same operation twice gives you back the original value for each rule:

(0 ⊕ 0) ⊕ 0 = 0 ⊕ 0 = 0
(0 ⊕ 1) ⊕ 1 = 1 ⊕ 1 = 0
(1 ⊕ 0) ⊕ 0 = 1 ⊕ 0 = 1
(1 ⊕ 1) ⊕ 1 = 0 ⊕ 1 = 1

Thus the same operation at Bob’s end cancels out the key, and the teletypewriter can print the plaintext. Vernam’s invention and its further developments became extremely important in modern ciphers such as the ones in Sections 4.3 and 5.2 of The Mathematics of Secrets.

But let’s finish this post by going back to Bacon’s cipher.  I’ve changed it up a little — the covertext below is made up of two different kinds of words, not two different kinds of letters.  Can you figure out the two different kinds and decipher the hidden message?

It’s very important always to understand that students and examiners of cryptography are often confused in considering our Francis Bacon and another Bacon: esteemed Roger. It is easy to address even issues as evidently confusing as one of this nature. It becomes clear when you observe they lived different eras.

Answer to Cipher Challenge #2: Subliminal Channels

Given the hints, a good first assumption is that the ciphertext numbers have to be combined in such a way as to get rid of all of the fractions and give a whole number between 1 and 52.  If you look carefully, you’ll see that 1/5 is always paired with 3/5, 2/5 with 1/5, 3/5 with 4/5, and 4/5 with 2/5.  In each case, twice the first one plus the second one gives you a whole number:

2 × (1/5) + 3/5 = 5/5 = 1
2 × (2/5) + 1/5 = 5/5 = 1
2 × (3/5) + 4/5 = 10/5 = 2
2 × (4/5) + 2/5 = 10/5 = 2

Also, twice the second one minus the first one gives you a whole number:

2 × (3/5) – 1/5 = 5/5 = 1
2 × (1/5) – 2/5 = 0/5 = 0
2 × (4/5) – 3/5 = 5/5 = 1
2 × (2/5) – 4/5 = 0/5 = 0

Applying

to the ciphertext gives the first plaintext:

39 31 45 45 27 33 31 40 47 39 28 31 44 41
 m  e  s  s  a  g  e  n  u  m  b  e  r  o
40 31 35 45 46 34 31 39 31 30 35 47 39
 n  e  i  s  t  h  e  m  e  d  i  u  m

And applying

to the ciphertext gives the second plaintext:

20  8  5 19  5  3 15 14  4 16 12  1  9 14 
 t  h  e  s  e  c  o  n  d  p  l  a  i  n
20  5 24 20  9 19  1 20 12  1 18  7  5
 t  e  x  t  i  s  a  t  l  a  r  g  e

To deduce the encryption process, we have to solve our two equations for C1 and C2.  Subtracting the second equation from twice the first gives:


so

Adding the first equation to twice the second gives:


so

Joshua Holden is professor of mathematics at the Rose-Hulman Institute of Technology.

Cipher challenge #2 from Joshua Holden: Subliminal channels

The Mathematics of Secrets by Joshua Holden takes readers on a tour of the mathematics behind cryptography. Most books about cryptography are organized historically, or around how codes and ciphers have been used in government and military intelligence or bank transactions. Holden instead focuses on how mathematical principles underpin the ways that different codes and ciphers operate. Discussing the majority of ancient and modern ciphers currently known, The Mathematics of Secrets sheds light on both code making and code breaking. Over the next few weeks, we’ll be running a series of cipher challenges from Joshua Holden. The first was on Merkle’s puzzles. Today’s focuses on subliminal channels:

As I explain in Section 1.6 of The Mathematics of Secrets, in 1929 Lester Hill invented the first general method for encrypting messages using a set of multiple equations in multiple unknowns.  A less general version, however, had already appeared in 1926, submitted by an 18-year-old to a cryptography column in a detective magazine.  This was Jack Levine, who would later become a prolific researcher in several areas of mathematics, including cryptography.

Levine’s system was billed as a way of encrypting two different messages at the same time.  Maybe one of them was the real message and the other was a dummy message–if the message was intercepted, the interceptor could be thrown off the scent by showing them the dummy message.  This sort of system is now known as a subliminal channel.

The system starts with numbering the letters of the alphabet in two different ways:

   a  b  c  d  e  f  g  h  i  j  k  l  m
  27 28 29 30 31 32 33 34 35 36 37 38 39
   1  2  3  4  5  6  7  8  9 10 11 12 13
  
   n  o  p  q  r  s  t  u  v  w  x  y  z
  40 41 42 43 44 45 46 47 48 49 50 51 52
  14 15 16 17 18 19 20 21 22 23 24 25 26

Suppose the first plaintext, or unencrypted message, is “tuesday” and the second plaintext is “tonight.”  We use the first set of numbers for the first plaintext:

   t  u  e  s  d  a  y
  46 47 31 45 30 27 51

and the second set for the second plaintext:

   t  o  n  i  g  h  t
  20 15 14  9  7  8 20

The encrypted message, or ciphertext, is made up of pairs of numbers.  The first number in each pair is half the sum of the two message numbers, and the second number is half the difference:

    t       u        e       s       d       a        y
   46      47       31      45      30      27       51
  
    t       o        n       i       g       h        t
   20      15       14       9       7       8       20
  
33,13    31,16  22½,8½   27,18 18½,11½  17½,9½  35½,15½

To decrypt the first message, just take the sum of the two numbers in the pair, and to decrypt the second message just take the difference.  This works because if P1 is the first plaintext number and P2 is the second, then the first ciphertext number is

and the second is

Then the plaintext can be recovered from the ciphertext using

and

This system is not as secure as Hill’s because it gives away too much information.  For starters, the existence and nature of the fractions is a clue to the encryption process.  (The editor of the cryptography column suggested doubling the numbers to avoid the fractions, but then the pattern of odd and even numbers would still give information away.)  Also, the first number in each pair is always between 14 and 39 and is always larger than the second number, which is always between ½ and 25 ½.  This suggests that subtraction might be relevant, and the fact that there are twice as many numbers as letters might make a codebreaker suspect the existence of a second message and a second process.  Hill’s system solves some of these issues, but the problem of information leakage continues to be relevant with modern-day ciphers.

With those hints in mind, can you break the cipher used in the following message?

11 3/5, 15 4/5   10 4/5,  9 2/5   17,     11        14 1/5, 16 3/5
 9 4/5,  7 2/5   12 3/5,  7 4/5    9 2/5, 12  1/5   13 1/5, 13 3/5
18,     11       12 2/5, 14 1/5    8 4/5, 10  2/5   12 1/5,  6 3/5
15 4/5, 12 2/5   13 3/5, 13 4/5   12,     16        11 2/5,  8 1/5
 9 1/5, 16 3/5   14,     17       16 3/5, 12  4/5    9 4/5, 14 2/5
12 1/5,  6 3/5   11 3/5, 15 4/5   10,     11        11 4/5,  6 2/5
10 2/5, 14 1/5   17 2/5, 12 1/5   14 3/5,  9  4/5

Once you have the two plaintexts, can you deduce the process used to encrypt them?

 

Answer to Cipher Challenge #1: Merkle’s Puzzles

The hole in the version of Merkle’s puzzles is that the shift we used for encrypting is vulnerable to a known-plaintext attack. That means that if Eve knows the ciphertext and part of the plaintext, she can get the rest of the plaintext. In Cipher Challenge #1, she knew that the word “ten” is part of the plaintext. So she shifts it until she finds a ciphertext that matches one of the puzzles:

ten
UFO
VGP

“Aha!” says Eve. “The first puzzle starts with VGP, so it must decrypt to ten!” Then she decrypts the rest of the puzzle:

VGPVY QUGXG PVYGP VAQPG UKZVG GPUGX GPVGG PBTPU XSNHT JZFEB
whqwz rvhyh qwzhq wbrqh vlawh hqvhy hqwhh qcuqv ytoiu kagfc
xirxa swizi rxair xcsri wmbxi irwiz irxii rdvrw zupjv lbhgd
yjsyb txjaj sybjs ydtsj xncyj jsxja jsyjj sewsx avqkw mcihe
                             ⋮
qbkqt lpbsb kqtbk qvlkb pfuqb bkpbs bkqbb kwokp snico euazw
rclru mqctc lrucl rwmlc qgvrc clqct clrcc lxplq tojdp fvbax
sdmsv nrdud msvdm sxnmd rhwsd dmrdu dmsdd myqmr upkeq gwcby
tentw oseve ntwen tyone sixte ensev entee nzrns vqlfr hxdcz

So the secret key is 2, 7, 21, 16.

The hole can be fixed by using a cipher that is less vulnerable to known-plaintext attacks. Sections 4.4 and 4.5 of The Mathematics of Secrets give examples of ciphers that would be more secure.

Cipher challenge #1 from Joshua Holden: Merkle’s Puzzles

The Mathematics of Secrets by Joshua Holden takes readers on a tour of the mathematics behind cryptography. Most books about cryptography are organized historically, or around how codes and ciphers have been used in government and military intelligence or bank transactions. Holden instead focuses on how mathematical principles underpin the ways that different codes and ciphers operate. Discussing the majority of ancient and modern ciphers currently known, The Mathematics of Secrets sheds light on both code making and code breaking. Over the next few weeks, we’ll be running a series of cipher challenges from Joshua Holden. Presenting the first, on Merkle’s puzzles. 

For over two thousand years, everyone assumed that before Alice and Bob start sending secret messages, they’d need to get together somewhere where an eavesdropper couldn’t overhear them in order to agree on the secret key they would use. In the fall of 1974, Ralph Merkle was an undergraduate at the University of California, Berkeley, and taking a class in computer security. He began wondering if there was a way around the assumption that everyone had always made. Was it possible for Alice to send Bob a message without having them agree on a key beforehand? Systems that do this are now called public-key cryptography, and they are a key ingredient in Internet commerce. Maybe Alice and Bob could agree on a key through some process that the eavesdropper couldn’t understand, even if she could overhear it.

Merkle’s idea, which is now commonly known as Merkle’s puzzles, was slow to be accepted and went through several revisions. Here is the version that was finally published. Alice starts by creating a large number of encrypted messages (the puzzles) and sends them to Bob.

The beginning of Merkle’s puzzles.

Merkle suggested that the encryption should be chosen so that breaking each puzzle by brute force is “tedious, but quite possible.” For our very small example, we will just use a cipher which shifts each letter in the message by a specified number of letters. Here are ten puzzles:

VGPVY QUGXG PVYGP VAQPG UKZVG GPUGX GPVGG PBTPU XSNHT JZFEB
GJBAV ARSVI RFRIR AGRRA GJRYI RFRIR AGRRA VTDHC BMABD QMPUP
AFSPO JOFUF FOUFO TFWFO UXFOU ZGJWF TFWFO UFFOI RCXJQ EHHZF
JIZJI ZNDSO RZIOT ADAOZ ZINZQ ZIOZZ IWOPL KDWJH SEXRJ IKAVV
YBJSY DSNSJ YJJSY BJSYD KNAJX JAJSK TZWXJ AJSYJ JSFNY UZAKM
QCTCL RFPCC RUCLR WDMSP RCCLD GDRCC LQCTC LRCCL JLXUW HAYDT
ADLUA FMVBY ALUVU LVULZ LCLUZ LCLUA LLUGE AMPWB PSEQG IKDSV
JXHUU VYLUJ XHUUJ UDDYD UIULU DJUUD AUTRC SGBOD ALQUS ERDWN
RDUDM SDDMS VDMSX RDUDM SDDMM HMDSD DMRHW SDDMR DUDMS DDMAW
BEMTD MBEMV BGBPZ MMMQO PBMMV AMDMV NQDMA MDMVB MMVUR YCEZC

Alice explains to Bob that each puzzle consists of three sets of numbers. The first number is an ID number to identify the puzzle. The second set of numbers is a secret key from a more secure cipher which Alice and Bob could actually use to communicate. The last number is the same for all puzzles and is a check so that Bob can make sure he has solved the puzzle correctly. Finally, the puzzles are padded with random letters so that they are all the same length, and each puzzle is encrypted by shifting a different number of letters.

Bob picks one of the puzzles at random and solves it by a brute force search. He then sends Alice the ID number encrypted in the puzzle.

Bob solves the puzzle.

For example, if he picked the puzzle on the fifth line above, he might try shifting the letters:

YBJSY DSNSJ YJJSY BJSYD KNAJX JAJSK TZWXJ AJSYJ JSFNY UZAKM
zcktz etotk zkktz cktze lobky kbktl uaxyk bktzk ktgoz vabln
adlua fupul allua dluaf mpclz lclum vbyzl clual luhpa wbcmo
bemvb gvqvm bmmvb emvbg nqdma mdmvn wczam dmvbm mviqb xcdnp

qtbkq vkfkb qbbkq tbkqv cfsbp bsbkc lropb sbkqb bkxfq mrsce
ruclr wlglc rcclr uclrw dgtcq ctcld mspqc tclrc clygr nstdf
svdms xmhmd sddms vdmsx ehudr dudme ntqrd udmsd dmzhs otueg
twent ynine teent wenty fives evenf ourse vente enait puvfh

Now he knows the ID number is “twenty” and the secret key is 19, 25, 7, 4. He sends Alice “twenty”.

Alice has a list of the decrypted puzzles, sorted by ID number:

ID secret key check
zero nineteen ten seven twentyfive seventeen
one one six twenty fifteen seventeen
two nine five seventeen twelve seventeen
three five three ten nine seventeen
seventeen twenty seventeen nineteen sixteen seventeen
twenty nineteen twentyfive seven four seventeen
twentyfour ten one one seven seventeen

So she can also look up the secret key and find that it is 19, 25, 7, 4. Now Alice and Bob both know a secret key to a secure cipher, and they can start sending encrypted messages. (For examples of ciphers they might use, see Sections 1.6, 4.4, and 4.5 of The Mathematics of Secrets.)

Alice and Bob both have the secret key.

Can Eve the eavesdropper figure out the secret key? Let’s see what she has overheard. She has the encryptions of all of the puzzles, and the check number. She doesn’t know which puzzle Bob picked, but she does know that the ID number was “twenty”. And she doesn’t have Alice’s list of decrypted puzzles. It looks like she has to solve all of the puzzles before she can figure out which one Bob picked and get the secret key. This of course is possible, but will take her a lot longer than the procedure took Alice or Bob.

Eve can’t keep up.

Merkle’s puzzles were always a proof of concept — even Merkle knew that they wouldn’t work in practice. Alice and Bob’s advantage over Eve just isn’t large enough. Nevertheless, they had a direct impact on the development of public-key systems that are still very much in use on the Internet, such as the ones in Chapters 7 and 8 of The Mathematics of Secrets.

Actually, the version of Merkle’s puzzles that I’ve given here has a hole in it. The shift cipher has a weakness that lets Eve use Bob’s ID number to figure out which puzzle he solved without solving them herself. Can you use it to find the secret key which goes with ID number “ten”?

Joshua Holden: The secrets behind secret messages

“Cryptography is all about secrets, and throughout most of its history the whole field has been shrouded in secrecy.  The result has been that just knowing about cryptography seems dangerous and even mystical.”

In The Mathematics of Secrets: Cryptography from Caesar Ciphers to Digital EncryptionJoshua Holden provides the mathematical principles behind ancient and modern cryptic codes and ciphers. Using famous ciphers such as the Caesar Cipher, Holden reveals the key mathematical idea behind each, revealing how such ciphers are made, and how they are broken.  Holden recently took the time to answer questions about his book and cryptography.


There are lots of interesting things related to secret messages to talk abouthistory, sociology, politics, military studies, technology. Why should people be interested in the mathematics of cryptography? 
 
JH: Modern cryptography is a science, and like all modern science it relies on mathematics.  If you want to really understand what modern cryptography can and can’t do you need to know something about that mathematical foundation. Otherwise you’re just taking someone’s word for whether messages are secure, and because of all those sociological and political factors that might not be a wise thing to do. Besides that, I think the particular kinds of mathematics used in cryptography are really pretty. 
 
What kinds of mathematics are used in modern cryptography? Do you have to have a Ph.D. in mathematics to understand it? 
 
JH: I once taught a class on cryptography in which I said that the prerequisite was high school algebra.  Probably I should have said that the prerequisite was high school algebra and a willingness to think hard about it.  Most (but not all) of the mathematics is of the sort often called “discrete.”  That means it deals with things you can count, like whole numbers and squares in a grid, and not with things like irrational numbers and curves in a plane.  There’s also a fair amount of statistics, especially in the codebreaking aspects of cryptography.  All of the mathematics in this book is accessible to college undergraduates and most of it is understandable by moderately advanced high school students who are willing to put in some time with it. 
 
What is one myth about cryptography that you would like to address? 
 
JH: Cryptography is all about secrets, and throughout most of its history the whole field has been shrouded in secrecy.  The result has been that just knowing about cryptography seems dangerous and even mystical. In the Renaissance it was associated with black magic and a famous book on cryptography was banned by the Catholic Church. At the same time, the Church was using cryptography to keep its own messages secret while revealing as little about its techniques as possible. Through most of history, in fact, cryptography was used largely by militaries and governments who felt that their methods should be hidden from the world at large. That began to be challenged in the 19th century when Auguste Kerckhoffs declared that a good cryptographic system should be secure with only the bare minimum of information kept secret. 
 
Nowadays we can relate this idea to the open-source software movement. When more people are allowed to hunt for “bugs” (that is, security failures) the quality of the overall system is likely to go up. Even governments are beginning to get on board with some of the systems they use, although most still keep their highest-level systems tightly classified. Some professional cryptographers still claim that the public can’t possibly understand enough modern cryptography to be useful. Instead of keeping their writings secret they deliberately make it hard for anyone outside the field to understand them. It’s true that a deep understanding of the field takes years of study, but I don’t believe that people should be discouraged from trying to understand the basics. 
 
I invented a secret code once that none of my friends could break. Is it worth any money? 
 
JH: Like many sorts of inventing, coming up with a cryptographic system looks easy at first.  Unlike most inventions, however, it’s not always obvious if a secret code doesn’t “work.” It’s easy to get into the mindset that there’s only one way to break a system so all you have to do is test that way.  Professional codebreakers know that on the contrary, there are no rules for what’s allowed in breaking codes. Often the methods for codebreaking with are totally unsuspected by the codemakers. My favorite involves putting a chip card, such as a credit card with a microchip, into a microwave oven and turning it on. Looking at the output of the card when bombarded 
by radiation could reveal information about the encrypted information on the card! 
 
That being said, many cryptographic systems throughout history have indeed been invented by amateurs, and many systems invented by professionals turned out to be insecure, sometimes laughably so. The moral is, don’t rely on your own judgment, anymore than you should in medical or legal matters. Get a second opinion from a professional you trustyour local university is a good place to start.   
 
A lot of news reports lately are saying that new kinds of computers are about to break all of the cryptography used on the Internet. Other reports say that criminals and terrorists using unbreakable cryptography are about to take over the Internet. Are we in big trouble? 
 
JH: Probably not. As you might expect, both of these claims have an element of truth to them, and both of them are frequently blown way out of proportion. A lot of experts do expect that a new type of computer that uses quantum mechanics will “soon” become a reality, although there is some disagreement about what “soon” means. In August 2015 the U.S. National Security Agency announced that it was planning to introduce a new list of cryptography methods that would resist quantum computers but it has not announced a timetable for the introduction. Government agencies are concerned about protecting data that might have to remain secure for decades into the future, so the NSA is trying to prepare now for computers that could still be 10 or 20 years into the future. 
 
In the meantime, should we worry about bad guys with unbreakable cryptography? It’s true that pretty much anyone in the world can now get a hold of software that, when used properly, is secure against any publicly known attacks. The key here is “when used properly. In addition to the things I mentioned above, professional codebreakers know that hardly any system is always used properly. And when a system is used improperly even once, that can give an experienced codebreaker the information they need to read all the messages sent with that system.  Law enforcement and national security personnel can put that together with information gathered in other waysurveillance, confidential informants, analysis of metadata and transmission characteristics, etc.and still have a potent tool against wrongdoers. 
 
There are a lot of difficult political questions about whether we should try to restrict the availability of strong encryption. On the flip side, there are questions about how much information law enforcement and security agencies should be able to gather. My book doesn’t directly address those questions, but I hope that it gives readers the tools to understand the capabilities of codemakers and codebreakers. Without that you really do the best job of answering those political questions.

Joshua Holden is professor of mathematics at the Rose-Hulman Institute of Technology in Terre Haute, IN. His most recent book is The Mathematics of Secrets: Cryptography from Caesar Ciphers to Digital Encryption.