Information Theory, Sport, Betting and the Stock Market

Posted on August 5, 2012 by



It’s interesting how Information Theory, Las Vegas and Wall Street have been intertwined over the years.

Claude Shannon, the father of Information Theory

You may not have heard of Claude Shannon, but his ideas made the modern information age possible. He

Claude Shannon with his robot mouse.

created the field of Information Theory when he published a book “The Mathematical Theory of Communication.” This little book is remarkable not only for the ideas it contains, but because it is so easily accessible. His theories set limits on how much information can be pushed through a wire, and they are leveraged every day in networking, wireless communications, cryptography and computing research.

Shannon was also a juggler and a gambler and he created the first robot mouse able to navigate a maze. He cracked German codes during the war. Then after the war he invented sampling theory. (I’ll bet he enjoyed whiskey and cigars, but I haven’t found an online citation to prove it.)

As a hobby, he co-created one of the first wearable computers with Edward O. Thorp. The device was used to improve the odds when playing roulette. (I’m interested in roulette also, check out roulette

Information Theory meets investing

You might have noticed that the stock market isn’t much different from Vegas, and it turns out that the same mathematics apply.  The good news is that the math works out much better when betting at NYSE than at Cesears Palace. From wikipedia:

Shannon and Thorp also applied the same theory, later known as the Kelly criterion, to the stock market with even better results. Over the decades, Kelly’s scientific formula has become a part of mainstream investment theory and the most prominent users, well-known and successful billionaire investors Warren BuffettBill Gross, and Jim Simons use Kelly methods. Warren Buffett met Thorp the first time in 1968. It’s said that Buffett uses a form of the Kelly criterion in deciding how much money to put into various holdings. Also Elwyn Berlekamp had applied the same logical algorithm for Axcom Trading Advisors, an alternative investment management company, that he had founded. Berlekamp’s company was acquired by Jim Simons and his Renaissance Technologies Corp hedge fund in 1992, whereafter its investment instruments were either subsumed into (or essentially renamed as) Renaissance’s flagship Medallion Fund.

The next wave: Thomas Cover

Thomas Cover was an intellectual descendant of Shannon.  According to Stanford’s article about him:

Thomas Cover

Cover was a man of remarkable breadth in his research interests, making notable contributions in fields ranging from information theory and mathematical statistics to data compression, pattern recognition and stock market investment strategies.

His work has affected my career in several ways. I was introduced to his Information Theory work by Erik Verriest at GT in a graduate Information Theory course. Cover’s book (with Joy Thomas) is the standard for this field, and it was the book Verriest used. I first learned about Claude Shannon from Thomas Cover, and I leveraged Information Theory in my own Ph.D. dissertation as a way to measure diversity in robot teams.

I crossed paths with Cover later, as I became of fan of instance-based learning methods when Frank Dellaert pointed me to the K-Nearest Neighbor algorithm. Cover & Hart published “Nearest neighbor pattern classification” in 1967. Their seminal paper provides a theoretical underpinning for the tremendous volume of research in nearest neighbor methods that has followed.

Cover was also a fan of sport and betting. Again from Stanford’s page:

This expertise led to Cover’s tenure as a statistician for the California State Lottery, under contract from 1986 to 1994, while he remained on the faculty at Stanford. During his time with the lottery, Cover variously designed tests for lottery balls and wheels, analyzed the payoff structure of games and devised ways to beat the lottery in order to anticipate and defeat fraud.

Cover loved sports. As a boy, he played in the Little League World Series, ran cross-country and won the San Bernardino tennis championship. At MIT, he played varsity tennis. His athletic experiences led to a highly popular class at Stanford, Mathematics of Sports, and several of his students went on to become statisticians in the NFL and Major League Baseball.

This deep interest in games led to a discovery that the mathematics of games can be applied to other forms of gambling, as well, namely stock market investment. Beginning in 1980, and playing out over the course of the next quarter century, Cover and several co-authors published a series of papers on mathematical approaches to portfolio strategy drawn from the principles of information theory. It’s no wonder that these days, real money online casinos for US players tries very hard to supress such information.

And finally, a mystery:

Cover left behind a mysterious idea (my words) that he called “The Universal Portfolio” for allocating funds to stocks in a portfolio. Unlike any other allocation method before it, this one has performance guarantees about future return. But nobody seems to have been able to transform it into a successful investment fund. Cover founded a hedge fund based on his idea, but there’s very little information about it. Did it succeed? Why or why not? If you know, please let me know.

The Universal Portfolio, now called the Cover Portfolio by most, was a brilliant idea with at least one drawback: It requires a nearly infinite amount of computing power to calculate. One of my colleagues at Georgia Tech, Santosh Vempala worked with Adam Kalai to find randomized algorithms that might offer similar investment performance at lower computing costs.

You can read Cover’s original paper on the method here.

keywords: Information Theory, betting, gambling, Claude Shannon, Thomas Cover