Human Predictability = 93%!?
Human predictability is a key premise of modern digital services ranging from Google search to Amazon’s product recommendations, and will also play a key role in the Responsible Business in the Blogosphere project. We all enjoy that Google’s PageRank can help predict which web-page we are out looking for, based on a few query terms and the global link pattern of the Internet. Likewise, Amazon’s business is based on their ability to predict a given customer’s interests in products from patterns observed in other users’ preferences. My own view is that predictability of human behavior is a necessary condition for our success in collaboration and communication: How could languages work without predictability?
Hence, it should not be surprise that human behavior is predictable, but exactly how high is human predictability? This question was addressed recently and the answer: Individual human mobility patterns may be predicted with an accuracy of 93%! This was the remarkable result of a large mobility study published in Science, on February 18, 2010, by Chaoming Song et al., of the Barabasi Lab in Boston. The result still echoes in the news: “Location data from mobile phones has revealed that the vast majority of human movement is predictable” in The Telegraph, UK, on February 24, “You’re predictable! So says your cell phone” at MSNBC, US, and “Researcher: People are very predictable” at UPI.com, US, both on March 5.
The study is part of a larger investigation of mobile phone data that has earlier produced high profile publications on stability of communities and quite a bit of controversy related to privacy issues. The data is based on mobile phone billing records form an un-disclosed European country. The Barabasi Lab has a FAQ related to the study where lead author Chaoming Song writes: “The data we used was collected for billing purposes as required by law, and anonymized by the provider. We have no personal information (name, age, or address) on individual users. For us they are just like particles in a gas that move and interact with each other. We obtained IRB (IRB = Internal review board, LKH) approval specifically for this project and all researchers have taken a course on privacy and a test before working with this data”.
The paper shows that mobility patterns deviate dramatically between individuals following a so-called ‘long tail’ power law, so that few people move a lot while most people have small ‘radii of gyration’. While there is huge variability in how much people move, it turns out that the individual sequence of movements is predictable. It should be noted that the paper does not provide actual predictions of human behavior with an accuracy of 93%. Using a simple heuristic (e.g., based on time of the day), they can lower bound the predictability at 70%, while they use information theoretic methods to bound the predictability from above by the quoted rate of 93%. Thus, strictly speaking we have an interval 70-93% in which actual value of the predictability resides. The spread of the upper bound is low: most subjects are predictable in the range 80-93%. This is the most surprising results according to Song.
These findings are encouraging for quantitative social sciences: If we find the right variables human behavior can be modeled. From a privacy point of view it may be more worrisome that in order to provide efficient location based services we may be forced to log each individual’s behavior.