Big Data & Algorithms – What We Want or the End of Free Will?

SONY DSCOne of the biggest brain-busters when it comes to Christianity is the concept of free will. Do we make our own choices or are we simply following a script that God created for us? Put differently – if God created us, and controls all of the individual factors that determine our preferences, do individuals really have the freedom to make their own decisions?

Don’t worry – I won’t try to answer those age-old questions in this brief blog post. I do, however, think it is worth spending a few minutes discussing the ongoing recording and digitization of our lives and it’s impact on our ability to make our own choices.

As we’ve discussed in earlier posts (link), one of the most important developments in the past 5-7 years has been the mass collection of data about every aspect of our lives. From where we are, what we buy, and what we look at online, to who we talk to and what we say. Increasingly, someone somewhere is recording that data. While it can be argued that this development is either helpful or harmful, the fact that this data is being collected isn’t in dispute.

Given that fact, it’s reasonable to ask, what the heck are they using all this data for?

The answer to this question is fairly simple. The personal data collected is being used to create algorithms. An algorithm is a formula (or group of formulas) that are used to predict the outcome of something. By collecting and analyzing raw data (like your buying habits, what you view online, or how and when you talk to others), it is possible to identify patterns that can be predicted by algorithms. The more personal data these organizations can collect and analyze, the more accurate they will be. Algorithms can be used to t predict group behavior on both a large and small scale. And they’re getting pretty good at it. A February 12th article in the New York Times reported that Target’s algorithms for predicting shopper behavior are so accurate that they can determine not only if a woman is pregnant, but how far along she is in her pregnancy. In one particular example, a father became alarmed when his teenage daughter began receiving mailers from Target for baby products. Only after complaining to his local Target store and then having an extended discussion with his daughter did he learn that she was actually pregnant. In a separate article, the New York Times also reported that Netflix used similar algorithms when putting together their new TV series “House of Cards”. The online video rental company analyzed the movies and shows their subscribers streamed most often. What they found was:

1. A large percentage of their subscribers streamed movies by the director David Fincher.
2. The original British version of the series “House of Cards” was also streamed in large numbers.
3. Viewers of both of the above also tended to stream films featuring Kevin Spacey.

With this information, Netflix constructed an online series consisting of exactly these three elements. The analytics were right on the money. Netflix’s “House of Cards” went on to be the most streamed piece of content in the US, along with 40 other countries.

Indeed – the science of web analytics is advancing to the point where it can be used to predict the outcome of anything.

That brings us back to our discussion of free will.

As more aspects of our personal lives are recorded and analyzed, those collecting this data will develop algorithms that are more and more accurate to better predict our behavior. As years and years of data accumulate and good algorithms are proven correct over time, it is reasonable to assume that the decision makers who use these algorithms will begin to look at the predictions with increasing certainty.

Given that these algorithms will increasingly have direct control over what we see, hear, learn, and buy, they will directly impact on our lives. Even before we’re born (see: Target Baby), these algorithms will alter the life options we see based on the choices of people demographically similar to us. As we move through life, this digital characterization of who we are will become more detailed, further editing the options available to us.

Just as religious scholars have long questioned man’s relationship with God, we must ask ourselves about the impact of an algorithm driven world and our ability to make choices.

Specifically – will it be possible for individuals to make independent choices, to make choices based on their own desires, in a world where algorithms control all the options that are available?

Or, is it possible that with well-constructed algorithms, we can produce a new, better world where what we want can be mathematically determined before we know we want it?

One thing is for sure – given the pace that data is being collected and the improving accuracy of algorithms, we will all learn the answer soon enough…

Share your thoughts in the comments…


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s