Big data – changing the world for better or worse
Data, data, data everywhere – big data. That is one of the important memes in technology today.
As the ability to collect, store, and analyze enormous amounts of data has grown, some interesting and very important characteristics of very large collections of data have been discovered. These are what is known as emergent phenomena. As far as I am aware, emergent behavior was first noticed in condensed matter physics. As Nobel prize-winning physicist P. W. Anderson pithily stated it, “More is different”.
Basically, macroscopic collections of matter behave in ways that cannot be predicted from looking at small numbers of molecules. New and different fundamental principles emerge when a large collection of molecules interact with one another. Emergent phenomena are now recognized in biological systems as well.
I have been struck by the similarity of what was noticed in solid state physics to what is happening as the properties and uses of huge collections of data become better understood. Just as “more is different” applied to matter, big data has characteristics that are not obvious from looking at small collections of facts. New and different fundamental ideas emerge. The impact on society is already large and will only grow.
At the New York Times. Steve Lohr has written an article that describe how big data is being used. He describes what the term “big data” means and how it is gathered:
Big Data is a shorthand label that typically means applying the tools of artificial intelligence, like machine learning, to vast new troves of data beyond that captured in standard databases. The new data sources include Web-browsing data trails, social network communications, sensor data and surveillance data.
The consequences of all this data being gathered and analyzed has both good and bad sides. On the bad side, our personal privacy has already been severely eroded. Powerful tools for massive government intrusion into our lives are becoming available.
Some of the good side is described in another New York Times article by Steve Lohr. He writes:
Computing may be on the cusp of another such wave. This one, many researchers and entrepreneurs say, will be based on smarter machines and software that will automate more tasks and help people make better decisions in business, science and government.
All in all, we may be on the cusp of another big change caused by technology. The information revolution is far from mature and has more surprises for us.
Did you enjoy this post? Why not leave a comment below and continue the conversation, or subscribe to my feed and get articles like this delivered automatically to your feed reader.
Comments
No comments yet.
Sorry, the comment form is closed at this time.