The Growing World of Big Data

Dr. Kristian Kersting spends a lot of time stuck in traffic. Commuting from his home in Bonn to the Technische Universität (TU) Dortmund, where he is professor of computer science, Kersting has stretches of idle time during which he may call a colleague, send an email, or browse social media or a news website. In those moments Kersting is just one of more than six billion mobile phone users worldwide contributing information about their location, personal interests, and even health and purchasing habits to the largest cache of data ever collected about people and their behavior.

Just fifteen years ago, the words “Big Data” conjured images of massive server rooms vibrating with the thrum of cooling fans and twinkling with thousands of flashing lights. Today, the massive streams of data fed to all industries, from logistics and manufacturing to media and advertising, are increasingly generated from small, inexpensive devices. These include smartphones as well as custom sensors that analyze everything from atmospheric particles to the traffic congestion that slows Dr. Kersting’s commute.

The challenge of previous decades—gathering enough data to yield meaningful analyses—has been replaced by an equal and opposite challenge: how to best store, analyze, and utilize the almost unfathomably large trove of information generated by a growing number of people and devices on the planet each day.

A strong foundation of public and private support underlies Germany’s efforts to become a leader in big data analysis and deployment, including specialized centers for data research at universities throughout the country, as well as industry partners including IBM and Cisco, both of which have made considerable investments in accelerating big data R & D in Germany. The Berlin Big Data Center, Germany’s first standalone center for big data research, was founded in 2014 with a mission to feed the innovation pipeline and train a new generation of data scientists.

Teasing Insight from Data

Dr. Katharina Morik, head of the Collaborative Research Center 876 and professor of computer science at TU Dortmund, is one of thousands of researchers in Germany creating systems to tease out the most useful information from a sea of data and put the information to work in nearly every industry.

Such insights can only be gleaned by pairing big data with artificial intelligence — inextricably linked fields that both inform and advance each other, one providing raw information and the other the ability to extract, distill, and use that information in ways both pragmatic and novel.

Big data and AI come together through machine learning, which enables robotic and automated systems to continuously learn from real-time data feeds. The marriage of big data and AI has resulted in robots capable of assisting during surgeries and working therapeutically with developmentally disabled children, and has transformed the fields of logistics, healthcare, advertising, transportation, retail, manufacturing and more. Fueled by big data, scientists have created sophisticated machine learning algorithms capable of predicting human behavior. Such algorithms can be used for targeting advertisements and influencing purchasing decisions and can have broad impacts in fields such as global public health, where data collected by smartphones can allow scientists to detect and monitor infectious disease outbreaks and improve disaster preparedness.

Signal and Noise

Experts including Morik agree that making the most of big data relies in part on the integration of real-time analytics into devices themselves, a challenge that she and her colleagues, as well as others around the world, are quickly solving.

Even with revolutionary upgrades in bandwidth, those increases represent a massive burden on wireless networks. By integrating smart analytics directly into a device—a traffic sensor, for example, or a smart-container sensor used to track cargo shipments—only the information necessary for the task at hand, be it rerouting commuters or improving a shipping route, are transmitted through the network. “We can reduce the amount of data so we keep only the good signal, and not all the noise,” Morik explained.

Dr. Kersting believes researchers in Germany and worldwide are just beginning to tap the full potential of big data. “I compare the advances we’re making today to the early breakthroughs of microscopes and telescopes—innovations that allowed us to look at places we’d never seen,” he said. “This is the dream of humankind— to catch a glimpse of what wasn’t known before. We are bringing light to dark data.”