People, organizations and things will continue to generate ever-larger volumes of data in the months and years ahead, but expect to see some changes in how that information will be put to use, not just by humans but by artificial intelligence (AI) as well.
A recent survey by the analyst firm Gartner Inc., for instance, found that while big data technologies are seeing more overall investment, that spending will be done by fewer companies over the next couple years. And that trend is likely to lead to a growing divide between organizations that “know” and those that don’t, according to IDC.
Big data users will also need to ask deeper questions about the kinds of information they are and aren’t collecting and analyzing. Beyond that, they’ll need to honestly consider their willingness to act on results that might conflict with old ways of doing business, according to Gartner.
Need for Unbiased Models, Openness to New Ideas
That last lesson is one that Gartner research director Robert Hetu pointed out shortly after the unexpected results of the U.S. presidential election in November.
“The failure to accurately predict the outcome of the elections has caused some backlash against big data and algorithms,” Hetu wrote in a Nov. 10 blog post. “This is misguided. The real issue is failure to build unbiased models that will identify trends that do not fit neatly into our present understanding. This is one of the most urgent challenges for big data, advanced analytics and algorithms.”
Identifying trends through big data analytics is one thing, but persuading people to accept the reality of those trends and act on them is something else entirely, Hetu noted. Before the election, for example, there were a few analysts who recognized the outcome the data was pointing to, but they were “repeatedly laughed at and disregarded,” he said.
This is the foundation of the problem, according to Hetu. “Organizations must foster environments where new ideas are embraced and safely explored,” he said. “This is how we will grow the convergence of things we know.”
While people need to be open to new ideas, organizations will also have to develop better and smarter algorithms and AI systems to automate responses to insights revealed through big data analytics. That’s especially important as technologies, such as cognitive computing, get better at teasing valuable findings out of unstructured as well as structured data.
More Machine Learning, Team-Based Data Science
“Near-term opportunities for cognitive systems are in industries such as banking, securities and investments, and manufacturing,” IDC program director Jessica Goepfert said in an October statement about a report on global cognitive computing and AI spending. “In these segments, we find a wealth of unstructured data, a desire to harness insights from this information, and an openness to innovative technologies.”
In its report, IDC predicted that healthcare and manufacturing will be the biggest drivers of cognitive computing and AI revenues between now and 2020, while the education sector will also invest heavily in such technologies.
Earlier this month, Tony Baer, principal analyst in information management at Ovum, predicted that machine learning in particular “will be the biggest disruptor for big data analytics in 2017.” That trend will also make it increasingly important for organizations to treat data science as a “team sport,” he added.
IBM’s big data evangelist James Kobielus has also predicted many changes to how businesses and other organizations will put data scientists to work in the coming year. According to Kobielus, data science skills will become increasingly important for traditional IT professionals, and that more data scientists will work within cloud-based development environments and find training through crowdsourcing. Additionally, he predicted an increase in the number of “self-taught citizen data scientists” that will work alongside traditional experts on high-priority enterprise 
projects.
“People who can design AI-powered products that combine robotics, embodied cognition, IoT fog computing, deep learning, predictive analytics, emotion analytics, geospatial contextualization, conversational engagement and wearable form factors will be in hot demand,” Kobielus said. “We are at the start of an amazing period of mind-blowing innovation in the consumer and industrial economy. Every human artifact is being retrofitted with AI capabilities or being designed from the ground up either to function as intelligent assistants or to handle many chores autonomously that their owners can’t or prefer not to do themselves.”
