While the term “big data” is often thrown around by tech analysts and vendors, many people in IT are still unclear on what it means – and why they should care about it.
As with most newly formed tech buzzwords, there’s a lot of confusion over what “big data” actually means.
Half of organizations say they’re concerned about managing big data, according to a recent survey from vendor LogLogic and consulting firm Echelon One. However, 38% also said they don’t understand what big data is, and another 27% have just a partial understanding.
What does ‘big data’ mean?
So what is big data? It refers to the tools used to process the enormous amounts of information organizations are creating and storing now. The amount of data being created is growing exponentially. IBM, for instance, estimates that 90% of the data that exists right now was created in the past two years.
And a lot of that is data that can’t be easily organized using databases or other traditional IT tools. One common example is content made up of words pulled from social networking or news feeds. Vendors have popped up with tools to help businesses automatically gather and analyze that content.
So is big data a real IT issue or hype drummed up by vendors? Answer: a little bit of both. While the term is used in conjunction with products used to process those large sets of information, businesses in many industries would likely agree that they hold huge amounts of data that can’t be easily analyzed in traditional databases. That can include everything from streams of blog posts or news articles mentioning the company or its industry to data collected from RFID sensors.
The challenge of big data is turning those terabytes of information into structured data that can be analyzed and used to help the business. For a few examples, businesses can use big data to:
- analyze network logs to determine risks and improve security
- process website data to better understand visitor behavior, and
- study data about customer interactions to predict future behavior.
Who needs it?
As the term might imply, big data right now is mostly a worry for big companies. However, most businesses, even small ones, will find they have some streams of unstructured data being recorded. And cloud services will allow those small businesses to take advantage of big data technologies.
Companies of all sizes have begun using big data analytics, according to a 2011 survey conducted by TDWI. Among the 325 organizations surveyed, 34% said they use some type of advanced analytics with big data.
As the big data trend grows, IT managers at all organizations will need to plan for it. Experts recommend they look at what data the company has access to and whether advanced analysis would benefit the business.
It would help to meet with business leaders throughout the company to get a sense of what more they would like to get from their data.