On December 7, 2011, we held our fifth Strata Online Conference. This series of free web events brings together analysts, innovators and researchers from a variety of fields. Each conference, we look at a particular facet of the move to big data — from personal analytics, to disruptive startups, to enterprise adoption.
This time, we focused on how businesses are going to embrace big data, and where the challenges lie. It was a perfect opportunity to survey the attendees and get a glimpse into enterprise adoption of big data. Out of the roughly 350 attendees, approximately 100 agreed to give us their feedback on a number of questions we asked. Here are the results.
Some basic facts
While the attendees worked for a mix of commercial, educational, government, and non-profit companies, the vast majority (82%) worked for a commercial, for-profit company.
Most of the attendees’ organizations were also fairly large — more than half of them had 500 co-workers, and 22% of them had more than 10,000.
We used this demographic information to segment and better analyze the other three questions we asked.
Big data adoption and challenges
We then asked attendees about their journey to big data. Fewer than 20% of them already have a big data solution in place — which we clarified to mean some kind of massive-scale, sharded, NoSQL, parallel data query system that may employ interactivity and machine-assisted data exploration. More than a quarter said they have no plans at this time.
While it’s relatively early days for adoption, more than 60% of attendees said they were in the process of gathering information on big data and what it meant to them. This is a spurious result at best: we’re of course selecting an audience that wants to be an audience. Nevertheless, the volume of attendees and their feedback suggests that deployment is ramping up: if you’re a big data vendor, this is the time to be fighting for mindshare.
When it comes to actually deploying big data, companies have plenty of challenges. The big ones seem to be:
- Data privacy and governance.
- Defining what big data actually is.
- Integrating big data with legacy systems.
- A lack of big data skills.
- The cost of tools.
Analyzing a bit further
These results might be informative, but what we really want to know is how they correlate. After all, Strata is a data conference: we’d be remiss if we didn’t crunch things a bit!
First, we wondered whether there’s a relationship between the size of a company and the kinds of problems it’s experiencing with big data.
Our results suggest that governance and skill shortages are problems for larger companies, and that smaller businesses worry much less about data privacy and integrating legacy systems. Cost concerns come largely from mid-sized businesses.
Then we wondered whether adoption is tied to company size.
Among our attendees, smaller firms were ahead of the game: none of the companies larger than 500 employees said they had big data in place today.
We also found that educational, government, and NGO respondents didn’t list cost as a top concern, suggesting that they may have a tolerance for open-source or home-grown approaches.
Of course, the number of responses from these segments isn’t statistically significant, but it warrants further study, particularly for commercial offerings trying to sell outside the for-profit world.
Finally, we wondered whether the things a company worries about change as it goes from “just browsing” to “trying to build.”
Concerns do seem to shift over the course of adoption and maturity. Early on, companies struggle to define what big data is and worry about staffing. As they get closer to implementation, their attention shifts to legacy system integration. Once they have a system, talent shortages and a variety of other, more specific concerns emerge.
While not a hard-core study — respondents weren’t randomly selected, the number of responses within some segments isn’t statistically significant, and so on — this feedback does suggest that there’s a large demand for clear information on what big data is and how it’ll change business, and that as enterprises move to adopt these technologies they’ll face integration headaches and staffing issues.
The next free Strata Online Conference will be held on January 25. We’ll be taking a look at what’s in store for the upcoming Strata Conference (Feb 28-March 1 in Santa Clara, Calif).
Strata 2012 — The 2012 Strata Conference, being held Feb. 28-March 1 in Santa Clara, Calif., will offer three full days of hands-on data training and information-rich sessions. Strata brings together the people, tools, and technologies you need to make data work.
Related: