In a world where consultancies offer a hefty list of big data services, businesses still struggle to understand what value big data actually brings and what its most efficient use can be. Before committing to big data initiatives, companies tend to search for their competitors’ real-life examples and evaluate the success of their endeavors. So, our data consultants decided to save a mile on the investigation path for those interested in big data usage and conducted secondary research based on 11 dedicated studies and reports published between 2015 and 2019. We also spiced our research up with the voices of well-known companies that shared their experience in big data adoption.
How companies of different sizes use big data
In this section, we’ll refer to the following segments: small, mid-sized, large and very large organizations. This categorization is based on the number of employees in a business or an institution:
- Small organizations (1-100 employees).
- Mid-sized organizations (101-1,000 employees).
- Large organizations (1,001- 5,000 employees).
- Very large organizations (more than 5,000 employees).
Very large organizations (5,000+ employees) are the main adopters of big data: 70% of such businesses and institutions report that they already use big data. [1] | Among all organization segments, very large organizations (5,000+ employees) are most interested in using big data for data warehouse optimization. [1] | |
43-45% of small, mid-sized and large organizations (fewer than 5,000 employees) already use big data, and all the segments are similarly open to the future use. [1] | Top 3 big data use cases for mid-sized, large and very large organizations (fewer than 5,000 employees) are data warehouse optimization, predictive maintenance and customer analytics. [1] | |
Of all organization segments, small organizations (up to 100 employees) are most interested in using big data for customer analytics. [1] | ||
How different industries use big data
Three industries most active in big data usage are telecommunications, healthcare, and financial services. [2]
Telecommunications
Top 3 use cases for telecoms are customer acquisition (93%), network optimization (85%), and customer retention (81%). [2] | ||
The telecommunications industry is an absolute leader in terms of big data adoption – 87% of telecom companies already benefit from big data, while the remaining 13% say that they may use big data in the future. [1] | Telecoms plan to enrich their portfolio of big data use cases with location-based device analysis (46%) and revenue assurance (45%). The optimization of prices, call centers and networks is also among the priorities. [2] | |
Healthcare
Almost 60% of healthcare organizations already use big data and nearly all the remaining ones are open to adopting big data initiatives in the future. [1] | Personalized treatment (98%), patient admissions prediction (92%) and practice management and optimization (92%) are the most popular big data use cases among healthcare organizations. [2] | |
Healthcare organizations plan to further expand their current big data usage with patient segmentation (31%) and clinical research optimization (25%). [2] | ||
Financial services
76% of financial services institutions are currently big data users. [1] | Financial services institutions use big data for customer analytics to personalize their offers (93%), as well as for risk assessment (89%), fraud detection (86%) and security threat detection (86%). [2] | |
Top 3 extra use cases that financial services institutions planned to add in 2017-2018 were location-based security analysis (66%), algorithmic trading (57%), and influencer analysis (37%). [2] | In 2017, the top area that financial services institutions were investing in was predictive analytics (38%). However, in 2018’s list of priorities, it fell to the second place (with 29%), giving way to a new leader – AI and machine learning. [3] |
Education
In education, the rate of big data adoption so far is the lowest – only 25% – when compared with telecommunications (87%), financial services (76%), healthcare (60%) and technology industries (60%). However, 67% of respondents don’t rule big data out as a future possibility. [1] |
Insurance
Insurers expect that big data can help most efficiently in the areas of pricing, underwriting and risk selection (92%), management decisions (84%), loss control and claim management (76%). [4] |
What use cases prevail for each big data technology
Hadoop use cases
Runtime environment for advanced analytics, memory for raw or detailed data, and data preparation and integration are top 3 use cases for Hadoop. [5] | Customer intelligence leads the list of Hadoop projects. [5] | |
While 39% of organizations use Hadoop as a data lake, the popularity of this use case will fall by 2% over the coming three years. [6] | ||
Spark use cases
Top 3 Spark-based projects are business/customer intelligence (68%), data warehousing (52%), and real-time or streaming solutions (45%). [7] | ||
55% of organizations use Spark for data processing, engineering and ETL tasks. [8] | 33% of companies use Spark in their machine learning initiatives. [8] | |
What value big data brings
Organizations value managing data in real time (70%) and accessing relevant data rapidly (68%) most. [2] | The biggest value that big data delivers are decreased expenses (49.2%) and newly created avenues for innovation (44.3%). [10] | |
48.4% of organizations assess their results from big data as highly successful. [10] | While 69.4% of organizations started using big data to establish a data-driven culture, only 27.9% report successful results. [10] | |
84% of enterprises invest in advanced analytics to support improved business decision making. [11] | Advanced analytics (36%), improved customer service (23%) and decreased expenses (13%) are top 3 priorities for investing into big data and AI. [11] |
The history of big data usage in numbers
Big data adoption is constantly growing: the number of companies using big data has dramatically increased from just 17% in 2015 to 53% in 2017. In 2018, 97.2% of companies indicated that they were investing in big data and AI. [1], [11] | ||
In 2015-2017, companies named data warehouse optimization as #1 big data use case, while in 2018 the focus shifted to advanced analytics. [1], [11] | Predictive maintenance has appeared on companies’ radars only in 2017 and has got straight to top 3 big data use cases. [1] | |
Within 2015-2017, sales and marketing (in every industry) were the areas where data and analytics brought significant or fundamental changes. [9] | ||
Big data stories of big companies
Check what Walmart, PepsiCo, JPMorgan Chase, Rolls-Royce, and Uber have to say about their big data experience.
“Over time, the need for more insights has resulted in over 100 petabytes of analytical data that needs to be cleaned, stored, and served with minimum latency through our Hadoop-based big data platform. Since 2014, we have worked to develop a big data solution that ensures data reliability, scalability, and ease-of-use, and are now focusing on increasing our platform’s speed and efficiency.”
Reza Shiftehfar, Hadoop Platform Team Leader at Uber
“Walmart relies on big data to get a real-time view of the workflow in the pharmacy, distribution centers and throughout our stores and e-commerce.”
“[About their big data platform Pep Worx] We were able to launch the product [Quaker Overnight Oats] using very targeted media, all the way through targeted in-store support, to engage those most valuable shoppers and bring the product to life at retail in a unique way. These priority customers drove 80% of the product’s sales growth in the first 12 weeks after launch.”
Jeff Swearingen, Senior Vice President of Marketing at PepsiCo
“Artificial intelligence, big data and machine learning are helping us reduce risk and fraud, upgrade service, improve underwriting and enhance marketing across the firm.”
Jamie Dimon, Chairman and Chief Executive Officer at JPMorgan Chase
“We have huge clusters of high-power computing which are used in the design process. We generate tens of terabytes of data on each simulation of one of our jet engines. We then have to use some pretty sophisticated computer techniques to look into that massive dataset and visualize whether that particular product we’ve designed is good or bad. Visualizing big data is just as important as the techniques we use for manipulating it.”
Paul Stein, Chief Scientific Officer at Rolls-Royce
A quick final look at big data
The findings of our secondary research are in line with our hands-on experience: businesses increasingly adopt big data, and, overall, they are highly satisfied with the results of their initiatives. Though the majority of big data use cases are about data storage and processing, they cover multiple business aspects, such as customer analytics, risk assessment and fraud detection. So, each business can find the relevant use case to satisfy their particular needs.
References
[1] 2017 Big Data Analytics Market Study by Dresner Advisory Services
[2] IDC/Dell EMC, Big Data: Turning Promise Into Reality
[3] Survey Report 2018: Big Data Analytics for Financial Services
[4] 2016 Predictive Modeling Benchmark Survey (U.S.) by Willis Towers Watson
[5] Business Application Research Center, Why Companies Use Big Data Analytics
[6] TDWI, Hadoop for the Enterprise
[7] Databricks, Apache Spark Survey 2016 Report
[8] Apache Spark Market Survey by Taneja Group
[9] McKinsey, Analytics comes of age