Big data Analytics – A disruptive technology !!

Big data is the most talked term these days in the analytics world. It will have big transformative impact on all the aspects of the business.

Most of the companies now have realized that there is a huge competitive advantage in analyzing the humongous data quickly & effectively for future insights.

Big data analytics is the disruptive technology bringing the 4^th aspect of Value to the already published TDWI’s 3Vs – Volume, Velocity & Variety.

• It enables business users to process every granular bit of data in quicker way removing the traditional need for sampling & then applying the models
• It encourages an investigative approach in users for data analysis since they get access to whole data
• It can reveal insights hidden in the data, which were previously too costly due to large data movements
• As per Gartner report, Big data is priority of SMB & it will drive $232 billion in spending through 2016.

Some of the technology platforms which are used for big data:

• Distributed file based: Hadoop-MapReduce (Cloudera, Hortonworks, MapR)
• Appliance based: Greenplum, IBM Puredata(Netezza), Oracle, Teradata
• Columnar databases: HP Vertica,  ParAccel, 1010data
• In-memory databases/tools: SAP Hana, Qlikview, Tableau
• Non relational/NoSQL: Cassandra, MongoDB, Splunk, Hbase

Hadoop is on the top of the list of technologies used for dealing with Big data due to its ultra high scalability & low cost compared to other platforms.  It is a suite of products linked together, which breaks up the large datasets into smaller chunks on commodity servers, and data processing is done in a distributed cluster environment to quickly return the results.

Some of the probable Big data use case in various industries:

Insurance: Collecting data from monitoring devices fixed in cars & providing the personalized insurance policies based on driving habits, Underwriting price optimization for insurance products, Claims fraud with social network link analysis.

Retail: Market basket analysis for entire merchandising instead of sample data, Sentiment analysis based on social media for improving brand perception, customer service, competition analysis, Customer & market segmentation, Weblog analysis for customer behavior.

Banking & Finance: Fraud detection with entire history data for better detection, Trade surveillance in capital markets, More accurate risk score to customers, Text mining on call center data.

Healthcare: Improve patient care with analyzing electronic Health Records (EHR) & reduce insurance payer costs, Reduce hospital readmission rates by analyzing information from discharged cards.

Manufacturing: Forecasting warranty costs & detecting issues in spare parts of finished goods, text mining to understand the complaints from customers for product improvements.

Because of nascent stage immaturity of Big data initiatives, there are many views of what is it & how it can be applied.  Organizations need to focus on Big data processing, while avoiding the movement of large volumes of data which is very costly.

Big data help make better decisions – faster, more efficiently with higher quality.