Big Data: Combine Structured with Unstructured Data for Best Results

With unstructured data, knowing what questions to ask is often difficult, but with the help of structured data (the relational data), one may develop a better process.

Unstructured data, sometimes categorized into a subcategory of ‘Big Data’ is comprised of word files, email, social media, video, audio, HTML, XML, PDF, Excel, image files and more.

Manufacturers, services companies, and government organizations are all taking advantage of unstructured data to improve their organizations’ responsiveness. This has resulted in hundreds of millions of dollars in cost savings and additional revenue for those companies.

Examples of Unstructured Data

Customer Interactions

Social Media

Operational/Process Documentation

News and Information

Call center recordings and logs

Customer surveys / feedback forms

Customer emails Field service reports Warranty claims Sales call logs

Reviews/ratings on community websites

Internet message board and forum posts

Twitter posts Blog posts Facebook entries

Voice of the Customer documentation

Documentation related to regulatory requirements

Meeting transcripts

Contracts and service level agreements

Financial analyses/reports

Documents, spreadsheets, and emails

News/RSS feeds Competitive websites

Third-party information services

Specifically, there are many insights that decision makers gain by unifying both unstructured and structured data in their analysis. Some of the major insights are: the status of customer opinion/sentiment, competitive intelligence, early detection of quality issues, ideas on new products or product enhancements.

Potential Insights


Typical Unstructured Data Sources

Customer opinion/sentiment

Internet message boards and forums, online communities such as Twitter and Facebook, blogs, customer surveys

Competitive intelligence

News feeds, information services, regulatory filings, field marketing reports, sales engagement reports, competitive websites, online communities, blogs

Product/service enhancements or new product/service ideas

Call center logs, customer surveys

Product/service quality, including early warning on quality issues

Internet message boards and forums, online communities, blogs

Industry-specific insights, such as:

            ·  Optimized treatment guidelines (healthcare)

            ·  Equity analysis/investment portfolio recommendations (financial services)

            ·  Scientific /legal discovery (pharmaceutical)

            ·  Optimal project staffing plans (consulting services)

            ·  Physician notes in healthcare documentation/claims

            ·  Customer and investor sentiment as shown on Internet message boards and forums, news feeds

            ·  Regulatory filings, patent filings, papers in scientific publications

            ·  Skills and experience descriptions in staff resumes

While unstructured data has great value, the real value of unstructured data is found when it is combined with the core structured data.

For example, the realization that customer sentiment is negative is even more valuable when combined with the recent sales transactions and customer information to understand how that sentiment has affected the demographics of the audience or the mix of products and services that was purchased.

A disjointed approach leads to a multi-stage process to handle and/or analyze data, making it virtually impossible to include rapidly changing data in an analysis.

One is going to need a hybrid platform a hybrid search-analytic platform that unifies structured and unstructured data and provides self-service exploration capability. You are going to find that Oracle Endeca  provides the platform designed to complement traditional BI platforms, which are typically tuned to serve reporting demands on structured data only.

Unifying unstructured data with structured data, therefore, can often be accomplished through a complementary architecture that is neither difficult nor expensive to implement.

With these changes, organizations provide their decision makers clear visibility into market dynamics, customer sentiment, and internal business operations.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s