Why SQL is the natural language for data analysis

By -

Analytics is a must-have component of every corporate data warehousing and big data project. It is the core driver for the business: the development of new products, better targeting of customers with promotions, hiring of new talent and retention of existing key talent. Yet the analysis of especially “big data environments”, data stored and processed outside of classical relational systems, continues to be a significant challenge for the majority companies. According to Gartner, 72% of companies are planning to increase their expenditure on big data yet 55% state they don’t have the necessary skills to make use of it.

A report by Accenture and GE (How the Industrial Internet is Changing the Competitive Landscape of Industries) found that 87% of enterprises believe big data analytics will redefine the competitive landscape of their industries within the next three years and 89% believe that companies that fail to adopt a big data analytics strategy in the next year risk losing market share and momentum.

Additionally, a recent Cloudera webcast (Pervasive Analytics Gets Real ) noted that, while all businesses understand that analytics drive value, most organizations leverage only an average of 12% of their enterprise data for analytics. This implies that there is a significant amount of business value and opportunity that is being completely missed.

These type of market analysis highlights the huge analytical challenge that many businesses face today. While many companies are willing to invest in the critical area of big data technology to create new “data reservoirs”, for most of them, the same level of focus in relation to the analysis of these new data sources is missing. This means that many will struggle to find a meaningful way to analyze and realize the benefits of this vital investment strategy.

Many of the early adopters of big data are managing the analysis of their data reservoirs through the use of specialized programming techniques on the big data ecosystem, such as MapReduce. This is leading to data being locked inside proprietary data silos, making cross-functional cross-data store analysis either extremely difficult or completely impossible. IT teams are struggling to adapt complex data-silo-specific program code to support new and evolving analytical requirements from business users. In many cases these additional requirements force teams to implement yet more data processing languages. Overall, these issues greatly complicate the conversion of big data led discoveries into new business opportunities: driving the development of new products, capturing increased market share and/or launching into completely new markets. Most companies are searching for a single rich, robust, productive, standards driven language that can provide unified access over all their data and drive rich, sophisticated analysis.

Already, many companies are seeing the benefits of using SQL to drive analysis of their big data reservoirs. In fact, SQL is fast becoming the default language for big data analytics. This is because it provides a mature and comprehensive framework for both data access (so projects can avoid creating data silos) and rich data analysis.

12c

The objective of this series of articles, which will appear over the coming weeks, is to explain why SQL is the natural language for amy kind of data analysis including big data and the benefits that this brings for application developers, DBAs and business users.

Why SQL is so successful

Data processing has seen many changes and significant technological advances over the last forty years. However, there has been one technology, one capability that has endured and evolved: the Structured Query Language or SQL. Many other technologies have come and gone but SQL has been a constant. In fact, SQL has not only been a constant, but it has also improved significantly over time. What is it about SQL that is so compelling? What has made it the most successful language? SQL’s enduring success is the result of a number of important and unique factors:

  • Powerful framework
  • Transparent optimization
  • Continuous evolution
  • Standards based

Over the coming weeks I will explore each of these key points in separate posts and explain what makes SQL such a compelling language for data analysis. So stay tuned…..

 

Technorati Tags: , ,

Comments

Post a Comment

Popular posts from this blog

My query just got faster - brief introduction to 12.2 in-memory cursor duration temp tables

By -
Image
Image courtesy of wikipedia This post covers one of the new SQL performance enhancements that we incorporated into Database 12c Release 2 . All of these enhancements are completely automatic, i.e. transparent to the calling app/developer code/query. These features are enabled by default because who doesn’t want their queries running faster with zero code changes? So in this post I am going to focus on the new In-Memory “cursor duration” temporary table  feature. Let’s start by looking at cursor duration temp tables… Above image courtesy of wikimedia.org What is a cursor duration temp table? This is a feature that has been around for quite a long time. Cursor duration temporary tables (CDTs) are used to materialize intermediate results of a query to improve performance or to support complex multi step query execution. The following types of queries commonly use cursor duration temp tables: WITH Clause and parallel recursive WITH Grouping Sets S...
Keep reading

SQL Pattern Matching Deep Dive - Part 1

By -
Image
There has been quite a lot of interest in the new 12c MATCH_RECOGNIZE feature for SQL pattern matching. Therefore, I thought now would be the perfect time to start a series of quick deep dive posts that explain how SQL pattern matching works. Over the coming weeks I will cover the following topics in a series of posts. This is the start of a series of posts based on a presentation that I put together for the recent annual  BIWA conferenc e at Oracle HQ. The Oracle BI, DW and Analytics user community always puts on a great conference and this year was the best yet. You can download any or all of the presentations from this year’s conference by following this link . My pattern matching deep dive presentation started life about a year ago as a post covering some of the new keywords in the explain plan that are linked to pattern matching, see here . It has now expanded to cover a much wider range of topics. Firstly, a huge vote of thanks to my development team (Lei Sheng and Chun-Chie...
Keep reading

SQL Pattern Matching Deep Dive - Part 6, state machines

By -
Image
The obvious way to start this particular post is to pose a couple of simple questions: what is a state machine and why should you care? In general I would say that you don't need to know about or care about state machines. That's the beauty of using SQL for pattern matching. The MATCH_RECOGNIZE clause encapsulates all the deep technical modelling and processing that has to be performed to run pattern matching on a data set. However, there are times when it is useful, probably vital, that you understand what is going on behind the scenes and one of the most obvious situations is when backtracking happens. Therefore, the content covered in this post is a going to be a gently lead-in into my next post where I am going to discuss the concept of “backtracking”  and the dreaded  ORA-30009 error. Let’s start our voyage of discovery…when you attempt to run a SQL statement containing a MATCH_RECOGNIZE clause during the compilation phase we generate a finite state machine ba...
Keep reading