Faster Data Movement from Hadoop to Oracle Database

By -

The OTN team has just posted a great video interview with Oracle ACE Director Gwen Shapira on YouTube. Gwen talks about integrating Hadoop with Oracle Database using Oracle's Loader for Hadoop and she gives her views on when to use Hadoop vs. when to use Oracle Database.

If the embedded video above does not work for any reason then follow this link to the YouTube site: http://www.youtube.com/watch?v=tBpHAyTH6MU.

One of the interesting points that Gwen makes during the interview is the superior performance (5X faster) offered by our Hadoop loader compared to the other open source data loading tools. The Oracle Loader for Hadoop is part Oracle's Big Data Connectors suite of software and this suite of tools supports a wide range of Oracle targets. These connectors are designed and optimized to integrate Apache Hadoop with specific Oracle products including Oracle Database, Oracle Endeca Information Discovery and Oracle Data Integrator. The suite of connectors includes the following features/products:

FeatureBenefit
Oracle Loader for Hadoop

Uses MapReduce processing to format and load data efficiently into Oracle Database for analysis.
Oracle SQL Connector for HDFS

Enables Oracle Database to access data seamlessly from Hadoop Distributed File System allowing for SQL processing.
Oracle R Connector for Hadoop

Gives R users high performance native access to Hadoop Distributed File System (HDFS) and MapReduce programming framework.
Oracle Data Integrator Application Adapter for Hadoop

Simplifies data integration with Hadoop through easy to use interface

 

There is more information about the benefits of using the Loader for Hadoop to load data into your Oracle database in the recently published whitepaper High Performance Connectors for Load and Access of Data from Hadoop to Oracle Database.  For technical information about our Big Data Connectors, including links to download the software, visit the home page on OTN which is here: http://www.oracle.com/technetwork/bdc/big-data-connectors/overview/index.html. For more general information about big data goto www.oracle.com/bigdata

Comments

Post a Comment

Popular posts from this blog

My query just got faster - brief introduction to 12.2 in-memory cursor duration temp tables

By -
Image
Image courtesy of wikipedia This post covers one of the new SQL performance enhancements that we incorporated into Database 12c Release 2 . All of these enhancements are completely automatic, i.e. transparent to the calling app/developer code/query. These features are enabled by default because who doesn’t want their queries running faster with zero code changes? So in this post I am going to focus on the new In-Memory “cursor duration” temporary table  feature. Let’s start by looking at cursor duration temp tables… Above image courtesy of wikimedia.org What is a cursor duration temp table? This is a feature that has been around for quite a long time. Cursor duration temporary tables (CDTs) are used to materialize intermediate results of a query to improve performance or to support complex multi step query execution. The following types of queries commonly use cursor duration temp tables: WITH Clause and parallel recursive WITH Grouping Sets S...
Keep reading

SQL Pattern Matching Deep Dive - Part 1

By -
Image
There has been quite a lot of interest in the new 12c MATCH_RECOGNIZE feature for SQL pattern matching. Therefore, I thought now would be the perfect time to start a series of quick deep dive posts that explain how SQL pattern matching works. Over the coming weeks I will cover the following topics in a series of posts. This is the start of a series of posts based on a presentation that I put together for the recent annual  BIWA conferenc e at Oracle HQ. The Oracle BI, DW and Analytics user community always puts on a great conference and this year was the best yet. You can download any or all of the presentations from this year’s conference by following this link . My pattern matching deep dive presentation started life about a year ago as a post covering some of the new keywords in the explain plan that are linked to pattern matching, see here . It has now expanded to cover a much wider range of topics. Firstly, a huge vote of thanks to my development team (Lei Sheng and Chun-Chie...
Keep reading

SQL Pattern Matching Deep Dive - Part 6, state machines

By -
Image
The obvious way to start this particular post is to pose a couple of simple questions: what is a state machine and why should you care? In general I would say that you don't need to know about or care about state machines. That's the beauty of using SQL for pattern matching. The MATCH_RECOGNIZE clause encapsulates all the deep technical modelling and processing that has to be performed to run pattern matching on a data set. However, there are times when it is useful, probably vital, that you understand what is going on behind the scenes and one of the most obvious situations is when backtracking happens. Therefore, the content covered in this post is a going to be a gently lead-in into my next post where I am going to discuss the concept of “backtracking”  and the dreaded  ORA-30009 error. Let’s start our voyage of discovery…when you attempt to run a SQL statement containing a MATCH_RECOGNIZE clause during the compilation phase we generate a finite state machine ba...
Keep reading