Loading Profile...
Sqoop and Oracle
I'm trying to use sqoop in order to dump data from Oracle 10.2.
I've installed cloudera hadoop-0.18 and hadoop-0.20. I'm using Oracle Instant Client 11.2.
I've tried with sqoop from hadoop-0.18, and I get some Oracle errors. I think in this release sqoop didn't supported Oracle. Am I right?
Then I tried with sqoop from hadoop-0.20. First question: is it possible to use sqoop from 0.20 release to import data in 0.18 release?
Anyway, I get some errors. What am I doing wrong?
========================================================================
sqoop --connect jdbc:oracle:thin:@10.50.1.114:1523/ahz7r11s --driver oracle.jdbc.driver.OracleDriver --username ******* --password ******* --table ******* --split-by ******* --where "******* "
09/10/14 22:48:52 WARN sqoop.ImportOptions: Setting your password on the command-line is insecure. Consider using -P instead.
09/10/14 22:48:52 INFO sqoop.Sqoop: Beginning code generation
09/10/14 22:49:03 ERROR manager.SqlManager: Error executing statement: java.sql.SQLException: READ_COMMITTED and SERIALIZABLE are the only valid transaction levels
09/10/14 22:49:03 ERROR manager.SqlManager: Error executing statement: java.sql.SQLException: READ_COMMITTED and SERIALIZABLE are the only valid transaction levels
09/10/14 22:49:03 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
at org.apache.hadoop.sqoop.orm.ClassWriter.generateFields(ClassWriter.java:252)
at org.apache.hadoop.sqoop.orm.ClassWriter.generateClassForColumns(ClassWriter.java:701)
at org.apache.hadoop.sqoop.orm.ClassWriter.generate(ClassWriter.java:597)
at org.apache.hadoop.sqoop.Sqoop.generateORM(Sqoop.java:75)
at org.apache.hadoop.sqoop.Sqoop.importTable(Sqoop.java:87)
at org.apache.hadoop.sqoop.Sqoop.run(Sqoop.java:175)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.sqoop.Sqoop.main(Sqoop.java:201)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:185)
========================================================================
I've installed cloudera hadoop-0.18 and hadoop-0.20. I'm using Oracle Instant Client 11.2.
I've tried with sqoop from hadoop-0.18, and I get some Oracle errors. I think in this release sqoop didn't supported Oracle. Am I right?
Then I tried with sqoop from hadoop-0.20. First question: is it possible to use sqoop from 0.20 release to import data in 0.18 release?
Anyway, I get some errors. What am I doing wrong?
========================================================================
sqoop --connect jdbc:oracle:thin:@10.50.1.114:1523/ahz7r11s --driver oracle.jdbc.driver.OracleDriver --username ******* --password ******* --table ******* --split-by ******* --where "******* "
09/10/14 22:48:52 WARN sqoop.ImportOptions: Setting your password on the command-line is insecure. Consider using -P instead.
09/10/14 22:48:52 INFO sqoop.Sqoop: Beginning code generation
09/10/14 22:49:03 ERROR manager.SqlManager: Error executing statement: java.sql.SQLException: READ_COMMITTED and SERIALIZABLE are the only valid transaction levels
09/10/14 22:49:03 ERROR manager.SqlManager: Error executing statement: java.sql.SQLException: READ_COMMITTED and SERIALIZABLE are the only valid transaction levels
09/10/14 22:49:03 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
java.lang.NullPointerException
at org.apache.hadoop.sqoop.orm.ClassWriter.generateFields(ClassWriter.java:252)
at org.apache.hadoop.sqoop.orm.ClassWriter.generateClassForColumns(ClassWriter.java:701)
at org.apache.hadoop.sqoop.orm.ClassWriter.generate(ClassWriter.java:597)
at org.apache.hadoop.sqoop.Sqoop.generateORM(Sqoop.java:75)
at org.apache.hadoop.sqoop.Sqoop.importTable(Sqoop.java:87)
at org.apache.hadoop.sqoop.Sqoop.run(Sqoop.java:175)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.sqoop.Sqoop.main(Sqoop.java:201)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:185)
========================================================================
I’m confident
1
person has this question
I have this question, too!
Tell me when someone answers.
The more people who ask this question, the more it gets noticed.
The more people who ask this question, the more it gets noticed.
-
Inappropriate?Hi tore,
Sqoop 0.18 does not support Oracle; you must use Sqoop 0.20, with a MapReduce 0.20-based cluster. (Sqoop 0.20 is incompatible with MapReduce 0.18.)
In getting this working, my first suggestion is that you do not manually specify --driver. Specifying the driver class yourself uses the GenericConnectionManager, instead of using the Oracle-specific ConnectionManager path. Just use --connect jdbc:oracle:thin:blabla and Sqoop should use the correct OracleManager.
If that also fails to work, please enable DEBUG logging (see http://archive.cloudera.com/docs/sqoo...), run again, and reply with the more-detailed log messages to this forum.
Thanks,
- Aaron
1 person says
this answers the question
-
Inappropriate?WOW. Something happened. I think that sqoop read my Oracle table (but I think I messed up something with hadoop versions).
Now I have a newbie question.
I installed hadoop 0.18 and hadoop 0.20 from Cloudera distribution.
I tested 0.18 installation with hive. It was working great.
Then I want to import data with sqoop and then read them from hive. I think that I can't do this now, because there's no hive in 0.20 Cloudera distribution. Am I right?
Thank you. -
Inappropriate?No, Hive works with CDH 0.20. You should use alternatives to configure hadoop-0.20 as the active version of Hadoop; Hive will then use that instead.
EMPLOYEE
