Hi Abe - Thanks for the response.  

I've updated my sqoop-env.sh file to have the following export vars

export HADOOP_HOME="/usr/local/Cellar/hadoop/2.4.0/libexec"
export HBASE_HOME="/usr/local/Cellar/hbase/0.98.1/libexec"
export HIVE_HOME="/usr/local/Cellar/hive/0.13.0/libexec"
export ZOOCFGDIR="/usr/local/etc/zookeeper"
export HCAT_HOME="/usr/local/Cellar/hive/0.13.0/libexec/hcatalog"

and to answer your question the hbase directory has the following file / directories

CHANGES.txt LICENSE.txt README.txt libexec

Now, when I try and run my import I get a new error (and the hbase table is actually created, just no data going into it)

The error is 

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected
at org.apache.sqoop.mapreduce.DelegatingOutputFormat.checkOutputSpecs(DelegatingOutputFormat.java:63)
at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:458)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:343)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:186)
at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:159)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:239)
at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:645)

I've read this is because of incompatible jar files but, like I said I'm a complete newb at all of this and if I could get a little more direction on what exactly I'm missing / is incompatible that would be fantastic. 

This is all on homebrew mac os x 10.9 hadoop 2.4 hbase .98 sqoop 1.4.4

On Thu, Jun 5, 2014 at 2:14 AM, Abraham Elmahrek <abe@cloudera.com> wrote:
Hey there,

What does the contents of the directory '/usr/local/Cellar/hbase/0.98.1' look like? The command 'find /usr/local/Cellar/hbase/0.98.1' should assist in figuring that out. Please paste the result of that command here.

The environment variables listed above are used in many different areas of Hadoop. I've typically seen them in startup scripts to resolve libraries and binaries. In this case, it's used to include jars in the classpath when you call Sqoop.

Are you able to import to HDFS?


On Wed, Jun 4, 2014 at 6:59 PM, Josh Millstein <joshua.millstein@gmail.com> wrote:

I've got hadoop / hbase / hive / set up and running (can create files on hdfs, run map reduce jobs, create a "table" in hbase and also in hive) on my mac with osx 10.9. I'm now trying to import data from a mysql table into sqoop (using query, not table-name etc). I am getting this error with this command.

sqoop import --connect jdbc:mysql:///joshLocal --username root --query "SELECT * FROM     BITLOG WHERE \$CONDITIONS" --split-by oozie_job.id --hbase-table bitlogTest --hbase-create-    table --column-family bitLogColumn

ERROR tool.ImportTool: Error during import: HBase jars are not present in classpath, cannot import to HBase!

I believe that all the export vars are correctly setup. I have the following in swoop-env.sh

export HADOOP_HOME="/usr/local/Cellar/hadoop/2.4.0"
export HBASE_HOME="/usr/local/Cellar/hbase/0.98.1"
export HIVE_HOME="/usr/local/Cellar/hive/0.13.0"
export ZOOCFGDIR="/usr/local/etc/zookeeper"
export HCAT_HOME="/usr/local/Cellar/hive/0.13.0/libexec/hcatalog"

one thing I did that gave a different message was to change the hbase home to point to HBASE_HOME/libexec/lib in swoop-env.sh. That gave me a

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/util/Tool
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)

error. I have seen some advice given that says I need to copy over hadoop security jar files to hbase's installation. I don't exactly know what files need to go over and if that is even an issue. The only reason I thought it might be is because java.security.SecureClassLoader is in the stacktrace.

I'm sorry if this is a really basic java question but I'm a complete novice with it.

One other even more basic java question. When we define HADOOP_HOME HBASE_HOME etc., what are we "telling" the other java programs that rely on that info. Are we saying "here is the executable java file" or are we saying "here are the jar file in the lib folder" . I don't quite understand what I should actually be pointing to because I don't know how that path is used.