sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: Fwd: Class invariant violation
Date Sat, 22 Mar 2014 16:37:29 GMT
Did you properly configured path to your Hadoop configuration files as is described in (and
the file sqoop.properties):

http://sqoop.apache.org/docs/1.99.3/Installation.html#configuring-server

Sqoop will run all jobs on LocalJobRunner in case that proper Hadoop configs are missing.
The default behaviour of LocalJobRunner is that operation "start job" is blocking which might
explain the behaviour you are seeing.

Jarcec

On Fri, Mar 21, 2014 at 07:17:32PM -0700, Andy Srine wrote:
> Thanks Jarek. I added some paths to the libraries and now it works better.
> But after creating a connection and a job, the "start job --jid 1" just
> hangs. I think it is still some kind of a Hadoop issue, but no clues except
> the following lines in a loop in the sqoop.log.
> 
> 2014-03-21 19:08:59,218 DEBUG repository.JdbcRepositoryTransaction
> [org.apache.sqoop.repository.JdbcRepositoryTransaction.commit(JdbcRepositoryTransaction.java:84)]
> Tx count-commit: 1, rollback: false
> 
> 2014-03-21 19:08:59,218 DEBUG repository.JdbcRepositoryTransaction
> [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:103)]
> Tx count-close: 0, rollback: false
> 
> 2014-03-21 19:08:59,218 INFO  repository.JdbcRepositoryTransaction
> [org.apache.sqoop.repository.JdbcRepositoryTransaction.close(JdbcRepositoryTransaction.java:111)]
> Attempting transaction commit
> 
> 2014-03-21 19:09:56,510 TRACE core.PropertiesConfigurationProvider
> [org.apache.sqoop.core.PropertiesConfigurationProvider$ConfigFilePoller.run(PropertiesConfigurationProvider.java:172)]
> Checking config file for changes: server/bin/../conf/sqoop.properties
> 
> 2014-03-21 19:10:56,511 TRACE core.PropertiesConfigurationProvider
> [org.apache.sqoop.core.PropertiesConfigurationProvider$ConfigFilePoller.run(PropertiesConfigurationProvider.java:172)]
> Checking config file for changes: server/bin/../conf/sqoop.properties
> 
> 
> 
> 
> On Thu, Mar 20, 2014 at 8:09 PM, Jarek Jarcec Cecho <jarcec@apache.org>wrote:
> 
> > Yes, it actually does - it seems that you do not have all the dependencies
> > on the classpath. Did you correctly configured the common.loader?
> >
> >
> > http://sqoop.apache.org/docs/1.99.3/Installation.html#installing-dependencies
> >
> > Jarcec
> >
> > On Thu, Mar 20, 2014 at 05:24:46PM -0700, Andy Srine wrote:
> > > Thanks Vasanth and Jarek. Removing the jar doesn't seem to work, but I do
> > > think its the Hadoop setup on my laptop causing issues for SQOOP. I am
> > > trying to run the hadoop 2 binaries and it seem to work, but it also
> > warns
> > > me that its "Not a native build." The only other error message I see is
> > in
> > > the localhost.log. Error below. Not sure if this confirms it's a hadoop
> > > issue or gives you guys more clues?
> > >
> > > Mar 20, 2014 4:33:50 PM org.apache.catalina.core.StandardContext
> > > listenerStart
> > >
> > > SEVERE: Exception sending context initialized event to listener instance
> > of
> > > class org.apache.sqoop.server.ServerInitializer
> > >
> > > java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
> > >
> > > at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:165)
> > >
> > > at
> > >
> > org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.initialize(MapreduceSubmissionEngine.java:78)
> > >
> > > at org.apache.sqoop.framework.JobManager.initialize(JobManager.java:215)
> > >
> > > at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:53)
> > >
> > > at
> > >
> > org.apache.sqoop.server.ServerInitializer.contextInitialized(ServerInitializer.java:36)
> > >
> > > at
> > >
> > org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4206)
> > >
> > > at
> > org.apache.catalina.core.StandardContext.start(StandardContext.java:4705)
> > >
> > > at
> > >
> > org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)
> > >
> > > at
> > org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
> > >
> > > at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:601)
> > >
> > > at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:943)
> > >
> > > at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:778)
> > >
> > > at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:504)
> > >
> > > at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317)
> > >
> > > at
> > >
> > org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)
> > >
> > > at
> > >
> > org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> > >
> > > at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065)
> > >
> > > at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)
> > >
> > > at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
> > >
> > > at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
> > >
> > > at
> > org.apache.catalina.core.StandardService.start(StandardService.java:525)
> > >
> > > at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
> > >
> > >
> > > --
> > > Thanks,
> > > Andy
> > >
> > > On Thu, Mar 20, 2014 at 1:30 PM, Jarek Jarcec Cecho <jarcec@apache.org
> > >wrote:
> > >
> > > > Hi Andy,
> > > > the class invariant is known problem of log4j on tomcat and it's
> > nothing
> > > > you have to worry about. I would also recommend you to check out
> > catalina
> > > > log file (usually named catalina.$date.log).
> > > >
> > > > Based on the sqoop.log my guess is that you are either missing Hadoop
> > > > libraries or proper Hadoop configuration. You might want to check
> > that. You
> > > > should be able to find proper exception containing more details in one
> > of
> > > > the logs.
> > > >
> > > > Jarcec
> > > >
> > > > On Wed, Mar 19, 2014 at 05:47:18PM -0700, Andy Srine wrote:
> > > > > Hi Guys,
> > > > >
> > > > >
> > > > > I am new to SQOOP and am stuck at the 5 minute demo. This is with
> > > > > sqoop-1.99.3-bin-hadoop200 on a mac. The error I see is in the
> > > > > catalina.out. I searched the web and the workaround suggested
> > > > >
> > ("org.apache.catalina.loader.WebappClassLoader.ENABLE_CLEAR_REFERENCES =
> > > > > false") doesn't seem to help. The sqoop.log has no errors, but says
> > > > > "shutting down". Any ideas?
> > > > >
> > > > >
> > > > > SQOOP.LOG
> > > > >
> > > > > ---------
> > > > >
> > > > > 2014-03-18 18:25:04,796 INFO  framework.FrameworkManager
> > > > >
> > > >
> > [org.apache.sqoop.framework.FrameworkManager.initialize(FrameworkManager.java:159)]
> > > > > Submission manager initialized: OK
> > > > >
> > > > > 2014-03-18 18:25:04,807 INFO  mapreduce.MapreduceSubmissionEngine
> > > > >
> > > >
> > [org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.initialize(MapreduceSubmissionEngine.java:75)]
> > > > > Initializing Map-reduce Submission Engine
> > > > >
> > > > > 2014-03-18 18:25:04,846 INFO  core.SqoopServer
> > > > > [org.apache.sqoop.core.SqoopServer.destroy(SqoopServer.java:35)]
> > Shutting
> > > > > down Sqoop server
> > > > >
> > > > >
> > > > > CATALINA.OUT
> > > > >
> > > > > ------------
> > > > >
> > > > > Mar 18, 2014 6:25:04 PM org.apache.catalina.core.StandardContext
> > start
> > > > >
> > > > > SEVERE: Error listenerStart
> > > > >
> > > > > Mar 18, 2014 6:25:04 PM org.apache.catalina.core.StandardContext
> > start
> > > > >
> > > > > SEVERE: Context [/sqoop] startup failed due to previous errors
> > > > >
> > > > > Mar 18, 2014 6:25:04 PM org.apache.catalina.loader.WebappClassLoader
> > > > > clearReferencesJdbc
> > > > >
> > > > > SEVERE: The web application [/sqoop] registered the JDBC driver
> > > > > [org.apache.derby.jdbc.AutoloadedDriver40] but failed to unregister
> > it
> > > > when
> > > > > the web application was stopped. To prevent a memory leak, the JDBC
> > > > Driver
> > > > > has been forcibly unregistered.
> > > > >
> > > > > Mar 18, 2014 6:25:04 PM org.apache.catalina.loader.WebappClassLoader
> > > > > clearReferencesThreads
> > > > >
> > > > > SEVERE: The web application [/sqoop] appears to have started a thread
> > > > named
> > > > > [sqoop-config-file-poller] but has failed to stop it. This is very
> > likely
> > > > > to create a memory leak.
> > > > >
> > > > > Mar 18, 2014 6:25:04 PM org.apache.catalina.loader.WebappClassLoader
> > > > > checkThreadLocalMapForLeaks
> > > > >
> > > > > SEVERE: The web application [/sqoop] created a ThreadLocal with key
> > of
> > > > type
> > > > > [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@7b24c107])
> > and a
> > > > > value of type [org.apache.derby.iapi.services.context.ContextManager]
> > > > > (value
> > [org.apache.derby.iapi.services.context.ContextManager@63c39669])
> > > > > but failed to remove it when the web application was stopped. This
is
> > > > very
> > > > > likely to create a memory leak.
> > > > >
> > > > > Mar 18, 2014 6:25:04 PM org.apache.catalina.loader.WebappClassLoader
> > > > > checkThreadLocalMapForLeaks
> > > > >
> > > > > SEVERE: The web application [/sqoop] created a ThreadLocal with key
> > of
> > > > type
> > > > > [java.lang.ThreadLocal] (val
> > > > >
> > > > > ue [java.lang.ThreadLocal@7b24c107]) and a value of type
> > > > > [org.apache.derby.iapi.services.context.Context
> > > > >
> > > > > Manager] (value
> > > > > [org.apache.derby.iapi.services.context.ContextManager@2887d605])
> > but
> > > > > failed to remove i
> > > > >
> > > > > t when the web application was stopped. This is very likely to
> > create a
> > > > > memory leak.
> > > > >
> > > > > log4j: log4j called after unloading, see
> > > > > http://logging.apache.org/log4j/1.2/faq.html#unload.
> > > > >
> > > > > java.lang.IllegalStateException: Class invariant violation
> > > > >
> > > > >         at
> > > > > org.apache.log4j.LogManager.getLoggerRepository(LogManager.java:199)
> > > > >
> > > > >         at org.apache.log4j.LogManager.getLogger(LogManager.java:228)
> > > > >
> > > > >         at org.apache.log4j.Logger.getLogger(Logger.java:117)
> > > > >
> > > > >         at
> > > > >
> > > >
> > org.apache.sqoop.connector.jdbc.GenericJdbcImportInitializer.<clinit>(GenericJdbcImportInitializer.java:42)
> > > > >
> > > > >         at sun.misc.Unsafe.ensureClassInitialized
> > > > >
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Andy
> > > >
> >
> 
> 
> 
> -- 
> Thanks,
> Andy

Mime
View raw message