manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Apache ManifoldCF job stuck up
Date Wed, 09 Jul 2014 17:21:14 GMT
Hi Lalit,

If you were using the multiprocess-file-example before, and you did not
shut things down cleanly, you should do: ./lock-clean.sh first.

Karl



On Wed, Jul 9, 2014 at 1:10 PM, lalit jangra <lalit.j.jangra@gmail.com>
wrote:

> Hi Karl,
>
> I think its my bad luck today. :(
>
> I tried the way you just suggested but still getting same error and job
> not able to run. What i did.
>
> 1. Created a brand new DB schema using ./executecommand.sh
> org.apache.manifoldcf.core.DBCreate username password
> 2. Initialized DB schema using ./initialize.sh
> 3. Start agent process A using ./start-agents.sh
> 4. Updated tomcat/bin/setenv.sh file to add
> "-Dorg.apache.manifoldcf.configfile" parameter to point to properties.xml
> under dist/multiprocess-file-example.
> 5. Started Tomcat.
> 6. Deployed all wars except combined war.
> 7. Started job
>
> Job still not able to take off with same error in logs.
>
> ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception tossed:  Service
> 'A' of type  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is
> already
> active
>
> I think its not my day today.
>
> Regards.
>
>
>
> On Wed, Jul 9, 2014 at 4:02 PM, Karl Wright <daddywri@gmail.com> wrote:
>
>> Hi Lalit,
>>
>> In theory the combined war will run with external locking, it's just that
>> nobody ever thought that would be how people would use it, and therefore it
>> hasn't been tested etc.
>>
>> I would suggest that for a development setup, you use the Jetty simple
>> example.  For a production setup, you should adopt the standard distributed
>> arrangement as follows:
>>
>> - a deployed mcf-crawler-ui war
>> - a deployed mcf-authority-service war
>> - an (optional) deployed mcf-api-service war
>> - one or more agents processes
>>
>> This is how it is done in both kinds of multiprocess examples, and is
>> described thoroughly in the how-to-build-and-deploy document online.
>>
>> Thanks,
>> Karl
>>
>>
>>
>> On Wed, Jul 9, 2014 at 10:56 AM, lalit jangra <lalit.j.jangra@gmail.com>
>> wrote:
>>
>>> Thanks Karl,
>>>
>>> My idea was as its name suggests mcf-combined-war as i was trying to
>>> take advantage of agent process inside war. My assumption is wrong here as
>>> i can see here.
>>>
>>> In this case, which war should i use - mcf-crawler-ui.war? If i use this
>>> or any of other wars, then should i start agent process?
>>>
>>> Please suggest.
>>>
>>> Regards.
>>>
>>>
>>> On Wed, Jul 9, 2014 at 3:48 PM, Karl Wright <daddywri@gmail.com> wrote:
>>>
>>>> Hi Lalit,
>>>>
>>>> Probably it is not a database issue, but how locking is configured.
>>>> Typically the combined war is run in a non-clustered environment with no
>>>> external synchronization required (as it is in the example I pointed you
>>>> at).  We don't currently have an example that uses combined war with
>>>> external locking -- that would not make much sense.  Can you tell me why
>>>> you are trying to do it that way?
>>>>
>>>> Karl
>>>>
>>>>
>>>>
>>>> On Wed, Jul 9, 2014 at 10:46 AM, lalit jangra <lalit.j.jangra@gmail.com
>>>> > wrote:
>>>>
>>>>> Hi Karl,
>>>>>
>>>>> if i try ./start-combined.sh, the job runs and i could see it
>>>>> finishing gracefully with updating index in solr.
>>>>>
>>>>> I assume there is some DB issue here? Any suggestions?
>>>>>
>>>>> Regards.
>>>>>
>>>>>
>>>>> On Wed, Jul 9, 2014 at 3:17 PM, Karl Wright <daddywri@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Lalit,
>>>>>>
>>>>>> Please do the following: In an unmodified dist/example directory,
run
>>>>>> the start-combined script.  Open the UI, create the connections,
and try to
>>>>>> run the job.  Does it work for you?
>>>>>>
>>>>>>
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jul 9, 2014 at 10:09 AM, lalit jangra <
>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>
>>>>>>> Thanks Karl,
>>>>>>>
>>>>>>> I have reconfigured everything and this time i am starting tomcat
>>>>>>> and deploying mcf-combined-war. I created output connection,
repository
>>>>>>> connection & job. Still when i start my job, it does not
go ahead and stuck
>>>>>>> at starting up status.
>>>>>>>
>>>>>>>
>>>>>>> Regards.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jul 9, 2014 at 12:49 PM, Karl Wright <daddywri@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Lalit,
>>>>>>>>
>>>>>>>> You do not use combined.war AND an agents process at the
same
>>>>>>>> time.  The combined war INCLUDES the agents process.
>>>>>>>>
>>>>>>>> Karl
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jul 9, 2014 at 6:18 AM, lalit jangra <
>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Karl,
>>>>>>>>>
>>>>>>>>> As per your suggestion, i built a new MCF 1.5.1 instance,
con from
>>>>>>>>> source , configured it and pointed to local postgresql
DB in
>>>>>>>>> properties.xml  under /multiprocess-file-example. Then
i tool a brand new
>>>>>>>>> tomcat and configured for MCF in it in setenv.sh for
properties.xml using
>>>>>>>>> "-Dorg.apache.manifoldcf.configfile" argument and deployed
mcf-combined.war
>>>>>>>>> using app manager.
>>>>>>>>>
>>>>>>>>> First i initialized DB using ./initialize.sh command
& the moment
>>>>>>>>> i started agent using ./start-agents.sh command under
>>>>>>>>> /multiprocess-file-example, i got the error, even when
tomcat is not
>>>>>>>>> started at this time. I created a job which never runs
& get stuck.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception
tossed:
>>>>>>>>>  Service 'A' of type
>>>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
is already
>>>>>>>>>   active
>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
Service
>>>>>>>>> 'A'  of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
is
>>>>>>>>>  already active
>>>>>>>>>
>>>>>>>>> But if i try to run jetty bundle using start.jar under
example, no
>>>>>>>>> such error but it does not fulfil my requirements.
>>>>>>>>>
>>>>>>>>> Regards.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jul 9, 2014 at 10:03 AM, Karl Wright <daddywri@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> So, lalit, if you run the multiprocess example without
any
>>>>>>>>>> changes, do
>>>>>>>>>> you see this?  I don't.
>>>>>>>>>>
>>>>>>>>>> Karl
>>>>>>>>>>
>>>>>>>>>> Sent from my Windows Phone
>>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: lalit jangra
>>>>>>>>>> Sent: 7/9/2014 12:17 AM
>>>>>>>>>> To: Karl Wright
>>>>>>>>>> Cc: user@manifoldcf.apache.org
>>>>>>>>>> Subject: Re: Apache ManifoldCF job stuck up
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks Karl,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I am currently running only single agent process
on single machine
>>>>>>>>>> without clustering. I have two environments and i
could see this
>>>>>>>>>> issue
>>>>>>>>>> coming up at both places.
>>>>>>>>>>
>>>>>>>>>> While trying to start agent, i could see below error.
When i am
>>>>>>>>>> trying
>>>>>>>>>> to start agent, it says below error and exits. But
there is no
>>>>>>>>>> agent
>>>>>>>>>> process already running.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [root@server1 multiprocess-file-example]# ./start-agents.sh
&
>>>>>>>>>>
>>>>>>>>>> [1] 5020
>>>>>>>>>>
>>>>>>>>>> [root@server1 multiprocess-file-example]# Running...
>>>>>>>>>>
>>>>>>>>>> Configuration file successfully read
>>>>>>>>>>
>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>>>>>>>>>> Service 'A'
>>>>>>>>>> of type 'AGENT' is already active
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.agents.AgentRun.doExecute(AgentRun.java:54)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.agents.BaseAgentsInitializationCommand.execute(BaseAgentsInitializationCommand.java:37)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.agents.AgentRun.main(AgentRun.java:93)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [1]+  Exit 1                  ./start-agents.sh
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> If i am able to successfully start agent using ./start-agents.sh,
>>>>>>>>>> still it throws same error but i have no other process
running.
>>>>>>>>>>
>>>>>>>>>> Regards.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Jul 8, 2014 at 2:38 PM, Karl Wright <daddywri@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi lalit,
>>>>>>>>>>
>>>>>>>>>> This occurs when you have more than one agents process
with the
>>>>>>>>>> same
>>>>>>>>>>  process id using the same shared file system directory
/
>>>>>>>>>> zookeeper
>>>>>>>>>>  cluster.  There is no other way it can occur.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>  Karl
>>>>>>>>>>
>>>>>>>>>> Sent from my Windows Phone
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>>  From: lalit jangra
>>>>>>>>>>  Sent: 7/8/2014 8:38 AM
>>>>>>>>>>  To: user@manifoldcf.apache.org
>>>>>>>>>>  Subject: Re: Apache ManifoldCF job stuck up
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks Karl,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I have tried steps you suggested and it worked on
one instance.
>>>>>>>>>>
>>>>>>>>>> But on another instance i am still not able to resolve
this issue.
>>>>>>>>>>  Along with steps you mentioned, i tried recreating
DB instance
>>>>>>>>>> again,
>>>>>>>>>>  setting up new MCF instance , cleaning locks and
then starting
>>>>>>>>>> agents
>>>>>>>>>>  first & then tomcat. But issue still persists.
>>>>>>>>>>
>>>>>>>>>> If i try to run ./start-agents.sh, i get this error
for agent A.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception
tossed:
>>>>>>>>>>  Service 'A' of type
>>>>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
is
>>>>>>>>>> already
>>>>>>>>>>  active
>>>>>>>>>>
>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>>>>>>>>>> Service 'A'
>>>>>>>>>>  of type
>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
is
>>>>>>>>>>  already active
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> But if i try to run ./start-agents-2.sh, i see similar
error but
>>>>>>>>>> agent B
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception
tossed:
>>>>>>>>>>  Service 'B' of type
>>>>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
is
>>>>>>>>>> already
>>>>>>>>>>  active
>>>>>>>>>>
>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>>>>>>>>>> Service 'B'
>>>>>>>>>>  of type
>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
is
>>>>>>>>>>  already active
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Regards.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 7, 2014 at 4:55 PM, Karl Wright <daddywri@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hi Lalit,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> If you are using file synchronization, you cannot
expect MCF to
>>>>>>>>>> clean
>>>>>>>>>>  up itself unless you shut it down cleanly.  You
should be either
>>>>>>>>>> using
>>>>>>>>>>  ^C or plain kill, NEVER kill -9.  kill -9 will leave
dangling
>>>>>>>>>> locks.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> To clean up dangling locks:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - shut ALL manifoldcf processes and web apps down
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - run the lock-clean script
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - start up the processes again
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  Zookeeper synchronization, by the way, does not
have this kind
>>>>>>>>>> of problem.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>  Karl
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Jul 7, 2014 at 11:49 AM, lalit jangra <
>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I configured MCF 1.5.1 to run with Postgresql DB
and tomcat 7.
>>>>>>>>>>  Initially i created all connections, created a an
alfresco job
>>>>>>>>>> and it
>>>>>>>>>>  all worked fine.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Next for updates, i stopped, tomcat and agent process
running.
>>>>>>>>>> Then i
>>>>>>>>>>  updated CmisRepositoryConnector.java with my own
code and run
>>>>>>>>>> "ant
>>>>>>>>>>  build" at root of MCF. It updated all code and jars
file.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Also properties.xml is reset under /dist/multiprocess-file/example
>>>>>>>>>>  which i updated to connect to Postgresql DB and
logging
>>>>>>>>>>  configurations.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I started tomcat then agent process.  Finally i started
job to
>>>>>>>>>> crawl
>>>>>>>>>>  alfresco but it got stuck and moving on. I checked
into
>>>>>>>>>>  /dist/multiprocess-file/example/logs/manifoldcf.log
file and
>>>>>>>>>> could see
>>>>>>>>>>  below error.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ERROR 2014-07-07 16:09:04,936 (Agents thread) - Exception
tossed:
>>>>>>>>>>  Service '' of type
>>>>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
is
>>>>>>>>>> already
>>>>>>>>>>  active
>>>>>>>>>>
>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>>>>>>>>>> Service ''
>>>>>>>>>>  of type
>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
is
>>>>>>>>>>  already active
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270)
>>>>>>>>>>
>>>>>>>>>>         at
>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon$AgentsThread.run(AgentsDaemon.java:208)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I created another job but that got stuck too.Is DB
got corrupt
>>>>>>>>>> due to rebuild?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Also is this the right way to build MCF (I hope its
correct way).
>>>>>>>>>> Now
>>>>>>>>>>  what should i do to fix this issue?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Please help.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>  Lalit Jangra.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>  Regards,
>>>>>>>>>>  Lalit Jangra.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Regards,
>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Regards,
>>>>>>>>> Lalit Jangra.
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Regards,
>>>>>>> Lalit Jangra.
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Regards,
>>>>> Lalit Jangra.
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Regards,
>>> Lalit Jangra.
>>>
>>
>>
>
>
> --
> Regards,
> Lalit Jangra.
>

Mime
View raw message