manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Apache ManifoldCF job stuck up
Date Thu, 10 Jul 2014 09:12:20 GMT
bq. So the only difference i could observe is DB which is HSQL vs Postgres,
rest everything is same.

Right, exactly.  Only this time we have some certainty that the
configuration is otherwise correct for synchronization and processes.

The next step is the following:

- START with the same multiprocess-file-example directory and processes you
verified yesterday
- Stop all MCF processes/web applications that are using that
multiprocess-file-example properties file:
  - Either stop the Tomcat process, or better yet go to the "admin" tomcat
application and stop all three MCF web applications
  - Go to the start-agents.sh window and ^C that process
  - Go to the start-database.sh window and ^C that process
- Change the properties.xml file under multiprocess-file-example to point
to the PostgresDB instance you want it to use.  This instance CANNOT be in
use by any other MCF cluster or process.
  - Replace the following THREE properties in properties.xml with your
PostgreSQL equivalent, as follows:

>>>>>>
  <property name="org.apache.manifoldcf.databaseimplementationclass"
value="org.apache.manifoldcf.core.database.DBInterfacePostgreSQL"/>
  <property name="org.apache.manifoldcf.dbsuperusername"
value="<something>"/>
  <property name="org.apache.manifoldcf.dbsuperuserpassword"
value="<something>"/>
<<<<<<

You may also choose to set these properties, to avoid collision with other
instances.  You can use whatever value you like for these, within what
PostgreSQL allows:

org.apache.manifoldcf.database.name
org.apache.manifoldcf.database.username
org.apache.manifoldcf.database.password

- Run the initialize script:
  - cd multiprocess-file-example ; initialize.sh .  If this works, you have
the database configured properly.  If it fails, look at the parameters you
provided and figure out what is wrong before proceeding

- start the ManifoldCF processes and web applications:
  - cd multiprocess-file-example ; start-agents.sh
  - go to the tomcat instance, and start it, or start all the web
applications

- create connections and jobs, and try a crawl.  Let me know how it works.

Karl



On Thu, Jul 10, 2014 at 12:25 AM, lalit jangra <lalit.j.jangra@gmail.com>
wrote:

> Thanks a lot Karl,
>
> Yes, it works perfectly as you suggested in last mail.
>
> So the only difference i could observe is DB which is HSQL vs Postgres,
> rest everything is same.
>
> Please suggest.
>
> Regards.
>
>
>
>
> On Wed, Jul 9, 2014 at 11:35 PM, Karl Wright <daddywri@gmail.com> wrote:
>
>> Hi Lalit,
>>
>>
>> Sorry, I thought my instructions were clear.  For this exercise, I want
>> you to use ONLY the example under dist/multiprocess-file-example.  DO NOT
>> use the example under dist/example.
>>
>> Here are the instructions again, clarified and fleshed out:
>>
>>
>> - Start with a fresh build of MCF
>> - Go to the multiprocess-file example and start that in Jetty, with NO
>> changes:
>>   - cd multiprocess-file-example
>>   - start-database.sh
>>   - initialize.sh
>>   - start-agents.sh
>>   - in a separate window: cd multiprocess-file-example ; start-webapps.sh
>>   - create connections and job, see if it runs
>>  - If that doesn't work, open a ticket.  If that DOES work, proceed as
>> follows:
>> - Shut down the webapps process:
>>   - go to the window where start-webapps.sh is running, and type ^C
>>  - Set up a fresh tomcat instance
>> - Configure tomcat to point to the multiprocess-file example
>> properties.xml:
>>   - Modify the tomcat startup scripts to include -D
>> org.apache.manifoldcf.configfile=<path_to_dist/multiprocess-file-example/properties.xml>
>>   - Start the new instance of tomcat
>> - Deploy the wars to that tomcat (except combined war)
>>   - Use the Tomcat admin tool to deploy mcf-crawler-ui.war,
>> mcf-authority-service.war, and mcf-api-service.war
>> - Go to the UI as served by Tomcat.  You should see your connections,
>> job, etc just as before.  Run the job.  Does it work, yes or no?
>>
>>
>> Thanks,
>> Karl
>>
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Jul 9, 2014 at 5:58 PM, lalit jangra <lalit.j.jangra@gmail.com>
>> wrote:
>>
>>> Thanks Karl,
>>>
>>> Indeed i really appreciate your help and promptness.
>>>
>>> I tried the way you suggest and below are results.
>>>
>>> 1. Took a fresh MCF, started it using start.jar under dist/example using
>>> jetty.
>>> 2. Under multiprocess-file-example, initialized DB & started agent using
>>> .start-agents.sh.
>>> 3. Created all connection and jobs. Started job & it worked fine.
>>> 4. Next i shutdown jetty as started in step1.
>>> 5. Setup a fresh tomcat and pointed to
>>> multiprocess-file-example/properties.xml in its setenv.sh using MCF_HOME.
>>> 6. Started tomcat & deployed all but combined war.
>>> 7. Accessed mcf-crawler-ui war & accessed job.
>>> 8. On Job tab i get connection refused error.
>>> 9. If i start DB using ./start-databse.sh, i get "Error getting
>>> connection:invalid authorization exception-not found:manifoldcf" message.
>>>
>>> Really embarrassing for me as its not working.
>>>
>>> Regards.
>>>
>>>
>>> On Wed, Jul 9, 2014 at 6:52 PM, Karl Wright <daddywri@gmail.com> wrote:
>>>
>>>> Hi Lalit,
>>>>
>>>> Since I'm quite busy at work today, can you do the following:
>>>>
>>>> - Start with a fresh build of MCF
>>>> - Go to the multiprocess-file example and start that in Jetty, with NO
>>>> changes
>>>> - If that doesn't work, open a ticket.  If that DOES work, proceed as
>>>> follows:
>>>> - Shut down the webapps process in the example
>>>> - Set up a fresh tomcat instance
>>>> - Configure tomcat to point to the multiprocess-file example
>>>> properties.xml.
>>>> - Deploy the wars to that tomcat (except combined war)
>>>> - Once again, try your job.  Let me know the result.
>>>>
>>>> Thanks,
>>>> Karl
>>>>
>>>>
>>>>
>>>> On Wed, Jul 9, 2014 at 1:47 PM, Karl Wright <daddywri@gmail.com> wrote:
>>>>
>>>>> Hi lalit,
>>>>>
>>>>> I don't see anything wrong with your environment.  I suspect your
>>>>> configuration is what is wrong.
>>>>>
>>>>> Karl
>>>>>
>>>>>
>>>>> On Wed, Jul 9, 2014 at 1:46 PM, lalit jangra <lalit.j.jangra@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> Thanks Karl,
>>>>>>
>>>>>> I agree with you. I am using Red Hat 6.4 with Postgres 9.2 & tomcat 7.
>>>>>>
>>>>>> Can this be an issue ?
>>>>>>
>>>>>> Regards.
>>>>>>
>>>>>>
>>>>>> On Wed, Jul 9, 2014 at 6:43 PM, Karl Wright <daddywri@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Lalit,
>>>>>>>
>>>>>>> I tried the zookeeper example here this morning and it worked
>>>>>>> perfectly.  I am happy to try the file system example too, when I have a
>>>>>>> moment, but I very much doubt there is any problem with that code
>>>>>>> specifically.
>>>>>>>
>>>>>>> Remember that this is NOT a database issue.  It is a synchronization
>>>>>>> issue ONLY.  So it does not matter what database instance you point it to.
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jul 9, 2014 at 1:32 PM, lalit jangra <
>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>
>>>>>>>> Thanks Karl,
>>>>>>>>
>>>>>>>> I did it as you mentioned but still getting same error. This time
>>>>>>>> also i setup new DB and tried. Also i could see syncharea folder empty
>>>>>>>> after lock cleaning.
>>>>>>>>
>>>>>>>> Regards.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jul 9, 2014 at 6:21 PM, Karl Wright <daddywri@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Lalit,
>>>>>>>>>
>>>>>>>>> If you were using the multiprocess-file-example before, and you
>>>>>>>>> did not shut things down cleanly, you should do: ./lock-clean.sh first.
>>>>>>>>>
>>>>>>>>> Karl
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jul 9, 2014 at 1:10 PM, lalit jangra <
>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Karl,
>>>>>>>>>>
>>>>>>>>>> I think its my bad luck today. :(
>>>>>>>>>>
>>>>>>>>>> I tried the way you just suggested but still getting same error
>>>>>>>>>> and job not able to run. What i did.
>>>>>>>>>>
>>>>>>>>>> 1. Created a brand new DB schema using ./executecommand.sh
>>>>>>>>>> org.apache.manifoldcf.core.DBCreate username password
>>>>>>>>>> 2. Initialized DB schema using ./initialize.sh
>>>>>>>>>> 3. Start agent process A using ./start-agents.sh
>>>>>>>>>> 4. Updated tomcat/bin/setenv.sh file to add
>>>>>>>>>> "-Dorg.apache.manifoldcf.configfile" parameter to point to properties.xml
>>>>>>>>>> under dist/multiprocess-file-example.
>>>>>>>>>> 5. Started Tomcat.
>>>>>>>>>> 6. Deployed all wars except combined war.
>>>>>>>>>> 7. Started job
>>>>>>>>>>
>>>>>>>>>> Job still not able to take off with same error in logs.
>>>>>>>>>>
>>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception tossed:
>>>>>>>>>>  Service 'A' of type
>>>>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is already
>>>>>>>>>> active
>>>>>>>>>>
>>>>>>>>>> I think its not my day today.
>>>>>>>>>>
>>>>>>>>>> Regards.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Jul 9, 2014 at 4:02 PM, Karl Wright <daddywri@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Lalit,
>>>>>>>>>>>
>>>>>>>>>>> In theory the combined war will run with external locking, it's
>>>>>>>>>>> just that nobody ever thought that would be how people would use it, and
>>>>>>>>>>> therefore it hasn't been tested etc.
>>>>>>>>>>>
>>>>>>>>>>> I would suggest that for a development setup, you use the Jetty
>>>>>>>>>>> simple example.  For a production setup, you should adopt the standard
>>>>>>>>>>> distributed arrangement as follows:
>>>>>>>>>>>
>>>>>>>>>>> - a deployed mcf-crawler-ui war
>>>>>>>>>>> - a deployed mcf-authority-service war
>>>>>>>>>>> - an (optional) deployed mcf-api-service war
>>>>>>>>>>> - one or more agents processes
>>>>>>>>>>>
>>>>>>>>>>> This is how it is done in both kinds of multiprocess examples,
>>>>>>>>>>> and is described thoroughly in the how-to-build-and-deploy document online.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Karl
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:56 AM, lalit jangra <
>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thanks Karl,
>>>>>>>>>>>>
>>>>>>>>>>>> My idea was as its name suggests mcf-combined-war as i was
>>>>>>>>>>>> trying to take advantage of agent process inside war. My assumption is
>>>>>>>>>>>> wrong here as i can see here.
>>>>>>>>>>>>
>>>>>>>>>>>> In this case, which war should i use - mcf-crawler-ui.war? If i
>>>>>>>>>>>> use this or any of other wars, then should i start agent process?
>>>>>>>>>>>>
>>>>>>>>>>>> Please suggest.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jul 9, 2014 at 3:48 PM, Karl Wright <daddywri@gmail.com
>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Lalit,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Probably it is not a database issue, but how locking is
>>>>>>>>>>>>> configured.  Typically the combined war is run in a non-clustered
>>>>>>>>>>>>> environment with no external synchronization required (as it is in the
>>>>>>>>>>>>> example I pointed you at).  We don't currently have an example that uses
>>>>>>>>>>>>> combined war with external locking -- that would not make much sense.  Can
>>>>>>>>>>>>> you tell me why you are trying to do it that way?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:46 AM, lalit jangra <
>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Karl,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> if i try ./start-combined.sh, the job runs and i could see it
>>>>>>>>>>>>>> finishing gracefully with updating index in solr.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I assume there is some DB issue here? Any suggestions?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 3:17 PM, Karl Wright <
>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Lalit,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please do the following: In an unmodified dist/example
>>>>>>>>>>>>>>> directory, run the start-combined script.  Open the UI, create the
>>>>>>>>>>>>>>> connections, and try to run the job.  Does it work for you?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:09 AM, lalit jangra <
>>>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks Karl,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have reconfigured everything and this time i am starting
>>>>>>>>>>>>>>>> tomcat and deploying mcf-combined-war. I created output connection,
>>>>>>>>>>>>>>>> repository connection & job. Still when i start my job, it does not go
>>>>>>>>>>>>>>>> ahead and stuck at starting up status.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 12:49 PM, Karl Wright <
>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi Lalit,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> You do not use combined.war AND an agents process at the
>>>>>>>>>>>>>>>>> same time.  The combined war INCLUDES the agents process.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 6:18 AM, lalit jangra <
>>>>>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Karl,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> As per your suggestion, i built a new MCF 1.5.1 instance,
>>>>>>>>>>>>>>>>>> con from source , configured it and pointed to local postgresql DB in
>>>>>>>>>>>>>>>>>> properties.xml  under /multiprocess-file-example. Then i tool a brand new
>>>>>>>>>>>>>>>>>> tomcat and configured for MCF in it in setenv.sh for properties.xml using
>>>>>>>>>>>>>>>>>> "-Dorg.apache.manifoldcf.configfile" argument and deployed mcf-combined.war
>>>>>>>>>>>>>>>>>> using app manager.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> First i initialized DB using ./initialize.sh command &
>>>>>>>>>>>>>>>>>> the moment i started agent using ./start-agents.sh command under
>>>>>>>>>>>>>>>>>> /multiprocess-file-example, i got the error, even when tomcat is not
>>>>>>>>>>>>>>>>>> started at this time. I created a job which never runs & get stuck.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception
>>>>>>>>>>>>>>>>>> tossed:  Service 'A' of type
>>>>>>>>>>>>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is already
>>>>>>>>>>>>>>>>>>   active
>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>>>>>>>>>>>>>>>>>> Service 'A'  of type
>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is  already active
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> But if i try to run jetty bundle using start.jar under
>>>>>>>>>>>>>>>>>> example, no such error but it does not fulfil my requirements.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:03 AM, Karl Wright <
>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> So, lalit, if you run the multiprocess example without
>>>>>>>>>>>>>>>>>>> any changes, do
>>>>>>>>>>>>>>>>>>> you see this?  I don't.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Sent from my Windows Phone
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>>>>> From: lalit jangra
>>>>>>>>>>>>>>>>>>> Sent: 7/9/2014 12:17 AM
>>>>>>>>>>>>>>>>>>> To: Karl Wright
>>>>>>>>>>>>>>>>>>> Cc: user@manifoldcf.apache.org
>>>>>>>>>>>>>>>>>>> Subject: Re: Apache ManifoldCF job stuck up
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks Karl,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I am currently running only single agent process on
>>>>>>>>>>>>>>>>>>> single machine
>>>>>>>>>>>>>>>>>>> without clustering. I have two environments and i could
>>>>>>>>>>>>>>>>>>> see this issue
>>>>>>>>>>>>>>>>>>> coming up at both places.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> While trying to start agent, i could see below error.
>>>>>>>>>>>>>>>>>>> When i am trying
>>>>>>>>>>>>>>>>>>> to start agent, it says below error and exits. But there
>>>>>>>>>>>>>>>>>>> is no agent
>>>>>>>>>>>>>>>>>>> process already running.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [root@server1 multiprocess-file-example]#
>>>>>>>>>>>>>>>>>>> ./start-agents.sh &
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [1] 5020
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [root@server1 multiprocess-file-example]# Running...
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Configuration file successfully read
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>>>>>>>>>>>>>>>>>>> Service 'A'
>>>>>>>>>>>>>>>>>>> of type 'AGENT' is already active
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.AgentRun.doExecute(AgentRun.java:54)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.BaseAgentsInitializationCommand.execute(BaseAgentsInitializationCommand.java:37)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.AgentRun.main(AgentRun.java:93)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [1]+  Exit 1                  ./start-agents.sh
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If i am able to successfully start agent using
>>>>>>>>>>>>>>>>>>> ./start-agents.sh,
>>>>>>>>>>>>>>>>>>> still it throws same error but i have no other process
>>>>>>>>>>>>>>>>>>> running.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Tue, Jul 8, 2014 at 2:38 PM, Karl Wright <
>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi lalit,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This occurs when you have more than one agents process
>>>>>>>>>>>>>>>>>>> with the same
>>>>>>>>>>>>>>>>>>>  process id using the same shared file system directory
>>>>>>>>>>>>>>>>>>> / zookeeper
>>>>>>>>>>>>>>>>>>>  cluster.  There is no other way it can occur.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>  Karl
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Sent from my Windows Phone
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>>>>>  From: lalit jangra
>>>>>>>>>>>>>>>>>>>  Sent: 7/8/2014 8:38 AM
>>>>>>>>>>>>>>>>>>>  To: user@manifoldcf.apache.org
>>>>>>>>>>>>>>>>>>>  Subject: Re: Apache ManifoldCF job stuck up
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks Karl,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I have tried steps you suggested and it worked on one
>>>>>>>>>>>>>>>>>>> instance.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> But on another instance i am still not able to resolve
>>>>>>>>>>>>>>>>>>> this issue.
>>>>>>>>>>>>>>>>>>>  Along with steps you mentioned, i tried recreating DB
>>>>>>>>>>>>>>>>>>> instance again,
>>>>>>>>>>>>>>>>>>>  setting up new MCF instance , cleaning locks and then
>>>>>>>>>>>>>>>>>>> starting agents
>>>>>>>>>>>>>>>>>>>  first & then tomcat. But issue still persists.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If i try to run ./start-agents.sh, i get this error for
>>>>>>>>>>>>>>>>>>> agent A.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) -
>>>>>>>>>>>>>>>>>>> Exception tossed:
>>>>>>>>>>>>>>>>>>>  Service 'A' of type
>>>>>>>>>>>>>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
>>>>>>>>>>>>>>>>>>> is already
>>>>>>>>>>>>>>>>>>>  active
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>>>>>>>>>>>>>>>>>>> Service 'A'
>>>>>>>>>>>>>>>>>>>  of type
>>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is
>>>>>>>>>>>>>>>>>>>  already active
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> But if i try to run ./start-agents-2.sh, i see similar
>>>>>>>>>>>>>>>>>>> error but agent B
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) -
>>>>>>>>>>>>>>>>>>> Exception tossed:
>>>>>>>>>>>>>>>>>>>  Service 'B' of type
>>>>>>>>>>>>>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
>>>>>>>>>>>>>>>>>>> is already
>>>>>>>>>>>>>>>>>>>  active
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>>>>>>>>>>>>>>>>>>> Service 'B'
>>>>>>>>>>>>>>>>>>>  of type
>>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is
>>>>>>>>>>>>>>>>>>>  already active
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Jul 7, 2014 at 4:55 PM, Karl Wright <
>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Lalit,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If you are using file synchronization, you cannot expect
>>>>>>>>>>>>>>>>>>> MCF to clean
>>>>>>>>>>>>>>>>>>>  up itself unless you shut it down cleanly.  You should
>>>>>>>>>>>>>>>>>>> be either using
>>>>>>>>>>>>>>>>>>>  ^C or plain kill, NEVER kill -9.  kill -9 will leave
>>>>>>>>>>>>>>>>>>> dangling locks.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> To clean up dangling locks:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> - shut ALL manifoldcf processes and web apps down
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> - run the lock-clean script
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> - start up the processes again
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>  Zookeeper synchronization, by the way, does not have
>>>>>>>>>>>>>>>>>>> this kind of problem.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>  Karl
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Jul 7, 2014 at 11:49 AM, lalit jangra <
>>>>>>>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I configured MCF 1.5.1 to run with Postgresql DB and
>>>>>>>>>>>>>>>>>>> tomcat 7.
>>>>>>>>>>>>>>>>>>>  Initially i created all connections, created a an
>>>>>>>>>>>>>>>>>>> alfresco job and it
>>>>>>>>>>>>>>>>>>>  all worked fine.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Next for updates, i stopped, tomcat and agent process
>>>>>>>>>>>>>>>>>>> running. Then i
>>>>>>>>>>>>>>>>>>>  updated CmisRepositoryConnector.java with my own code
>>>>>>>>>>>>>>>>>>> and run "ant
>>>>>>>>>>>>>>>>>>>  build" at root of MCF. It updated all code and jars
>>>>>>>>>>>>>>>>>>> file.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Also properties.xml is reset under
>>>>>>>>>>>>>>>>>>> /dist/multiprocess-file/example
>>>>>>>>>>>>>>>>>>>  which i updated to connect to Postgresql DB and logging
>>>>>>>>>>>>>>>>>>>  configurations.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I started tomcat then agent process.  Finally i started
>>>>>>>>>>>>>>>>>>> job to crawl
>>>>>>>>>>>>>>>>>>>  alfresco but it got stuck and moving on. I checked into
>>>>>>>>>>>>>>>>>>>  /dist/multiprocess-file/example/logs/manifoldcf.log
>>>>>>>>>>>>>>>>>>> file and could see
>>>>>>>>>>>>>>>>>>>  below error.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> ERROR 2014-07-07 16:09:04,936 (Agents thread) -
>>>>>>>>>>>>>>>>>>> Exception tossed:
>>>>>>>>>>>>>>>>>>>  Service '' of type
>>>>>>>>>>>>>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
>>>>>>>>>>>>>>>>>>> is already
>>>>>>>>>>>>>>>>>>>  active
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>>>>>>>>>>>>>>>>>>> Service ''
>>>>>>>>>>>>>>>>>>>  of type
>>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is
>>>>>>>>>>>>>>>>>>>  already active
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon$AgentsThread.run(AgentsDaemon.java:208)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I created another job but that got stuck too.Is DB got
>>>>>>>>>>>>>>>>>>> corrupt due to rebuild?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Also is this the right way to build MCF (I hope its
>>>>>>>>>>>>>>>>>>> correct way). Now
>>>>>>>>>>>>>>>>>>>  what should i do to fix this issue?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Please help.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>  Lalit Jangra.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>  Regards,
>>>>>>>>>>>>>>>>>>>  Lalit Jangra.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Regards,
>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Regards,
>>>>>>>> Lalit Jangra.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Lalit Jangra.
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Regards,
>>> Lalit Jangra.
>>>
>>
>>
>
>
> --
> Regards,
> Lalit Jangra.
>

Mime
View raw message