manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Apache ManifoldCF job stuck up
Date Fri, 11 Jul 2014 08:13:19 GMT
Hi Lalit,

Your problem is unquestionably configuration.  There is something you are
neglecting to do, or are doing wrong.  I have not been able to reproduce
your problem here even once.

Karl



On Fri, Jul 11, 2014 at 3:56 AM, lalit jangra <lalit.j.jangra@gmail.com>
wrote:

> Hi Karl,
>
> So what could be the issue: environmental or configuration? I am afraid if
> it goes unnoticed and we are stuck in production, that would be a
> nightmare.
>
> Except this build change, i am following same process every time. Also i
> am starting tomcat first and then agents.
>
> Regards.
>
>
> On Fri, Jul 11, 2014 at 8:52 AM, Karl Wright <daddywri@gmail.com> wrote:
>
>> Download dependencies come from maven either way; they either work or
>> fail to download.  That has nothing to do with your earlier problem.
>>
>>
>> Thanks,
>> Karl
>>
>> Sent from my Windows Phone
>> ------------------------------
>> From: lalit jangra
>> Sent: 7/11/2014 1:59 AM
>> To: Karl Wright
>>
>> Subject: Re: Apache ManifoldCF job stuck up
>>
>>  Hi Karl,
>>
>> I got it working finally.
>>
>> I changed the way i was downloading dependencies. Instead of using "ant
>> make-core-deps" command, i used lib dependencies supplied along with MCF
>> source and built it.
>>
>> This way i could not see this issue during testing till now and i have
>> kept it under observation. I am using Postgres 9.2 this time.
>>
>> Please suggest.
>>
>> Regards.
>>
>>
>> On Thu, Jul 10, 2014 at 5:31 PM, Karl Wright <daddywri@gmail.com> wrote:
>>
>>> Hi Lalit,
>>>
>>> I tried both PostgreSQL 8.4 and PostgreSQL 9.3.  Both worked fine.
>>> PostgreSQL 8.4 was on Windows Vista, and PostgreSQL 9.3 was on Windows 7.
>>>
>>> To answer your second question, downloading the manifoldcf 1.5.1 lib
>>> package from the Apache distribution server is safe; you can verify this by
>>> checking signatures (which are also provided).
>>>
>>> Karl
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Jul 10, 2014 at 12:26 PM, lalit jangra <lalit.j.jangra@gmail.com
>>> > wrote:
>>>
>>>> Thanks Karl for all your help.
>>>>
>>>> I setup everything again on my laptop(windows 7) with tomcat 7,
>>>> MCF1.5.1 & PostgresSQL 9.1.13.4. Luckily it worked. :)
>>>>
>>>> I could not see any agent related error although getting some
>>>> permission related warnings but i hope these are not show stoppers.
>>>>
>>>> WARN 2014-07-10 17:18:32,026 (Document delete stuffer thread) - Attempt
>>>> to set file lock
>>>> 'C:\acc-proposals\2013-14\Irish-Water\lalit\lj\iw\apache-manifoldcf-1.5.1\dist\multiprocess-file-example\.\.\syncharea\737\563\lock-_Cache_JOBSTATUSES.lock'
>>>> failed: Access is denied
>>>> java.io.IOException: Access is denied
>>>>
>>>> This time i changed two things, used Postgresql 9.1.13 & built MCF
>>>> after downloading all libraries and place then in MCf lib folderas
>>>> mentioned below.
>>>>
>>>> *EITHER:* overlay the lib directory from the corresponding lib
>>>> distribution (preferred, where possible). Then did ant make-deps and
>>>> finally ant build.
>>>>
>>>> I want to know what version of Postgresql you are trying & also
>>>> downloading lib and placing then in MCF is safe?I am testing it for
>>>> sometime and then will try to replicate it on RHEL6.4 with same stack.
>>>>
>>>> Again, I heartily appreciate your cooperation.
>>>>
>>>> Reagrds.
>>>>
>>>>
>>>> On Thu, Jul 10, 2014 at 11:33 AM, Karl Wright <daddywri@gmail.com>
>>>> wrote:
>>>>
>>>>> Just wanted to let you know I went through the steps I gave you above,
>>>>> on Windows, and had no problems at all.
>>>>>
>>>>> I will be able to do the same on Ubuntu but not until tomorrow morning.
>>>>>
>>>>> Thanks,
>>>>> Karl
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jul 10, 2014 at 6:12 AM, Karl Wright <daddywri@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Lalit,
>>>>>>
>>>>>> I still have not been able to repeat what you are seeing, here.  I
>>>>>> will keep trying.  If I am able to repeat it I will let you know.
>>>>>>
>>>>>> Otherwise, I have no advice for you.  The fact that it worked for you
>>>>>> before you changed the configuration to point to postgresql indicates that
>>>>>> the synchronization was properly configured and working as planned.  The
>>>>>> database does not affect synchronization and cannot lead to the error
>>>>>> message you see.
>>>>>>
>>>>>> The only thing I can think of at all is that there might be a cleanup
>>>>>> issue on process exit for the agents process.  But if that were true, the
>>>>>> lock-clean procedure (shutting all MCF processes and webapps) would work.
>>>>>> Since it doesn't for you, then I am truly puzzled.
>>>>>>
>>>>>> Sorry I could not help you.
>>>>>>
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jul 10, 2014 at 5:59 AM, lalit jangra <
>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>
>>>>>>> Thanks Karl,
>>>>>>>
>>>>>>> As you suggested, i followed below steps.
>>>>>>>
>>>>>>> 1. Stopped HSQL DB, agents & web applicaitons (by shutting down
>>>>>>> tomcat).
>>>>>>> 2. Updated properties.xml with below details for PostgreSQL & did
>>>>>>> leave all setting as it is.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>   <property name="org.apache.manifoldcf.databaseimplementationclass"
>>>>>>> value="org.apache.manifoldcf.core.database.DBInterfacePostgreSQL"/>
>>>>>>>
>>>>>>>   <property name="org.apache.manifoldcf.dbsuperusername"
>>>>>>> value="postgres"/>
>>>>>>>
>>>>>>>   <property name="org.apache.manifoldcf.dbsuperuserpassword"
>>>>>>> value="admin"/>
>>>>>>>
>>>>>>>   <property name="org.apache.manifoldcf.database.name"
>>>>>>> value="mcfdkw" />
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>   <property name="org.apache.manifoldcf.hsqldbdatabaseprotocol"
>>>>>>> value="hsql"/>
>>>>>>>
>>>>>>>   <property name="org.apache.manifoldcf.hsqldbdatabaseserver"
>>>>>>> value="localhost"/>
>>>>>>>
>>>>>>>   <property name="org.apache.manifoldcf.hsqldbdatabaseinstance"
>>>>>>> value="xdb"/>
>>>>>>>
>>>>>>>   <property name="org.apache.manifoldcf.database.maxhandles"
>>>>>>> value="500"/>
>>>>>>>
>>>>>>>   <property name="org.apache.manifoldcf.crawler.threads"
>>>>>>> value="500"/>
>>>>>>>
>>>>>>>
>>>>>>> 3. Initialized DB with ./initialize.sh script, works fine and i can
>>>>>>> see successful registration messages.
>>>>>>> 4. Started agents using ./start-agents.sh
>>>>>>> 5. Started Tomcat, created connections & jobs.Started crawl job.
>>>>>>> 6.  Job not taking off and still can see below error in logs.
>>>>>>>
>>>>>>>
>>>>>>> ERROR 2014-07-10 10:51:55,690 (Agents thread) - Exception tossed:
>>>>>>> Service 'A' of type
>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is already active
>>>>>>>
>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service
>>>>>>> 'A' of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is
>>>>>>> already active
>>>>>>>
>>>>>>> Regards.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jul 10, 2014 at 10:12 AM, Karl Wright <daddywri@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> bq. So the only difference i could observe is DB which is HSQL vs
>>>>>>>> Postgres, rest everything is same.
>>>>>>>>
>>>>>>>> Right, exactly.  Only this time we have some certainty that the
>>>>>>>> configuration is otherwise correct for synchronization and processes.
>>>>>>>>
>>>>>>>> The next step is the following:
>>>>>>>>
>>>>>>>> - START with the same multiprocess-file-example directory and
>>>>>>>> processes you verified yesterday
>>>>>>>> - Stop all MCF processes/web applications that are using that
>>>>>>>> multiprocess-file-example properties file:
>>>>>>>>   - Either stop the Tomcat process, or better yet go to the "admin"
>>>>>>>> tomcat application and stop all three MCF web applications
>>>>>>>>   - Go to the start-agents.sh window and ^C that process
>>>>>>>>   - Go to the start-database.sh window and ^C that process
>>>>>>>> - Change the properties.xml file under multiprocess-file-example to
>>>>>>>> point to the PostgresDB instance you want it to use.  This instance CANNOT
>>>>>>>> be in use by any other MCF cluster or process.
>>>>>>>>   - Replace the following THREE properties in properties.xml with
>>>>>>>> your PostgreSQL equivalent, as follows:
>>>>>>>>
>>>>>>>> >>>>>>
>>>>>>>>   <property
>>>>>>>> name="org.apache.manifoldcf.databaseimplementationclass"
>>>>>>>> value="org.apache.manifoldcf.core.database.DBInterfacePostgreSQL"/>
>>>>>>>>   <property name="org.apache.manifoldcf.dbsuperusername"
>>>>>>>> value="<something>"/>
>>>>>>>>   <property name="org.apache.manifoldcf.dbsuperuserpassword"
>>>>>>>> value="<something>"/>
>>>>>>>> <<<<<<
>>>>>>>>
>>>>>>>> You may also choose to set these properties, to avoid collision
>>>>>>>> with other instances.  You can use whatever value you like for these,
>>>>>>>> within what PostgreSQL allows:
>>>>>>>>
>>>>>>>> org.apache.manifoldcf.database.name
>>>>>>>> org.apache.manifoldcf.database.username
>>>>>>>> org.apache.manifoldcf.database.password
>>>>>>>>
>>>>>>>> - Run the initialize script:
>>>>>>>>   - cd multiprocess-file-example ; initialize.sh .  If this works,
>>>>>>>> you have the database configured properly.  If it fails, look at the
>>>>>>>> parameters you provided and figure out what is wrong before proceeding
>>>>>>>>
>>>>>>>>  - start the ManifoldCF processes and web applications:
>>>>>>>>   - cd multiprocess-file-example ; start-agents.sh
>>>>>>>>   - go to the tomcat instance, and start it, or start all the web
>>>>>>>> applications
>>>>>>>>
>>>>>>>> - create connections and jobs, and try a crawl.  Let me know how it
>>>>>>>> works.
>>>>>>>>
>>>>>>>> Karl
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jul 10, 2014 at 12:25 AM, lalit jangra <
>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Thanks a lot Karl,
>>>>>>>>>
>>>>>>>>> Yes, it works perfectly as you suggested in last mail.
>>>>>>>>>
>>>>>>>>> So the only difference i could observe is DB which is HSQL vs
>>>>>>>>> Postgres, rest everything is same.
>>>>>>>>>
>>>>>>>>> Please suggest.
>>>>>>>>>
>>>>>>>>> Regards.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jul 9, 2014 at 11:35 PM, Karl Wright <daddywri@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Lalit,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Sorry, I thought my instructions were clear.  For this exercise,
>>>>>>>>>> I want you to use ONLY the example under dist/multiprocess-file-example.
>>>>>>>>>> DO NOT use the example under dist/example.
>>>>>>>>>>
>>>>>>>>>> Here are the instructions again, clarified and fleshed out:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - Start with a fresh build of MCF
>>>>>>>>>> - Go to the multiprocess-file example and start that in Jetty,
>>>>>>>>>> with NO changes:
>>>>>>>>>>   - cd multiprocess-file-example
>>>>>>>>>>   - start-database.sh
>>>>>>>>>>   - initialize.sh
>>>>>>>>>>   - start-agents.sh
>>>>>>>>>>   - in a separate window: cd multiprocess-file-example ;
>>>>>>>>>> start-webapps.sh
>>>>>>>>>>   - create connections and job, see if it runs
>>>>>>>>>>  - If that doesn't work, open a ticket.  If that DOES work,
>>>>>>>>>> proceed as follows:
>>>>>>>>>> - Shut down the webapps process:
>>>>>>>>>>   - go to the window where start-webapps.sh is running, and type
>>>>>>>>>> ^C
>>>>>>>>>>  - Set up a fresh tomcat instance
>>>>>>>>>> - Configure tomcat to point to the multiprocess-file example
>>>>>>>>>> properties.xml:
>>>>>>>>>>   - Modify the tomcat startup scripts to include -D
>>>>>>>>>> org.apache.manifoldcf.configfile=<path_to_dist/multiprocess-file-example/properties.xml>
>>>>>>>>>>   - Start the new instance of tomcat
>>>>>>>>>> - Deploy the wars to that tomcat (except combined war)
>>>>>>>>>>   - Use the Tomcat admin tool to deploy mcf-crawler-ui.war,
>>>>>>>>>> mcf-authority-service.war, and mcf-api-service.war
>>>>>>>>>> - Go to the UI as served by Tomcat.  You should see your
>>>>>>>>>> connections, job, etc just as before.  Run the job.  Does it work, yes or
>>>>>>>>>> no?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Karl
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Jul 9, 2014 at 5:58 PM, lalit jangra <
>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks Karl,
>>>>>>>>>>>
>>>>>>>>>>> Indeed i really appreciate your help and promptness.
>>>>>>>>>>>
>>>>>>>>>>> I tried the way you suggest and below are results.
>>>>>>>>>>>
>>>>>>>>>>> 1. Took a fresh MCF, started it using start.jar under
>>>>>>>>>>> dist/example using jetty.
>>>>>>>>>>> 2. Under multiprocess-file-example, initialized DB & started
>>>>>>>>>>> agent using .start-agents.sh.
>>>>>>>>>>> 3. Created all connection and jobs. Started job & it worked fine.
>>>>>>>>>>> 4. Next i shutdown jetty as started in step1.
>>>>>>>>>>> 5. Setup a fresh tomcat and pointed to
>>>>>>>>>>> multiprocess-file-example/properties.xml in its setenv.sh using MCF_HOME.
>>>>>>>>>>> 6. Started tomcat & deployed all but combined war.
>>>>>>>>>>> 7. Accessed mcf-crawler-ui war & accessed job.
>>>>>>>>>>> 8. On Job tab i get connection refused error.
>>>>>>>>>>> 9. If i start DB using ./start-databse.sh, i get "Error getting
>>>>>>>>>>> connection:invalid authorization exception-not found:manifoldcf" message.
>>>>>>>>>>>
>>>>>>>>>>> Really embarrassing for me as its not working.
>>>>>>>>>>>
>>>>>>>>>>> Regards.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jul 9, 2014 at 6:52 PM, Karl Wright <daddywri@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Lalit,
>>>>>>>>>>>>
>>>>>>>>>>>> Since I'm quite busy at work today, can you do the following:
>>>>>>>>>>>>
>>>>>>>>>>>> - Start with a fresh build of MCF
>>>>>>>>>>>> - Go to the multiprocess-file example and start that in Jetty,
>>>>>>>>>>>> with NO changes
>>>>>>>>>>>> - If that doesn't work, open a ticket.  If that DOES work,
>>>>>>>>>>>> proceed as follows:
>>>>>>>>>>>> - Shut down the webapps process in the example
>>>>>>>>>>>> - Set up a fresh tomcat instance
>>>>>>>>>>>> - Configure tomcat to point to the multiprocess-file example
>>>>>>>>>>>> properties.xml.
>>>>>>>>>>>> - Deploy the wars to that tomcat (except combined war)
>>>>>>>>>>>> - Once again, try your job.  Let me know the result.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Karl
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jul 9, 2014 at 1:47 PM, Karl Wright <daddywri@gmail.com
>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi lalit,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I don't see anything wrong with your environment.  I suspect
>>>>>>>>>>>>> your configuration is what is wrong.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 1:46 PM, lalit jangra <
>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks Karl,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I agree with you. I am using Red Hat 6.4 with Postgres 9.2 &
>>>>>>>>>>>>>> tomcat 7.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Can this be an issue ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 6:43 PM, Karl Wright <
>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Lalit,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I tried the zookeeper example here this morning and it
>>>>>>>>>>>>>>> worked perfectly.  I am happy to try the file system example too, when I
>>>>>>>>>>>>>>> have a moment, but I very much doubt there is any problem with that code
>>>>>>>>>>>>>>> specifically.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Remember that this is NOT a database issue.  It is a
>>>>>>>>>>>>>>> synchronization issue ONLY.  So it does not matter what database instance
>>>>>>>>>>>>>>> you point it to.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 1:32 PM, lalit jangra <
>>>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks Karl,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I did it as you mentioned but still getting same error.
>>>>>>>>>>>>>>>> This time also i setup new DB and tried. Also i could see syncharea folder
>>>>>>>>>>>>>>>> empty after lock cleaning.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 6:21 PM, Karl Wright <
>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi Lalit,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If you were using the multiprocess-file-example before,
>>>>>>>>>>>>>>>>> and you did not shut things down cleanly, you should do: ./lock-clean.sh
>>>>>>>>>>>>>>>>> first.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 1:10 PM, lalit jangra <
>>>>>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Karl,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I think its my bad luck today. :(
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I tried the way you just suggested but still getting same
>>>>>>>>>>>>>>>>>> error and job not able to run. What i did.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1. Created a brand new DB schema using
>>>>>>>>>>>>>>>>>> ./executecommand.sh org.apache.manifoldcf.core.DBCreate username password
>>>>>>>>>>>>>>>>>> 2. Initialized DB schema using ./initialize.sh
>>>>>>>>>>>>>>>>>> 3. Start agent process A using ./start-agents.sh
>>>>>>>>>>>>>>>>>> 4. Updated tomcat/bin/setenv.sh file to add
>>>>>>>>>>>>>>>>>> "-Dorg.apache.manifoldcf.configfile" parameter to point to properties.xml
>>>>>>>>>>>>>>>>>> under dist/multiprocess-file-example.
>>>>>>>>>>>>>>>>>> 5. Started Tomcat.
>>>>>>>>>>>>>>>>>> 6. Deployed all wars except combined war.
>>>>>>>>>>>>>>>>>> 7. Started job
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Job still not able to take off with same error in logs.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception
>>>>>>>>>>>>>>>>>> tossed:  Service 'A' of type
>>>>>>>>>>>>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is already
>>>>>>>>>>>>>>>>>> active
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I think its not my day today.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 4:02 PM, Karl Wright <
>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Lalit,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> In theory the combined war will run with external
>>>>>>>>>>>>>>>>>>> locking, it's just that nobody ever thought that would be how people would
>>>>>>>>>>>>>>>>>>> use it, and therefore it hasn't been tested etc.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I would suggest that for a development setup, you use
>>>>>>>>>>>>>>>>>>> the Jetty simple example.  For a production setup, you should adopt the
>>>>>>>>>>>>>>>>>>> standard distributed arrangement as follows:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> - a deployed mcf-crawler-ui war
>>>>>>>>>>>>>>>>>>> - a deployed mcf-authority-service war
>>>>>>>>>>>>>>>>>>> - an (optional) deployed mcf-api-service war
>>>>>>>>>>>>>>>>>>> - one or more agents processes
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This is how it is done in both kinds of multiprocess
>>>>>>>>>>>>>>>>>>> examples, and is described thoroughly in the how-to-build-and-deploy
>>>>>>>>>>>>>>>>>>> document online.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:56 AM, lalit jangra <
>>>>>>>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thanks Karl,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> My idea was as its name suggests mcf-combined-war as i
>>>>>>>>>>>>>>>>>>>> was trying to take advantage of agent process inside war. My assumption is
>>>>>>>>>>>>>>>>>>>> wrong here as i can see here.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> In this case, which war should i use -
>>>>>>>>>>>>>>>>>>>> mcf-crawler-ui.war? If i use this or any of other wars, then should i start
>>>>>>>>>>>>>>>>>>>> agent process?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Please suggest.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 3:48 PM, Karl Wright <
>>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi Lalit,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Probably it is not a database issue, but how locking
>>>>>>>>>>>>>>>>>>>>> is configured.  Typically the combined war is run in a non-clustered
>>>>>>>>>>>>>>>>>>>>> environment with no external synchronization required (as it is in the
>>>>>>>>>>>>>>>>>>>>> example I pointed you at).  We don't currently have an example that uses
>>>>>>>>>>>>>>>>>>>>> combined war with external locking -- that would not make much sense.  Can
>>>>>>>>>>>>>>>>>>>>> you tell me why you are trying to do it that way?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:46 AM, lalit jangra <
>>>>>>>>>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi Karl,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> if i try ./start-combined.sh, the job runs and i
>>>>>>>>>>>>>>>>>>>>>> could see it finishing gracefully with updating index in solr.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I assume there is some DB issue here? Any suggestions?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 3:17 PM, Karl Wright <
>>>>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi Lalit,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Please do the following: In an unmodified
>>>>>>>>>>>>>>>>>>>>>>> dist/example directory, run the start-combined script.  Open the UI, create
>>>>>>>>>>>>>>>>>>>>>>> the connections, and try to run the job.  Does it work for you?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:09 AM, lalit jangra <
>>>>>>>>>>>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks Karl,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I have reconfigured everything and this time i am
>>>>>>>>>>>>>>>>>>>>>>>> starting tomcat and deploying mcf-combined-war. I created output
>>>>>>>>>>>>>>>>>>>>>>>> connection, repository connection & job. Still when i start my job, it does
>>>>>>>>>>>>>>>>>>>>>>>> not go ahead and stuck at starting up status.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 12:49 PM, Karl Wright <
>>>>>>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Hi Lalit,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> You do not use combined.war AND an agents process
>>>>>>>>>>>>>>>>>>>>>>>>> at the same time.  The combined war INCLUDES the agents process.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 6:18 AM, lalit jangra <
>>>>>>>>>>>>>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Karl,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> As per your suggestion, i built a new MCF 1.5.1
>>>>>>>>>>>>>>>>>>>>>>>>>> instance, con from source , configured it and pointed to local postgresql
>>>>>>>>>>>>>>>>>>>>>>>>>> DB in properties.xml  under /multiprocess-file-example. Then i tool a brand
>>>>>>>>>>>>>>>>>>>>>>>>>> new tomcat and configured for MCF in it in setenv.sh for properties.xml
>>>>>>>>>>>>>>>>>>>>>>>>>> using "-Dorg.apache.manifoldcf.configfile" argument and deployed
>>>>>>>>>>>>>>>>>>>>>>>>>> mcf-combined.war using app manager.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> First i initialized DB using ./initialize.sh
>>>>>>>>>>>>>>>>>>>>>>>>>> command & the moment i started agent using ./start-agents.sh command under
>>>>>>>>>>>>>>>>>>>>>>>>>> /multiprocess-file-example, i got the error, even when tomcat is not
>>>>>>>>>>>>>>>>>>>>>>>>>> started at this time. I created a job which never runs & get stuck.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) -
>>>>>>>>>>>>>>>>>>>>>>>>>> Exception tossed:  Service 'A' of type
>>>>>>>>>>>>>>>>>>>>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is already
>>>>>>>>>>>>>>>>>>>>>>>>>>   active
>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>>>>>>>>>>>>>>>>>>>>>>>>>> Service 'A'  of type
>>>>>>>>>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is  already active
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> But if i try to run jetty bundle using start.jar
>>>>>>>>>>>>>>>>>>>>>>>>>> under example, no such error but it does not fulfil my requirements.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:03 AM, Karl Wright <
>>>>>>>>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> So, lalit, if you run the multiprocess example
>>>>>>>>>>>>>>>>>>>>>>>>>>> without any changes, do
>>>>>>>>>>>>>>>>>>>>>>>>>>> you see this?  I don't.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent from my Windows Phone
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>>>>>>>>>>>>> From: lalit jangra
>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: 7/9/2014 12:17 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>> To: Karl Wright
>>>>>>>>>>>>>>>>>>>>>>>>>>> Cc: user@manifoldcf.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: Apache ManifoldCF job stuck up
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks Karl,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I am currently running only single agent process
>>>>>>>>>>>>>>>>>>>>>>>>>>> on single machine
>>>>>>>>>>>>>>>>>>>>>>>>>>> without clustering. I have two environments and
>>>>>>>>>>>>>>>>>>>>>>>>>>> i could see this issue
>>>>>>>>>>>>>>>>>>>>>>>>>>> coming up at both places.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> While trying to start agent, i could see below
>>>>>>>>>>>>>>>>>>>>>>>>>>> error. When i am trying
>>>>>>>>>>>>>>>>>>>>>>>>>>> to start agent, it says below error and exits.
>>>>>>>>>>>>>>>>>>>>>>>>>>> But there is no agent
>>>>>>>>>>>>>>>>>>>>>>>>>>> process already running.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> [root@server1 multiprocess-file-example]#
>>>>>>>>>>>>>>>>>>>>>>>>>>> ./start-agents.sh &
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> [1] 5020
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> [root@server1 multiprocess-file-example]#
>>>>>>>>>>>>>>>>>>>>>>>>>>> Running...
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Configuration file successfully read
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>>>>>>>>>>>>>>>>>>>>>>>>>>> Service 'A'
>>>>>>>>>>>>>>>>>>>>>>>>>>> of type 'AGENT' is already active
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.AgentRun.doExecute(AgentRun.java:54)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.BaseAgentsInitializationCommand.execute(BaseAgentsInitializationCommand.java:37)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.AgentRun.main(AgentRun.java:93)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]+  Exit 1                  ./start-agents.sh
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> If i am able to successfully start agent using
>>>>>>>>>>>>>>>>>>>>>>>>>>> ./start-agents.sh,
>>>>>>>>>>>>>>>>>>>>>>>>>>> still it throws same error but i have no other
>>>>>>>>>>>>>>>>>>>>>>>>>>> process running.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jul 8, 2014 at 2:38 PM, Karl Wright <
>>>>>>>>>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi lalit,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> This occurs when you have more than one agents
>>>>>>>>>>>>>>>>>>>>>>>>>>> process with the same
>>>>>>>>>>>>>>>>>>>>>>>>>>>  process id using the same shared file system
>>>>>>>>>>>>>>>>>>>>>>>>>>> directory / zookeeper
>>>>>>>>>>>>>>>>>>>>>>>>>>>  cluster.  There is no other way it can occur.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>  Karl
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent from my Windows Phone
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>>>>>>>>>>>>>  From: lalit jangra
>>>>>>>>>>>>>>>>>>>>>>>>>>>  Sent: 7/8/2014 8:38 AM
>>>>>>>>>>>>>>>>>>>>>>>>>>>  To: user@manifoldcf.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>>  Subject: Re: Apache ManifoldCF job stuck up
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks Karl,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I have tried steps you suggested and it worked
>>>>>>>>>>>>>>>>>>>>>>>>>>> on one instance.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> But on another instance i am still not able to
>>>>>>>>>>>>>>>>>>>>>>>>>>> resolve this issue.
>>>>>>>>>>>>>>>>>>>>>>>>>>>  Along with steps you mentioned, i tried
>>>>>>>>>>>>>>>>>>>>>>>>>>> recreating DB instance again,
>>>>>>>>>>>>>>>>>>>>>>>>>>>  setting up new MCF instance , cleaning locks
>>>>>>>>>>>>>>>>>>>>>>>>>>> and then starting agents
>>>>>>>>>>>>>>>>>>>>>>>>>>>  first & then tomcat. But issue still persists.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> If i try to run ./start-agents.sh, i get this
>>>>>>>>>>>>>>>>>>>>>>>>>>> error for agent A.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) -
>>>>>>>>>>>>>>>>>>>>>>>>>>> Exception tossed:
>>>>>>>>>>>>>>>>>>>>>>>>>>>  Service 'A' of type
>>>>>>>>>>>>>>>>>>>>>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
>>>>>>>>>>>>>>>>>>>>>>>>>>> is already
>>>>>>>>>>>>>>>>>>>>>>>>>>>  active
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>>>>>>>>>>>>>>>>>>>>>>>>>>> Service 'A'
>>>>>>>>>>>>>>>>>>>>>>>>>>>  of type
>>>>>>>>>>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is
>>>>>>>>>>>>>>>>>>>>>>>>>>>  already active
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> But if i try to run ./start-agents-2.sh, i see
>>>>>>>>>>>>>>>>>>>>>>>>>>> similar error but agent B
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) -
>>>>>>>>>>>>>>>>>>>>>>>>>>> Exception tossed:
>>>>>>>>>>>>>>>>>>>>>>>>>>>  Service 'B' of type
>>>>>>>>>>>>>>>>>>>>>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
>>>>>>>>>>>>>>>>>>>>>>>>>>> is already
>>>>>>>>>>>>>>>>>>>>>>>>>>>  active
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>>>>>>>>>>>>>>>>>>>>>>>>>>> Service 'B'
>>>>>>>>>>>>>>>>>>>>>>>>>>>  of type
>>>>>>>>>>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is
>>>>>>>>>>>>>>>>>>>>>>>>>>>  already active
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jul 7, 2014 at 4:55 PM, Karl Wright <
>>>>>>>>>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Lalit,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> If you are using file synchronization, you
>>>>>>>>>>>>>>>>>>>>>>>>>>> cannot expect MCF to clean
>>>>>>>>>>>>>>>>>>>>>>>>>>>  up itself unless you shut it down cleanly.  You
>>>>>>>>>>>>>>>>>>>>>>>>>>> should be either using
>>>>>>>>>>>>>>>>>>>>>>>>>>>  ^C or plain kill, NEVER kill -9.  kill -9 will
>>>>>>>>>>>>>>>>>>>>>>>>>>> leave dangling locks.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> To clean up dangling locks:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> - shut ALL manifoldcf processes and web apps down
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> - run the lock-clean script
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> - start up the processes again
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>  Zookeeper synchronization, by the way, does not
>>>>>>>>>>>>>>>>>>>>>>>>>>> have this kind of problem.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>  Karl
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Jul 7, 2014 at 11:49 AM, lalit jangra <
>>>>>>>>>>>>>>>>>>>>>>>>>>> lalit.j.jangra@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I configured MCF 1.5.1 to run with Postgresql DB
>>>>>>>>>>>>>>>>>>>>>>>>>>> and tomcat 7.
>>>>>>>>>>>>>>>>>>>>>>>>>>>  Initially i created all connections, created a
>>>>>>>>>>>>>>>>>>>>>>>>>>> an alfresco job and it
>>>>>>>>>>>>>>>>>>>>>>>>>>>  all worked fine.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Next for updates, i stopped, tomcat and agent
>>>>>>>>>>>>>>>>>>>>>>>>>>> process running. Then i
>>>>>>>>>>>>>>>>>>>>>>>>>>>  updated CmisRepositoryConnector.java with my
>>>>>>>>>>>>>>>>>>>>>>>>>>> own code and run "ant
>>>>>>>>>>>>>>>>>>>>>>>>>>>  build" at root of MCF. It updated all code and
>>>>>>>>>>>>>>>>>>>>>>>>>>> jars file.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Also properties.xml is reset under
>>>>>>>>>>>>>>>>>>>>>>>>>>> /dist/multiprocess-file/example
>>>>>>>>>>>>>>>>>>>>>>>>>>>  which i updated to connect to Postgresql DB and
>>>>>>>>>>>>>>>>>>>>>>>>>>> logging
>>>>>>>>>>>>>>>>>>>>>>>>>>>  configurations.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I started tomcat then agent process.  Finally i
>>>>>>>>>>>>>>>>>>>>>>>>>>> started job to crawl
>>>>>>>>>>>>>>>>>>>>>>>>>>>  alfresco but it got stuck and moving on. I
>>>>>>>>>>>>>>>>>>>>>>>>>>> checked into
>>>>>>>>>>>>>>>>>>>>>>>>>>>  /dist/multiprocess-file/example/logs/manifoldcf.log
>>>>>>>>>>>>>>>>>>>>>>>>>>> file and could see
>>>>>>>>>>>>>>>>>>>>>>>>>>>  below error.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> ERROR 2014-07-07 16:09:04,936 (Agents thread) -
>>>>>>>>>>>>>>>>>>>>>>>>>>> Exception tossed:
>>>>>>>>>>>>>>>>>>>>>>>>>>>  Service '' of type
>>>>>>>>>>>>>>>>>>>>>>>>>>>  'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent'
>>>>>>>>>>>>>>>>>>>>>>>>>>> is already
>>>>>>>>>>>>>>>>>>>>>>>>>>>  active
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException:
>>>>>>>>>>>>>>>>>>>>>>>>>>> Service ''
>>>>>>>>>>>>>>>>>>>>>>>>>>>  of type
>>>>>>>>>>>>>>>>>>>>>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is
>>>>>>>>>>>>>>>>>>>>>>>>>>>  already active
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>         at
>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon$AgentsThread.run(AgentsDaemon.java:208)
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I created another job but that got stuck too.Is
>>>>>>>>>>>>>>>>>>>>>>>>>>> DB got corrupt due to rebuild?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Also is this the right way to build MCF (I hope
>>>>>>>>>>>>>>>>>>>>>>>>>>> its correct way). Now
>>>>>>>>>>>>>>>>>>>>>>>>>>>  what should i do to fix this issue?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Please help.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>  Lalit Jangra.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>  Regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>>  Lalit Jangra.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Regards,
>>>>>>>>>>> Lalit Jangra.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Regards,
>>>>>>>>> Lalit Jangra.
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Regards,
>>>>>>> Lalit Jangra.
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Lalit.
>>>>
>>>
>>>
>>
>>
>> --
>> Regards,
>> Lalit.
>>
>
>
>
> --
> Regards,
> Lalit.
>

Mime
View raw message