manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erlend Garåsen <e.f.gara...@usit.uio.no>
Subject Re: Running ManifoldCF on Resin
Date Mon, 07 Mar 2011 10:24:09 GMT

I'm sorry about the late reply, I have been out of office.

The reason why it didn't work is because I didn't read the documentation 
properly. I simply forgot to run the following command after I started 
Resin:
./processes/script/executecommand.sh org.apache.manifoldcf.agents.AgentRun

I thought it was sufficient to start my application server.

And I'm sorry about the import expansions in the Java classes. I think I 
need a separate version of Eclipse for the Apache projects without all 
the settings I have for the other Java projects I'm working with at the 
university. It will not happen again.

I think I will create another ticket if you don't mind. It can be 
discussed whether it is a bug, but the ManifoldCF pages cannot be 
displayed properly by Firefox if the application is deployed on Resin. 
Opera and Safari works. I have mentioned this before, and the problem is 
that the CSS file is probably not loaded. This is the only problem I 
have with ManifoldCF and Resin.

BTW, do you think the 0.2-incubating version will be released before 
June this year?

And thanks for the quick reply and for resolving this issue.

Erlend


On 04.03.11 16.32, Karl Wright wrote:
> I ran the patch against the standard postgresql tests, which do a
> number of file-system crawls, and all was well.
> So I am still very curious as to why you are seeing the hang.  Let me
> know what I can do to help chase this down.
>
> Thanks!
> Karl
>
> On Fri, Mar 4, 2011 at 9:40 AM, Karl Wright<daddywri@gmail.com>  wrote:
>> Yes, please submit the patch.
>>
>> Can you give me more details of the start up issue?  I'd like to try
>> it here; are these pages I can get at?
>>
>> The crawler leaves the "starting up" state when the seeds have been
>> added to the queue.  The fact that this seeding process does not seem
>> to be completing means that something is pretty badly broken
>> somewhere. For the web connector (which from your history I am
>> presuming you are using), the seeding phase doesn't do much else than
>> write records to the database.
>>
>> I think you may be misinterpreting how to use the Simple History
>> report.  You are supposed to select what activities you want to see
>> from the pulldown in the upper right.  By default, all of them are
>> selected, and the initial report displayed is based on that.  If it is
>> empty (that is, you see column headers but nothing else), then NO
>> activity at all has occurred in the time window given for that
>> connection.
>>
>> Thanks,
>> Karl
>>
>> On Fri, Mar 4, 2011 at 9:28 AM, Erlend Garåsen<e.f.garasen@usit.uio.no>  wrote:
>>>
>>> Maybe I should create the PostgreSQL patch without any more testing. I have
>>> some problems to run the jobs inside the application, but I doubt the reason
>>> is my PostgreSQL code changes since the same happens with Apache Derby. The
>>> status of the job says "Starting up" and nothing shows up in the
>>> manifoldcf.log nor in the history logs available inside the application.
>>>
>>> If I select "Simple History", the list is empty, but in the activities list
>>> (in the upper right corner) I see that "robots parse" is the last activity
>>> run.
>>>
>>> This is not something Resin specific since the same happens under Tomcat.
>>>
>>> My contribution is clean and I have only made a few changes in the
>>> DBInterfacePostgreSQL and ManifoldCF class. Nothing is changed in the
>>> Database super class.
>>>
>>> I haven't run "ant test-pg" since it seems to need a local Postgresql server
>>> running. But "ant test" reports no errors.
>>>
>>> I have done an svn up before I created the patch in order to include the
>>> latest changes from trunk.
>>>
>>> The documentation needs to be updated and I can suggest what to add later.
>>>
>>> BTW, it should be interesting to know what causes the problems described
>>> above since I really want to run my jobs.
>>>
>>> PS: I do not have access to the Eclipse 3.2 codestyle.xml file, even if I'm
>>> logging in, so I had to turn of formatting in Eclipse in order to keep the
>>> existing formatting (I get the following error when I click on the link to
>>> the xml file: "You are not allowed to do AttachFile on this page.")
>>>
>>> Erlend
>>>
>>>
>>>
>>> I don't think it is something PG specific either,
>>>
>>> On 25.02.11 16.15, Karl Wright wrote:
>>>>
>>>> The DBCreate command requires privs necessary to create the database
>>>> instance in the first place.  It uses the admin credentials that you
>>>> supply to it as command args.  These are NOT the same credentials used
>>>> to access the database instance, after it has been created.
>>>>
>>>> Thus, you can skip the DBCreate step if you have already created the
>>>> database by hand.
>>>>
>>>> It does not surprise me that PostgreSQL's JDBC driver uses the global
>>>> Java trust store.  Since there is only one database active for any
>>>> installation, it is perfectly OK for the global trust store to be used
>>>> for this purpose.  It will, however, probably require the addition of
>>>> some documentation as to how to set it up properly on various
>>>> platforms.
>>>>
>>>> Thanks,
>>>> Karl
>>>>
>>>> On Fri, Feb 25, 2011 at 9:46 AM, Erlend Garåsen<e.f.garasen@usit.uio.no>
>>>>   wrote:
>>>>>
>>>>> I think we need another parameter as well:
>>>>> org.apache.manifoldcf.postgresql.port
>>>>>
>>>>> I was going to test my code today, but figured out that the following
>>>>> command class needs admin privileges to the PostgreSQL database, which
I
>>>>> don't and should never have:
>>>>> org.apache.manifoldcf.core.DBCreate
>>>>>
>>>>> So I skipped that part. Anyway, the DBCreate method is also supposed
to
>>>>> work
>>>>> with my contribution if I had the sufficient privileges.
>>>>>
>>>>> I went on by using the Install method class instead, but ended up with
a
>>>>> Handshake exception:
>>>>> javax.net.ssl.SSLHandshakeException:
>>>>> sun.security.validator.ValidatorException: PKIX path building failed:
>>>>> sun.security.provider.certpath.SunCertPathBuilderException: unable to
>>>>> find
>>>>> valid certification path to requested target.
>>>>>
>>>>> After I added our PG server's certificate to my local keystore, things
>>>>> began
>>>>> to work as it should:
>>>>> ->    Agent tables installed
>>>>>
>>>>> To be continued. I have a lot of things to do at the moment, and I cannot
>>>>> test my contribution properly yet since I'm at home with a slower
>>>>> bandwidth.
>>>>>
>>>>> To be continued. :)
>>>>>
>>>>> Erlend
>>>>>
>>>>>
>>>>> On 04.02.11 16.30, Karl Wright wrote:
>>>>>>
>>>>>> Sure, feel free to create a ticket and try to come up with a patch.
>>>>>> Since the DBInterfacePostgreSQL class is responsible for any remote
>>>>>> communication, I would make those parameters local to the postgresql
>>>>>> implementation.  So the parameters would be something like:
>>>>>>
>>>>>> org.apache.manifoldcf.postgresql.hostname
>>>>>> org.apache.manifoldcf.postgresql.ssl
>>>>>>
>>>>>> or some such.  Then, you'd want to load the necessary parameters
in
>>>>>> DBInterfacePostgreSQL.java itself.  (I wouldn't bother with MySQL
>>>>>> since that's not officially supported at this time.)
>>>>>>
>>>>>> The only problem I see is that the constructor architecture of
>>>>>> Database.java may make it a challenge to load the appropriate
>>>>>> parameters in the DBInterfacePostgreSQL constructor.  But I think
you
>>>>>> could make it work, and if not, we should probably fix this anyway
to
>>>>>> make it easier.
>>>>>>
>>>>>> Thanks,
>>>>>> Karl
>>>>>>
>>>>>> On Fri, Feb 4, 2011 at 10:17 AM, Erlend Garåsen<e.f.garasen@usit.uio.no>
>>>>>>   wrote:
>>>>>>>
>>>>>>> You're right! I didn't read the command section in the documentation
>>>>>>> properly. ManifoldCF does also run on Resin now. The only problem
with
>>>>>>> Resin
>>>>>>> is that the necessary resources, such as CSS, are not loaded.
All the
>>>>>>> other
>>>>>>> errors have disappeared, which means that it should be possible
to run
>>>>>>> the
>>>>>>> application on Resin.
>>>>>>>
>>>>>>> Do you think I should create a ticket about running an external
>>>>>>> PostgreSQL
>>>>>>> database as well? I think I know which classes to modify. I see
there
>>>>>>> are
>>>>>>> classes for MySQL as well, so perhaps the issue should be generally,
>>>>>>> i.e.
>>>>>>> not PG specific.
>>>>>>>
>>>>>>> I was thinking about extending the property list like this:
>>>>>>> org.apache.manifoldcf.database.hostname (not required, defaults
to
>>>>>>> localhost)
>>>>>>> org.apache.manifoldcf.database.ssl (not required, defaults to
false)
>>>>>>>
>>>>>>> For PostgreSQL I think it should be quite easy to implement this,
but
>>>>>>> I'm
>>>>>>> not sure when it comes to MySQL. Haven't touched that database
system
>>>>>>> for
>>>>>>> years.
>>>>>>>
>>>>>>> Erlend
>>>>>>>
>>>>>>>
>>>>>>> On 04.02.11 15.42, Karl Wright wrote:
>>>>>>>>
>>>>>>>> Ok, have you followed the instructions for a multi-process
>>>>>>>> installation in how-to-build-and-deploy.html?
>>>>>>>> You will need to create the database (using DBCreate), and
install the
>>>>>>>> schema (using Install), and register the crawler agent (using
>>>>>>>> Register).  You will also need to create the properties.xml
and
>>>>>>>> logging.ini files by hand.
>>>>>>>>
>>>>>>>> Let me know if this helps.
>>>>>>>> Karl
>>>>>>>>
>>>>>>>> On Fri, Feb 4, 2011 at 9:35 AM, Erlend
>>>>>>>> Garåsen<e.f.garasen@usit.uio.no>
>>>>>>>>   wrote:
>>>>>>>>>
>>>>>>>>> I guess the problem is that the database does not exist.
When I
>>>>>>>>> installed
>>>>>>>>> it
>>>>>>>>> on Tomcat, I got an error message telling me that the
schema TEST
>>>>>>>>> does
>>>>>>>>> not
>>>>>>>>> exist. I tried to build the database with the username
and password
>>>>>>>>> "test"
>>>>>>>>> by the following command:
>>>>>>>>> ./processes/script/executecommand.sh
>>>>>>>>> org.apache.manifoldcf.core.DBCreate
>>>>>>>>> test test
>>>>>>>>>
>>>>>>>>> And I also configured these two lines in my properties.xml
file:
>>>>>>>>>   <property name="org.apache.manifoldcf.database.username"
>>>>>>>>> value="test"/>
>>>>>>>>>   <property name="org.apache.manifoldcf.database.password"
>>>>>>>>> value="test"/>
>>>>>>>>>
>>>>>>>>> I'm using the Derby database since I cannot see how I
could contact
>>>>>>>>> an
>>>>>>>>> external PostgreSQL server on another host. PG is one
of several
>>>>>>>>> database
>>>>>>>>> systems the university supports. I guess there should
be an option
>>>>>>>>> for
>>>>>>>>> configuring external hosts and optionally turn on SSL
encryption.
>>>>>>>>> Maybe
>>>>>>>>> another ticket to create.
>>>>>>>>>
>>>>>>>>> But first I want to run ManifoldCF on Tomcat using Derby,
then deploy
>>>>>>>>> it
>>>>>>>>> on
>>>>>>>>> Resin and then try to configure an external PostgreSQL
server.
>>>>>>>>>
>>>>>>>>> Erlend
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 03.02.11 16.32, Karl Wright wrote:
>>>>>>>>>>
>>>>>>>>>> Sure, if you want, create a ticket, and see if you
can figure out
>>>>>>>>>> what
>>>>>>>>>> the problems are.  I'd start with not being able
to reach the
>>>>>>>>>> stylesheet, actually.  The "redirection" error sound
like the
>>>>>>>>>> crawler
>>>>>>>>>> UI is trying to present an error and resin is handling
one of the
>>>>>>>>>> java
>>>>>>>>>> standard tags in a non-standard way.
>>>>>>>>>>
>>>>>>>>>> Karl
>>>>>>>>>>
>>>>>>>>>> On Thu, Feb 3, 2011 at 10:28 AM, Erlend
>>>>>>>>>> Garåsen<e.f.garasen@usit.uio.no>
>>>>>>>>>>   wrote:
>>>>>>>>>>>
>>>>>>>>>>> Resin is de facto application server at the university,
but we also
>>>>>>>>>>> support
>>>>>>>>>>> Tomcat.
>>>>>>>>>>>
>>>>>>>>>>> I can try to deploy the war files on Tomcat instead.
The error
>>>>>>>>>>> message
>>>>>>>>>>> comes
>>>>>>>>>>> from Resin, probably caused by a redirect after
the http header is
>>>>>>>>>>> written.
>>>>>>>>>>> I can notify you, eventually create a patch,
if I manage to run
>>>>>>>>>>> ManifoldCF
>>>>>>>>>>> on Resin.
>>>>>>>>>>>
>>>>>>>>>>> Erlend
>>>>>>>>>>>
>>>>>>>>>>> On 03.02.11 16.16, Karl Wright wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I have never tried this on Resin, just on
Tomcat and Jetty.
>>>>>>>>>>>>   Clearly
>>>>>>>>>>>> Resin as an application server is having
difficulty doing some
>>>>>>>>>>>> very
>>>>>>>>>>>> basic stuff.
>>>>>>>>>>>>
>>>>>>>>>>>> FWIW, when we first started the Jetty development,
I had to do
>>>>>>>>>>>> nothing
>>>>>>>>>>>> whatsoever to the crawler UI or the war packaging
to make it work.
>>>>>>>>>>>>   It
>>>>>>>>>>>> just worked.  Your experiences with Resin
are making me wonder if
>>>>>>>>>>>> Resin is indeed working up to Java standards.
>>>>>>>>>>>>
>>>>>>>>>>>> Karl
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Feb 3, 2011 at 9:31 AM, Erlend
>>>>>>>>>>>> Garåsen<e.f.garasen@usit.uio.no>
>>>>>>>>>>>>   wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have placed the three war files into
Resin's webapps directory
>>>>>>>>>>>>> and
>>>>>>>>>>>>> set
>>>>>>>>>>>>> the
>>>>>>>>>>>>> following system variable inside resin.xml:
>>>>>>>>>>>>>
>>>>>>>>>>>>> <system-property
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.manifoldcf.configfile="/Users/erlendfg/apps/manifoldcf/properties.xml"
>>>>>>>>>>>>> />
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have also created my own home directory
for ManifoldCF and
>>>>>>>>>>>>> copied
>>>>>>>>>>>>> the
>>>>>>>>>>>>> necessary files from the example directory.
When I start Resin
>>>>>>>>>>>>> and
>>>>>>>>>>>>> goes
>>>>>>>>>>>>> to
>>>>>>>>>>>>> ... mcf-crawler-ui/, the start page shows
up, but the page looks
>>>>>>>>>>>>> ugly
>>>>>>>>>>>>> (I
>>>>>>>>>>>>> guess the path to CSS files etc. are
broken). And when I try to
>>>>>>>>>>>>> click
>>>>>>>>>>>>> on
>>>>>>>>>>>>> a
>>>>>>>>>>>>> link, e.g. "List Output Connections",
I get the following error:
>>>>>>>>>>>>> java.lang.IllegalStateException: can't
forward after writing HTTP
>>>>>>>>>>>>> headers
>>>>>>>>>>>>>
>>>>>>>>>>>>> I find nothing in manifoldcf.log.
>>>>>>>>>>>>>
>>>>>>>>>>>>> BTW: PostgreSQL is not configured. I
will start using Derby until
>>>>>>>>>>>>> it
>>>>>>>>>>>>> runs
>>>>>>>>>>>>> stable on Resin.
>>>>>>>>>>>>>
>>>>>>>>>>>>> E
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Erlend Garåsen
>>>>>>>>>>>>> Center for Information Technology Services
>>>>>>>>>>>>> University of Oslo
>>>>>>>>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO,
Norway
>>>>>>>>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970,
Mobile: (+47) 91380968,
>>>>>>>>>>>>> VIP:
>>>>>>>>>>>>> 31050
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Erlend Garåsen
>>>>>>>>>>> Center for Information Technology Services
>>>>>>>>>>> University of Oslo
>>>>>>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>>>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile:
(+47) 91380968,
>>>>>>>>>>> VIP:
>>>>>>>>>>> 31050
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Erlend Garåsen
>>>>>>>>> Center for Information Technology Services
>>>>>>>>> University of Oslo
>>>>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47)
91380968, VIP:
>>>>>>>>> 31050
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Erlend Garåsen
>>>>>>> Center for Information Technology Services
>>>>>>> University of Oslo
>>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968,
VIP:
>>>>>>> 31050
>>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Erlend Garåsen
>>>>> Center for Information Technology Services
>>>>> University of Oslo
>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP:
>>>>> 31050
>>>>>
>>>
>>>
>>> --
>>> Erlend Garåsen
>>> Center for Information Technology Services
>>> University of Oslo
>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
>>>
>>


-- 
Erlend Garåsen
Center for Information Technology Services
University of Oslo
P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050

Mime
View raw message