manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Running ManifoldCF on Resin
Date Fri, 04 Feb 2011 16:07:22 GMT
It looks like it's unable to connect to Solr.

This is currently treated as a "hard error" and it will abort the crawl.

Karl

On Fri, Feb 4, 2011 at 10:58 AM, Karl Wright <daddywri@gmail.com> wrote:
> Do you have a full stack trace with that message?  I would expect one
> in manifoldcf.log.
> Karl
>
>
> On Fri, Feb 4, 2011 at 10:55 AM, Erlend Garåsen <e.f.garasen@usit.uio.no> wrote:
>>
>> OK, I will create two issues, but the patches will probably not be created
>> until I'm back in Norway. I'm going to Spain next week and will work with
>> completely different things the next two weeks.
>>
>> BTW, the Resin issue is quite funny. The CSS file is loaded if I use Safari.
>> It does not work on Firefox if ManifoldCF is deployed on Resin. It seems
>> that everything works, except I cannot start a new job, perhaps because I
>> have forgot to start a particular agent. I'm getting this error:
>> Error communicating with Ingestion API: Connection refused
>>
>> Erlend
>>
>>
>> On 04.02.11 16.30, Karl Wright wrote:
>>>
>>> Sure, feel free to create a ticket and try to come up with a patch.
>>> Since the DBInterfacePostgreSQL class is responsible for any remote
>>> communication, I would make those parameters local to the postgresql
>>> implementation.  So the parameters would be something like:
>>>
>>> org.apache.manifoldcf.postgresql.hostname
>>> org.apache.manifoldcf.postgresql.ssl
>>>
>>> or some such.  Then, you'd want to load the necessary parameters in
>>> DBInterfacePostgreSQL.java itself.  (I wouldn't bother with MySQL
>>> since that's not officially supported at this time.)
>>>
>>> The only problem I see is that the constructor architecture of
>>> Database.java may make it a challenge to load the appropriate
>>> parameters in the DBInterfacePostgreSQL constructor.  But I think you
>>> could make it work, and if not, we should probably fix this anyway to
>>> make it easier.
>>>
>>> Thanks,
>>> Karl
>>>
>>> On Fri, Feb 4, 2011 at 10:17 AM, Erlend Garåsen<e.f.garasen@usit.uio.no>
>>>  wrote:
>>>>
>>>> You're right! I didn't read the command section in the documentation
>>>> properly. ManifoldCF does also run on Resin now. The only problem with
>>>> Resin
>>>> is that the necessary resources, such as CSS, are not loaded. All the
>>>> other
>>>> errors have disappeared, which means that it should be possible to run
>>>> the
>>>> application on Resin.
>>>>
>>>> Do you think I should create a ticket about running an external
>>>> PostgreSQL
>>>> database as well? I think I know which classes to modify. I see there are
>>>> classes for MySQL as well, so perhaps the issue should be generally, i.e.
>>>> not PG specific.
>>>>
>>>> I was thinking about extending the property list like this:
>>>> org.apache.manifoldcf.database.hostname (not required, defaults to
>>>> localhost)
>>>> org.apache.manifoldcf.database.ssl (not required, defaults to false)
>>>>
>>>> For PostgreSQL I think it should be quite easy to implement this, but I'm
>>>> not sure when it comes to MySQL. Haven't touched that database system for
>>>> years.
>>>>
>>>> Erlend
>>>>
>>>>
>>>> On 04.02.11 15.42, Karl Wright wrote:
>>>>>
>>>>> Ok, have you followed the instructions for a multi-process
>>>>> installation in how-to-build-and-deploy.html?
>>>>> You will need to create the database (using DBCreate), and install the
>>>>> schema (using Install), and register the crawler agent (using
>>>>> Register).  You will also need to create the properties.xml and
>>>>> logging.ini files by hand.
>>>>>
>>>>> Let me know if this helps.
>>>>> Karl
>>>>>
>>>>> On Fri, Feb 4, 2011 at 9:35 AM, Erlend Garåsen<e.f.garasen@usit.uio.no>
>>>>>  wrote:
>>>>>>
>>>>>> I guess the problem is that the database does not exist. When I
>>>>>> installed
>>>>>> it
>>>>>> on Tomcat, I got an error message telling me that the schema TEST
does
>>>>>> not
>>>>>> exist. I tried to build the database with the username and password
>>>>>> "test"
>>>>>> by the following command:
>>>>>> ./processes/script/executecommand.sh
>>>>>> org.apache.manifoldcf.core.DBCreate
>>>>>> test test
>>>>>>
>>>>>> And I also configured these two lines in my properties.xml file:
>>>>>>  <property name="org.apache.manifoldcf.database.username"
>>>>>> value="test"/>
>>>>>>  <property name="org.apache.manifoldcf.database.password"
>>>>>> value="test"/>
>>>>>>
>>>>>> I'm using the Derby database since I cannot see how I could contact
an
>>>>>> external PostgreSQL server on another host. PG is one of several
>>>>>> database
>>>>>> systems the university supports. I guess there should be an option
for
>>>>>> configuring external hosts and optionally turn on SSL encryption.
Maybe
>>>>>> another ticket to create.
>>>>>>
>>>>>> But first I want to run ManifoldCF on Tomcat using Derby, then deploy
>>>>>> it
>>>>>> on
>>>>>> Resin and then try to configure an external PostgreSQL server.
>>>>>>
>>>>>> Erlend
>>>>>>
>>>>>>
>>>>>> On 03.02.11 16.32, Karl Wright wrote:
>>>>>>>
>>>>>>> Sure, if you want, create a ticket, and see if you can figure
out what
>>>>>>> the problems are.  I'd start with not being able to reach the
>>>>>>> stylesheet, actually.  The "redirection" error sound like the
crawler
>>>>>>> UI is trying to present an error and resin is handling one of
the java
>>>>>>> standard tags in a non-standard way.
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>> On Thu, Feb 3, 2011 at 10:28 AM, Erlend
>>>>>>> Garåsen<e.f.garasen@usit.uio.no>
>>>>>>>  wrote:
>>>>>>>>
>>>>>>>> Resin is de facto application server at the university, but
we also
>>>>>>>> support
>>>>>>>> Tomcat.
>>>>>>>>
>>>>>>>> I can try to deploy the war files on Tomcat instead. The
error
>>>>>>>> message
>>>>>>>> comes
>>>>>>>> from Resin, probably caused by a redirect after the http
header is
>>>>>>>> written.
>>>>>>>> I can notify you, eventually create a patch, if I manage
to run
>>>>>>>> ManifoldCF
>>>>>>>> on Resin.
>>>>>>>>
>>>>>>>> Erlend
>>>>>>>>
>>>>>>>> On 03.02.11 16.16, Karl Wright wrote:
>>>>>>>>>
>>>>>>>>> I have never tried this on Resin, just on Tomcat and
Jetty.  Clearly
>>>>>>>>> Resin as an application server is having difficulty doing
some very
>>>>>>>>> basic stuff.
>>>>>>>>>
>>>>>>>>> FWIW, when we first started the Jetty development, I
had to do
>>>>>>>>> nothing
>>>>>>>>> whatsoever to the crawler UI or the war packaging to
make it work.
>>>>>>>>>  It
>>>>>>>>> just worked.  Your experiences with Resin are making
me wonder if
>>>>>>>>> Resin is indeed working up to Java standards.
>>>>>>>>>
>>>>>>>>> Karl
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Feb 3, 2011 at 9:31 AM, Erlend
>>>>>>>>> Garåsen<e.f.garasen@usit.uio.no>
>>>>>>>>>  wrote:
>>>>>>>>>>
>>>>>>>>>> I have placed the three war files into Resin's webapps
directory
>>>>>>>>>> and
>>>>>>>>>> set
>>>>>>>>>> the
>>>>>>>>>> following system variable inside resin.xml:
>>>>>>>>>>
>>>>>>>>>> <system-property
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.manifoldcf.configfile="/Users/erlendfg/apps/manifoldcf/properties.xml"
>>>>>>>>>> />
>>>>>>>>>>
>>>>>>>>>> I have also created my own home directory for ManifoldCF
and copied
>>>>>>>>>> the
>>>>>>>>>> necessary files from the example directory. When
I start Resin and
>>>>>>>>>> goes
>>>>>>>>>> to
>>>>>>>>>> ... mcf-crawler-ui/, the start page shows up, but
the page looks
>>>>>>>>>> ugly
>>>>>>>>>> (I
>>>>>>>>>> guess the path to CSS files etc. are broken). And
when I try to
>>>>>>>>>> click
>>>>>>>>>> on
>>>>>>>>>> a
>>>>>>>>>> link, e.g. "List Output Connections", I get the following
error:
>>>>>>>>>> java.lang.IllegalStateException: can't forward after
writing HTTP
>>>>>>>>>> headers
>>>>>>>>>>
>>>>>>>>>> I find nothing in manifoldcf.log.
>>>>>>>>>>
>>>>>>>>>> BTW: PostgreSQL is not configured. I will start using
Derby until
>>>>>>>>>> it
>>>>>>>>>> runs
>>>>>>>>>> stable on Resin.
>>>>>>>>>>
>>>>>>>>>> E
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Erlend Garåsen
>>>>>>>>>> Center for Information Technology Services
>>>>>>>>>> University of Oslo
>>>>>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile:
(+47) 91380968,
>>>>>>>>>> VIP:
>>>>>>>>>> 31050
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Erlend Garåsen
>>>>>>>> Center for Information Technology Services
>>>>>>>> University of Oslo
>>>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968,
VIP:
>>>>>>>> 31050
>>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Erlend Garåsen
>>>>>> Center for Information Technology Services
>>>>>> University of Oslo
>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968,
VIP:
>>>>>> 31050
>>>>>>
>>>>
>>>>
>>>> --
>>>> Erlend Garåsen
>>>> Center for Information Technology Services
>>>> University of Oslo
>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP:
>>>> 31050
>>>>
>>
>>
>> --
>> Erlend Garåsen
>> Center for Information Technology Services
>> University of Oslo
>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
>>
>

Mime
View raw message