manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erlend Garåsen <e.f.gara...@usit.uio.no>
Subject Re: Running ManifoldCF on Resin
Date Mon, 07 Mar 2011 16:21:10 GMT

It seems that the reason why the CSS file is not processed correctly is 
due to a bug in Resin:
http://bugs.caucho.com/view.php?id=4180

It's caused by the MacRoman char set which prevents Firefox to read the 
CSS file. This bug is fixed in the latest version of Resin, 4.0.15, 
which was released for a month ago.

Erlend

On 07.03.11 12.13, Karl Wright wrote:
> Hi Erlend,
>
> I'd originally proposed 4/15 for 0.2 release.  And yes, please open a
> new ticket if there are problems running under Resin; it would be
> great to figure out what the issue was.  I'll triage this problem is
> required for 0.2 release.
>
> We do have to get the release candidate together by 3/22 or so if
> we're going to make that proposed release schedule, because we need to
> get people to sign off, and that takes time.  So I hope you'd be able
> to do a little research on it between now and then.
>
> Thanks,
> Karl
>
> On Mon, Mar 7, 2011 at 5:24 AM, Erlend Garåsen<e.f.garasen@usit.uio.no>  wrote:
>>
>> I'm sorry about the late reply, I have been out of office.
>>
>> The reason why it didn't work is because I didn't read the documentation
>> properly. I simply forgot to run the following command after I started
>> Resin:
>> ./processes/script/executecommand.sh org.apache.manifoldcf.agents.AgentRun
>>
>> I thought it was sufficient to start my application server.
>>
>> And I'm sorry about the import expansions in the Java classes. I think I
>> need a separate version of Eclipse for the Apache projects without all the
>> settings I have for the other Java projects I'm working with at the
>> university. It will not happen again.
>>
>> I think I will create another ticket if you don't mind. It can be discussed
>> whether it is a bug, but the ManifoldCF pages cannot be displayed properly
>> by Firefox if the application is deployed on Resin. Opera and Safari works.
>> I have mentioned this before, and the problem is that the CSS file is
>> probably not loaded. This is the only problem I have with ManifoldCF and
>> Resin.
>>
>> BTW, do you think the 0.2-incubating version will be released before June
>> this year?
>>
>> And thanks for the quick reply and for resolving this issue.
>>
>> Erlend
>>
>>
>> On 04.03.11 16.32, Karl Wright wrote:
>>>
>>> I ran the patch against the standard postgresql tests, which do a
>>> number of file-system crawls, and all was well.
>>> So I am still very curious as to why you are seeing the hang.  Let me
>>> know what I can do to help chase this down.
>>>
>>> Thanks!
>>> Karl
>>>
>>> On Fri, Mar 4, 2011 at 9:40 AM, Karl Wright<daddywri@gmail.com>    wrote:
>>>>
>>>> Yes, please submit the patch.
>>>>
>>>> Can you give me more details of the start up issue?  I'd like to try
>>>> it here; are these pages I can get at?
>>>>
>>>> The crawler leaves the "starting up" state when the seeds have been
>>>> added to the queue.  The fact that this seeding process does not seem
>>>> to be completing means that something is pretty badly broken
>>>> somewhere. For the web connector (which from your history I am
>>>> presuming you are using), the seeding phase doesn't do much else than
>>>> write records to the database.
>>>>
>>>> I think you may be misinterpreting how to use the Simple History
>>>> report.  You are supposed to select what activities you want to see
>>>> from the pulldown in the upper right.  By default, all of them are
>>>> selected, and the initial report displayed is based on that.  If it is
>>>> empty (that is, you see column headers but nothing else), then NO
>>>> activity at all has occurred in the time window given for that
>>>> connection.
>>>>
>>>> Thanks,
>>>> Karl
>>>>
>>>> On Fri, Mar 4, 2011 at 9:28 AM, Erlend Garåsen<e.f.garasen@usit.uio.no>
>>>>   wrote:
>>>>>
>>>>> Maybe I should create the PostgreSQL patch without any more testing.
I
>>>>> have
>>>>> some problems to run the jobs inside the application, but I doubt the
>>>>> reason
>>>>> is my PostgreSQL code changes since the same happens with Apache Derby.
>>>>> The
>>>>> status of the job says "Starting up" and nothing shows up in the
>>>>> manifoldcf.log nor in the history logs available inside the application.
>>>>>
>>>>> If I select "Simple History", the list is empty, but in the activities
>>>>> list
>>>>> (in the upper right corner) I see that "robots parse" is the last
>>>>> activity
>>>>> run.
>>>>>
>>>>> This is not something Resin specific since the same happens under
>>>>> Tomcat.
>>>>>
>>>>> My contribution is clean and I have only made a few changes in the
>>>>> DBInterfacePostgreSQL and ManifoldCF class. Nothing is changed in the
>>>>> Database super class.
>>>>>
>>>>> I haven't run "ant test-pg" since it seems to need a local Postgresql
>>>>> server
>>>>> running. But "ant test" reports no errors.
>>>>>
>>>>> I have done an svn up before I created the patch in order to include
the
>>>>> latest changes from trunk.
>>>>>
>>>>> The documentation needs to be updated and I can suggest what to add
>>>>> later.
>>>>>
>>>>> BTW, it should be interesting to know what causes the problems described
>>>>> above since I really want to run my jobs.
>>>>>
>>>>> PS: I do not have access to the Eclipse 3.2 codestyle.xml file, even
if
>>>>> I'm
>>>>> logging in, so I had to turn of formatting in Eclipse in order to keep
>>>>> the
>>>>> existing formatting (I get the following error when I click on the link
>>>>> to
>>>>> the xml file: "You are not allowed to do AttachFile on this page.")
>>>>>
>>>>> Erlend
>>>>>
>>>>>
>>>>>
>>>>> I don't think it is something PG specific either,
>>>>>
>>>>> On 25.02.11 16.15, Karl Wright wrote:
>>>>>>
>>>>>> The DBCreate command requires privs necessary to create the database
>>>>>> instance in the first place.  It uses the admin credentials that
you
>>>>>> supply to it as command args.  These are NOT the same credentials
used
>>>>>> to access the database instance, after it has been created.
>>>>>>
>>>>>> Thus, you can skip the DBCreate step if you have already created
the
>>>>>> database by hand.
>>>>>>
>>>>>> It does not surprise me that PostgreSQL's JDBC driver uses the global
>>>>>> Java trust store.  Since there is only one database active for any
>>>>>> installation, it is perfectly OK for the global trust store to be
used
>>>>>> for this purpose.  It will, however, probably require the addition
of
>>>>>> some documentation as to how to set it up properly on various
>>>>>> platforms.
>>>>>>
>>>>>> Thanks,
>>>>>> Karl
>>>>>>
>>>>>> On Fri, Feb 25, 2011 at 9:46 AM, Erlend
>>>>>> Garåsen<e.f.garasen@usit.uio.no>
>>>>>>   wrote:
>>>>>>>
>>>>>>> I think we need another parameter as well:
>>>>>>> org.apache.manifoldcf.postgresql.port
>>>>>>>
>>>>>>> I was going to test my code today, but figured out that the following
>>>>>>> command class needs admin privileges to the PostgreSQL database,
which
>>>>>>> I
>>>>>>> don't and should never have:
>>>>>>> org.apache.manifoldcf.core.DBCreate
>>>>>>>
>>>>>>> So I skipped that part. Anyway, the DBCreate method is also supposed
>>>>>>> to
>>>>>>> work
>>>>>>> with my contribution if I had the sufficient privileges.
>>>>>>>
>>>>>>> I went on by using the Install method class instead, but ended
up with
>>>>>>> a
>>>>>>> Handshake exception:
>>>>>>> javax.net.ssl.SSLHandshakeException:
>>>>>>> sun.security.validator.ValidatorException: PKIX path building
failed:
>>>>>>> sun.security.provider.certpath.SunCertPathBuilderException: unable
to
>>>>>>> find
>>>>>>> valid certification path to requested target.
>>>>>>>
>>>>>>> After I added our PG server's certificate to my local keystore,
things
>>>>>>> began
>>>>>>> to work as it should:
>>>>>>> ->      Agent tables installed
>>>>>>>
>>>>>>> To be continued. I have a lot of things to do at the moment,
and I
>>>>>>> cannot
>>>>>>> test my contribution properly yet since I'm at home with a slower
>>>>>>> bandwidth.
>>>>>>>
>>>>>>> To be continued. :)
>>>>>>>
>>>>>>> Erlend
>>>>>>>
>>>>>>>
>>>>>>> On 04.02.11 16.30, Karl Wright wrote:
>>>>>>>>
>>>>>>>> Sure, feel free to create a ticket and try to come up with
a patch.
>>>>>>>> Since the DBInterfacePostgreSQL class is responsible for
any remote
>>>>>>>> communication, I would make those parameters local to the
postgresql
>>>>>>>> implementation.  So the parameters would be something like:
>>>>>>>>
>>>>>>>> org.apache.manifoldcf.postgresql.hostname
>>>>>>>> org.apache.manifoldcf.postgresql.ssl
>>>>>>>>
>>>>>>>> or some such.  Then, you'd want to load the necessary parameters
in
>>>>>>>> DBInterfacePostgreSQL.java itself.  (I wouldn't bother with
MySQL
>>>>>>>> since that's not officially supported at this time.)
>>>>>>>>
>>>>>>>> The only problem I see is that the constructor architecture
of
>>>>>>>> Database.java may make it a challenge to load the appropriate
>>>>>>>> parameters in the DBInterfacePostgreSQL constructor.  But
I think you
>>>>>>>> could make it work, and if not, we should probably fix this
anyway to
>>>>>>>> make it easier.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Karl
>>>>>>>>
>>>>>>>> On Fri, Feb 4, 2011 at 10:17 AM, Erlend
>>>>>>>> Garåsen<e.f.garasen@usit.uio.no>
>>>>>>>>   wrote:
>>>>>>>>>
>>>>>>>>> You're right! I didn't read the command section in the
documentation
>>>>>>>>> properly. ManifoldCF does also run on Resin now. The
only problem
>>>>>>>>> with
>>>>>>>>> Resin
>>>>>>>>> is that the necessary resources, such as CSS, are not
loaded. All
>>>>>>>>> the
>>>>>>>>> other
>>>>>>>>> errors have disappeared, which means that it should be
possible to
>>>>>>>>> run
>>>>>>>>> the
>>>>>>>>> application on Resin.
>>>>>>>>>
>>>>>>>>> Do you think I should create a ticket about running an
external
>>>>>>>>> PostgreSQL
>>>>>>>>> database as well? I think I know which classes to modify.
I see
>>>>>>>>> there
>>>>>>>>> are
>>>>>>>>> classes for MySQL as well, so perhaps the issue should
be generally,
>>>>>>>>> i.e.
>>>>>>>>> not PG specific.
>>>>>>>>>
>>>>>>>>> I was thinking about extending the property list like
this:
>>>>>>>>> org.apache.manifoldcf.database.hostname (not required,
defaults to
>>>>>>>>> localhost)
>>>>>>>>> org.apache.manifoldcf.database.ssl (not required, defaults
to false)
>>>>>>>>>
>>>>>>>>> For PostgreSQL I think it should be quite easy to implement
this,
>>>>>>>>> but
>>>>>>>>> I'm
>>>>>>>>> not sure when it comes to MySQL. Haven't touched that
database
>>>>>>>>> system
>>>>>>>>> for
>>>>>>>>> years.
>>>>>>>>>
>>>>>>>>> Erlend
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 04.02.11 15.42, Karl Wright wrote:
>>>>>>>>>>
>>>>>>>>>> Ok, have you followed the instructions for a multi-process
>>>>>>>>>> installation in how-to-build-and-deploy.html?
>>>>>>>>>> You will need to create the database (using DBCreate),
and install
>>>>>>>>>> the
>>>>>>>>>> schema (using Install), and register the crawler
agent (using
>>>>>>>>>> Register).  You will also need to create the properties.xml
and
>>>>>>>>>> logging.ini files by hand.
>>>>>>>>>>
>>>>>>>>>> Let me know if this helps.
>>>>>>>>>> Karl
>>>>>>>>>>
>>>>>>>>>> On Fri, Feb 4, 2011 at 9:35 AM, Erlend
>>>>>>>>>> Garåsen<e.f.garasen@usit.uio.no>
>>>>>>>>>>   wrote:
>>>>>>>>>>>
>>>>>>>>>>> I guess the problem is that the database does
not exist. When I
>>>>>>>>>>> installed
>>>>>>>>>>> it
>>>>>>>>>>> on Tomcat, I got an error message telling me
that the schema TEST
>>>>>>>>>>> does
>>>>>>>>>>> not
>>>>>>>>>>> exist. I tried to build the database with the
username and
>>>>>>>>>>> password
>>>>>>>>>>> "test"
>>>>>>>>>>> by the following command:
>>>>>>>>>>> ./processes/script/executecommand.sh
>>>>>>>>>>> org.apache.manifoldcf.core.DBCreate
>>>>>>>>>>> test test
>>>>>>>>>>>
>>>>>>>>>>> And I also configured these two lines in my properties.xml
file:
>>>>>>>>>>>   <property name="org.apache.manifoldcf.database.username"
>>>>>>>>>>> value="test"/>
>>>>>>>>>>>   <property name="org.apache.manifoldcf.database.password"
>>>>>>>>>>> value="test"/>
>>>>>>>>>>>
>>>>>>>>>>> I'm using the Derby database since I cannot see
how I could
>>>>>>>>>>> contact
>>>>>>>>>>> an
>>>>>>>>>>> external PostgreSQL server on another host. PG
is one of several
>>>>>>>>>>> database
>>>>>>>>>>> systems the university supports. I guess there
should be an option
>>>>>>>>>>> for
>>>>>>>>>>> configuring external hosts and optionally turn
on SSL encryption.
>>>>>>>>>>> Maybe
>>>>>>>>>>> another ticket to create.
>>>>>>>>>>>
>>>>>>>>>>> But first I want to run ManifoldCF on Tomcat
using Derby, then
>>>>>>>>>>> deploy
>>>>>>>>>>> it
>>>>>>>>>>> on
>>>>>>>>>>> Resin and then try to configure an external PostgreSQL
server.
>>>>>>>>>>>
>>>>>>>>>>> Erlend
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 03.02.11 16.32, Karl Wright wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Sure, if you want, create a ticket, and see
if you can figure out
>>>>>>>>>>>> what
>>>>>>>>>>>> the problems are.  I'd start with not being
able to reach the
>>>>>>>>>>>> stylesheet, actually.  The "redirection"
error sound like the
>>>>>>>>>>>> crawler
>>>>>>>>>>>> UI is trying to present an error and resin
is handling one of the
>>>>>>>>>>>> java
>>>>>>>>>>>> standard tags in a non-standard way.
>>>>>>>>>>>>
>>>>>>>>>>>> Karl
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Feb 3, 2011 at 10:28 AM, Erlend
>>>>>>>>>>>> Garåsen<e.f.garasen@usit.uio.no>
>>>>>>>>>>>>   wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Resin is de facto application server
at the university, but we
>>>>>>>>>>>>> also
>>>>>>>>>>>>> support
>>>>>>>>>>>>> Tomcat.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I can try to deploy the war files on
Tomcat instead. The error
>>>>>>>>>>>>> message
>>>>>>>>>>>>> comes
>>>>>>>>>>>>> from Resin, probably caused by a redirect
after the http header
>>>>>>>>>>>>> is
>>>>>>>>>>>>> written.
>>>>>>>>>>>>> I can notify you, eventually create a
patch, if I manage to run
>>>>>>>>>>>>> ManifoldCF
>>>>>>>>>>>>> on Resin.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Erlend
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 03.02.11 16.16, Karl Wright wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have never tried this on Resin,
just on Tomcat and Jetty.
>>>>>>>>>>>>>>   Clearly
>>>>>>>>>>>>>> Resin as an application server is
having difficulty doing some
>>>>>>>>>>>>>> very
>>>>>>>>>>>>>> basic stuff.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> FWIW, when we first started the Jetty
development, I had to do
>>>>>>>>>>>>>> nothing
>>>>>>>>>>>>>> whatsoever to the crawler UI or the
war packaging to make it
>>>>>>>>>>>>>> work.
>>>>>>>>>>>>>>   It
>>>>>>>>>>>>>> just worked.  Your experiences with
Resin are making me wonder
>>>>>>>>>>>>>> if
>>>>>>>>>>>>>> Resin is indeed working up to Java
standards.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Feb 3, 2011 at 9:31 AM, Erlend
>>>>>>>>>>>>>> Garåsen<e.f.garasen@usit.uio.no>
>>>>>>>>>>>>>>   wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have placed the three war files
into Resin's webapps
>>>>>>>>>>>>>>> directory
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> following system variable inside
resin.xml:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> <system-property
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> org.apache.manifoldcf.configfile="/Users/erlendfg/apps/manifoldcf/properties.xml"
>>>>>>>>>>>>>>> />
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have also created my own home
directory for ManifoldCF and
>>>>>>>>>>>>>>> copied
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> necessary files from the example
directory. When I start Resin
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> ... mcf-crawler-ui/, the start
page shows up, but the page
>>>>>>>>>>>>>>> looks
>>>>>>>>>>>>>>> ugly
>>>>>>>>>>>>>>> (I
>>>>>>>>>>>>>>> guess the path to CSS files etc.
are broken). And when I try
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> click
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> link, e.g. "List Output Connections",
I get the following
>>>>>>>>>>>>>>> error:
>>>>>>>>>>>>>>> java.lang.IllegalStateException:
can't forward after writing
>>>>>>>>>>>>>>> HTTP
>>>>>>>>>>>>>>> headers
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I find nothing in manifoldcf.log.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> BTW: PostgreSQL is not configured.
I will start using Derby
>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> runs
>>>>>>>>>>>>>>> stable on Resin.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> E
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Erlend Garåsen
>>>>>>>>>>>>>>> Center for Information Technology
Services
>>>>>>>>>>>>>>> University of Oslo
>>>>>>>>>>>>>>> P.O. Box 1086 Blindern, N-0317
OSLO, Norway
>>>>>>>>>>>>>>> Ph: (+47) 22840193, Fax: (+47)
22852970, Mobile: (+47)
>>>>>>>>>>>>>>> 91380968,
>>>>>>>>>>>>>>> VIP:
>>>>>>>>>>>>>>> 31050
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Erlend Garåsen
>>>>>>>>>>>>> Center for Information Technology Services
>>>>>>>>>>>>> University of Oslo
>>>>>>>>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO,
Norway
>>>>>>>>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970,
Mobile: (+47) 91380968,
>>>>>>>>>>>>> VIP:
>>>>>>>>>>>>> 31050
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Erlend Garåsen
>>>>>>>>>>> Center for Information Technology Services
>>>>>>>>>>> University of Oslo
>>>>>>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>>>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile:
(+47) 91380968,
>>>>>>>>>>> VIP:
>>>>>>>>>>> 31050
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Erlend Garåsen
>>>>>>>>> Center for Information Technology Services
>>>>>>>>> University of Oslo
>>>>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47)
91380968,
>>>>>>>>> VIP:
>>>>>>>>> 31050
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Erlend Garåsen
>>>>>>> Center for Information Technology Services
>>>>>>> University of Oslo
>>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968,
VIP:
>>>>>>> 31050
>>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Erlend Garåsen
>>>>> Center for Information Technology Services
>>>>> University of Oslo
>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP:
>>>>> 31050
>>>>>
>>>>
>>
>>
>> --
>> Erlend Garåsen
>> Center for Information Technology Services
>> University of Oslo
>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
>>


-- 
Erlend Garåsen
Center for Information Technology Services
University of Oslo
P.O. Box 1086 Blindern, N-0317 OSLO, Norway
Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050

Mime
View raw message