manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörn Franke <jornfra...@gmail.com>
Subject Re: CSWS Connector : ServiceConstructionException: Failed to create service
Date Thu, 16 Jan 2020 17:07:57 GMT
Ok I understand. I will try and let you know. Thanks again very much for your fast and detailed
answer. Really appreciated. I hope I can give back with the solution to fetch WSDLs from https
and maybe a solution to this problem (maybe other will face this as well).

About the connector: the WSDL is successfully fetched via https (not file - no clue why) -
after the modification I made. The only problem I see now is that the xsd to which the WSDL
is referring are not fetched. The bizarre thing is that the https url that it mention for
the xsd is absolutely correct. So I assume it does not understand an http url, maybe that
is related to configuration.

> Am 16.01.2020 um 14:53 schrieb Karl Wright <daddywri@gmail.com>:
> 
> 
> The WSDLS are bundled with the jar.  We intended this to be the ONLY way the wsdls were
accessed, and made lots of changes to the wsdls accordingly, so that they referenced other
wsdls via the "file system".  The wsdls are the fixed up ones that are used to build the java
stubs locally, plus a config file that's supposed to tell CXF how to resolve referenced wsdls.
 That config file may or may not be correct, because we never were able to get CXF to use
the local resource wsdls during actual connection.
> 
> Except now they seem to be both fetched via https AND locally sourced.  I have no idea
how that can be.  I had assumed it was done one way or the other but not both.
> 
> Perhaps the problem is that the configuration file is being read but the resource wsdls
are not being found?  Removing the meta-inf from the jar would then force everything to go
through https.  Ideally I'd love it if that wasn't needed and we could get the resource fetch
working everywhere.
> 
> 
> Karl
> 
> 
>> On Thu, Jan 16, 2020 at 8:20 AM Jörn Franke <jornfranke@gmail.com> wrote:
>> Well i am not sure how they solved it - I will share a tested solution in Jira and
everyone can check. Maybe their wsdl is accessible through http?
>> 
>> What works is doing call through https,  but thee fetching of WSDL did not - as this
is through another mechanism.
>> 
>> I don’t think that the open text is different, the WSDL look very similar to the
repository. 
>> 
>> The strange thing is that for this error message it tries to access the xsd through
a https url (which is perfectly accessible for the server). 
>> Could it be that the connector restrict itself somehow to local file system only
or similar?
>> Have you faced this issue before?
>> 
>> 
>> 
>>>> Am 16.01.2020 um 12:56 schrieb Karl Wright <daddywri@gmail.com>:
>>>> 
>>> 
>>> I should say that we have (AFAICT) at least two independent installations of
the csws connector working in the field, at least one of them using secure connections.
>>> 
>>> Karl
>>> 
>>> 
>>>> On Thu, Jan 16, 2020 at 6:54 AM Karl Wright <daddywri@gmail.com> wrote:
>>>> We solved the WSDL fetching through HTTPS, or thought we had, by restructuring
the code according to a number of articles we found.  This was supposedly tested and worked
in one installation.  Nobody has ever reported issues with the wsdls being fetched however;
I worry that you may have a different version of OpenText that is incompatible with the one
we developed against.  That's the problem with this kind of architecture; unless the wsdls
are included in the jar there can be issues.  We tried to do that too but were unable to get
it to work.
>>>> 
>>>> Karl
>>>> 
>>>> 
>>>>> On Thu, Jan 16, 2020 at 5:34 AM Jörn Franke <jornfranke@gmail.com>
wrote:
>>>>> Ok i fixed the source to fetch WSDL from https (it is not perfect yet
as it does not use the truststore yet but this I can fix) - I will share later in a Jira.
>>>>> It is however now unable to locate the imported document /Authentication?xsd=2
relative to Authenticaton?wsdl#types1
>>>>> 
>>>>> I will look into this, but if someone has come cross it then please let
me know.
>>>>> 
>>>>>>> Am 16.01.2020 um 10:22 schrieb Jörn Franke <jornfranke@gmail.com>:
>>>>>>> 
>>>>>> 
>>>>>> Coming back to the original topic. I believe SSL was never fully
solved from what i read in the corresponding issue. Apparently, the fetching of the WSDL itself
through https was not possible. Do you remember still some insights beyond what is written
in the issue ?
>>>>>> 
>>>>>>>> Am 16.01.2020 um 00:37 schrieb Karl Wright <daddywri@gmail.com>:
>>>>>>>> 
>>>>>>> 
>>>>>>> Let me think about that option.
>>>>>>> 
>>>>>>> Karl
>>>>>>> 
>>>>>>> 
>>>>>>>> On Wed, Jan 15, 2020 at 5:38 PM Jörn Franke <jornfranke@gmail.com>
wrote:
>>>>>>>> We could make it configurable, e.g. in properties.xml. Here
people could set it to SSL, TLS, TLSv1.2 (to restrict it to TLS1.2 => some companies may
want that!). Is this a viable option? That would be also future proof. We can leave it by
default to SSL, but we should put in the example config files TLS by default (so new starters
do not get even the idea to use an outdated protocol) AND put a comment with recommendation
to use/enforce always newest protocols for security reasons. Of course, the choice is then
with the people using the software.
>>>>>>>> Could that be something sensible from your point of view?
>>>>>>>> 
>>>>>>>>> On Wed, Jan 15, 2020 at 11:14 PM Karl Wright <daddywri@gmail.com>
wrote:
>>>>>>>>> It's rather immaterial what browsers do here.  What's
important is what  *existing servers* support, since that is what we're connecting with.
>>>>>>>>> 
>>>>>>>>> I tend to agree that *most* people have probably upgraded
to web servers that support TLS.  But we can't guarantee it, nor can we assume that people
have upgraded to the most modern version of TLS exclusively.  In fact I think we can assume
they have *not*.  When the SSL issues were discovered a couple of years back, the standard
recommendation was simply to *disable* SSLv1 and SSLv2, not to upgrade to Java 11 or some
such.  We still support (and have people using!!) early forms of NTLM (v1 to be specific),
for instance.  We're not going to be able to wag the dog here.  Breaking changes of this kind
usually mean we go to a whole new major version of MCF.
>>>>>>>>> 
>>>>>>>>> However, if you can show that SSLContext.getSSLFactory("TLS")
produces a SSLSocketFactory that works with all versions of TLS and SSL that do not have known
security holes, I would support changing over to that.  If it turns out we need much more
specificity about the kind of SSLSocketFactory we produce, then we need a better solution
anyhow for handling multiple protocols in one socket factory.
>>>>>>>>> 
>>>>>>>>> Karl
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> On Wed, Jan 15, 2020 at 5:17 AM Jörn Franke <jornfranke@gmail.com>
wrote:
>>>>>>>>>> Hi Karl,
>>>>>>>>>> 
>>>>>>>>>> No it does not. I can look into that further, but
Current browsers stop supporting anything below TLSv1.2 in March 2020.
>>>>>>>>>> Then TLS exists since more than ten years. I expect
any server running nowadays will always have tls support.
>>>>>>>>>> SSL itself is not supported since some time now.
From a security perspective it should even break servers that run only SSL as they are inherently
insecure and also clients that only support SSL are adding to this.
>>>>>>>>>> However if you have an idea how this should be made
configurable then I can look into this. 
>>>>>>>>>> 
>>>>>>>>>> Best regards 
>>>>>>>>>> 
>>>>>>>>>>>> Am 15.01.2020 um 10:52 schrieb Karl Wright
<daddywri@gmail.com>:
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Hi,
>>>>>>>>>>> 
>>>>>>>>>>> Mcf currently requires jdk8.  Jdk11 is non trivial
to support because of the removal of many jdk classes connectors need.  It will be ported
at some point but not lightly.
>>>>>>>>>>> 
>>>>>>>>>>> Similarly, disabling SSL would certainly break
many installations upon upgrade  and we do not do that lightly.
>>>>>>>>>>> 
>>>>>>>>>>> The core methods that mcf supplies its connectors
should therefore be updated to support but not mandate tls.  The protocol specification one
gives to sslcontext is not a detailed one but rather a major version.  What I don't know is
whether"tlsv1" also allows for older protocols etc.
>>>>>>>>>>> 
>>>>>>>>>>> Karl
>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Jan 15, 2020, 1:19 AM Jörn Franke
<jornfranke@gmail.com> wrote:
>>>>>>>>>>>> Yes I am doing that but I will need to rebuild.
>>>>>>>>>>>> I don’t recommend TLSv1 - this is already
outphased and will lock out TLSv1.2. I try TLS only as it includes all TLS protocols (depends
on JDK).
>>>>>>>>>>>> 
>>>>>>>>>>>> SSL will not be supported by this (however
as I said there are other parts of the code where there is a getInstance(TLS). And some caveats:
On JDK6+7 TLS only means TLSv1 (and newer TLS Protocols are deactivated) on JDK8 it means
also that newer TLS protocols are enabled.
>>>>>>>>>>>> To be honest in my opinion - a SSL only one
is a significant security hole and given how old TLS support is JDK i would be surprised if
there is someone using such a server (most Organisations should switch to TLSv1.2 in any case
as all protocols below have been broken). 
>>>>>>>>>>>> While it works for all JDKs - probably JDK8
should be recommended as it seems to have all TLS protocols activated when using „TLS“.
Older JDKs seem to deactivate TLSv1.1 and TLSv1.2 when using TLS. I will write more about
this in the JIRA, once I verified that this solves the problem. 
>>>>>>>>>>>> Then TLSv1.3 is JDK11 only - I will investigate
what that implies.
>>>>>>>>>>>> Does ManifoldCf supports JDK11?
>>>>>>>>>>>> 
>>>>>>>>>>>>>> Am 15.01.2020 um 00:08 schrieb Karl
Wright <daddywri@gmail.com>:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I think you can just change the code
to read as follows when it creates the SSLContext:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> SSLContext ctx = SSLContext.getInstance("TLSv1");
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I don't know if TLS will downgrade to
SSL if that's all that's available.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Karl
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Tue, Jan 14, 2020 at 6:02 PM Jörn
Franke <jornfranke@gmail.com> wrote:
>>>>>>>>>>>>>> Yes it you do not change this setting
as what I suspect happens here. See my previous mail for details.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Am 14.01.2020 um 23:51 schrieb
Karl Wright <daddywri@gmail.com>:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> It looks looks TLS is actually
enabled in the SSLSocketFactory framework based on how you create the SSLSocketContext.  See:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> https://docs.oracle.com/cd/E19698-01/816-7609/security-83/index.html

>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Tue, Jan 14, 2020 at 5:48
PM Karl Wright <daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>> The design of ManifoldCF
deliberately manages keystores on a connection by connection basis, not globally.  If you
think the only way to implement TLS is via global keystore I very much doubt it.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I am on the road until late
tomorrow but somewhere along the line I can do some research into why TLS won't work as we
are currently doing it.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Tue, Jan 14, 2020
at 12:56 PM Jörn Franke <jornfranke@gmail.com> wrote:
>>>>>>>>>>>>>>>>> These are TLS only. So
maybe you have other servers where tls and ssl are possible and it downgrades to ssl.however,
this is speculation and I need to verify it. I have to rebuilt manifold for that. Probably
I have to reinstall everything as the keystorefactory is a dependency in the connector.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Am 14.01.2020
um 18:34 schrieb Karl Wright <daddywri@gmail.com>:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> If you can recommend
changes to support TLS, that would be great.  The basic infrastructure should still work;
it is just a custom keystone and associated SSLSocketFactory, which I think also is used for
TLS connections, unless I am missing something.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Tue, Jan 14,
2020, 9:38 AM Jörn Franke <jornfranke@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>> Yes this works
fine. I believe the error comes from the fact that TLS connections are not supported. 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Am 14.01.2020
um 15:31 schrieb Michael Cizmar <michael.cizmar@mcplusa.com>:
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> If you want
to test the url and the ssl, I would recommend attempting using SSLPoke to confirm that they
keystore is setup properly:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> https://github.com/MichalHecko/SSLPoke
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Michael
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> From: Karl
Wright <daddywri@gmail.com>
>>>>>>>>>>>>>>>>>>>> Reply-To:
"user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
>>>>>>>>>>>>>>>>>>>> Date: Tuesday,
January 14, 2020 at 7:21 AM
>>>>>>>>>>>>>>>>>>>> To: "user@manifoldcf.apache.org"
<user@manifoldcf.apache.org>
>>>>>>>>>>>>>>>>>>>> Subject:
Re: CSWS Connector : ServiceConstructionException: Failed to create service
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Hmm, others
have succeeded setting up SSL connections with the current code.  Hoping they chime in here.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Tue, Jan
14, 2020, 8:19 AM Jörn Franke <jornfranke@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> It seems
that it has indeed a certificate issue as it cannot find a valid certification path to the
target. The thing is: I added those certificates in the UI should it should not happen.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Am 10.01.2020
um 20:51 schrieb Jörn Franke <jornfranke@gmail.com>:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 2.15 ...
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I will try
on the weekend to see if I can get some logs out of it. 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Am 10.01.2020
um 19:02 schrieb Karl Wright <daddywri@gmail.com>:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Can I ask
what version of MCF you are using?  There were issues with SSL in the first release of the
csws connector if I recall correctly, that were fixed for the second release.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Fri, Jan
10, 2020 at 11:42 AM Jörn Franke <jornfranke@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I added root,
intermediate and server certificate (in base64 cer, it seems to be recognized by manifoldcf),
but I still get the same message. I will try to get somehow the full stacktrace 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Am 10.01.2020
um 17:21 schrieb Karl Wright <daddywri@gmail.com>:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> If you are
using SSL you need to have the proper certificate saved in the connection's keystore.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Fri, Jan
10, 2020 at 11:20 AM Jörn Franke <jornfranke@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> It is actually
a server using configuration of the command - driven multi-process model (but the agents executed
as a service and the war on a tomcat executed as a service) under Linux.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I thought
as well that it cannot reach the webservices, the question is why. On the same server I can
reach the webservices and fetch the WSDL without issues.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Maybe sth
related to ssl ?
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Am 10.01.2020
um 14:59 schrieb Karl Wright <daddywri@gmail.com>:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> How are you
running manifoldcf?  Single process example, or a custom setup of some kind?
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> This exception
is a "catch all" exception generated far below anything in ManifoldCF, but usually means it
cannot download the WSDLs from the service.  Getting the full exception dumped in the log
requires a "hack" to the check() method of the connector, but I'm pretty sure that's what's
happening anyway.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>  
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> On Fri, Jan
10, 2020 at 8:50 AM Jörn Franke <jornfranke@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> I tried to
use the CSWS connector, but already for the Authority connection I receive a org.apache.cxf.service.factory.ServiceConstructionException:
Failed to create service.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Unfortunately
I don’t see more details , also not in the log (debug is activated). I try to get a little
bit more output by modifying the connector, but maybe someone has already an idea why this
can happen?
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Are there
some special instructions to use it? The pointers to the webservices are correct, I tested
via Curl and SOAPUI.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Thank you.
>>>>>>>>>>>>>>>>>>>> Best regards

Mime
View raw message