manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Problem crawling windows share
Date Wed, 07 Dec 2011 11:41:38 GMT
The next step is to obtain two packet captures using WireShark.  These
should both be done from the machine that is not joined to the domain.
 The first capture should be the interaction that takes place when you
try to go to the foreign directory, and you supply the credentials,
and then you list the folders.  The second capture will require you to
do the same using JCIFS.  For that capture, you will need jcifs.jar
and a simple example class that provides jcifs with credentials and
tries to list the directory.  A simple class should be enough to do
this, including code such as:

import jcifs.smb.*;

NtlmPasswordAuthentication pa = new
NtlmPasswordAuthentication(domain,username,password);
SmbFile smbconnection = new SmbFile("smb://" + server + "/" + share,pa);
SmbFile[] files = smbconnection.listFiles();

for (int i = 0 ; i < files.length ; i++)
{
	System.out.println(file.toString());
}

I suggest you put together your test class first and make sure it
works, and then go after the captures.

Let me know how it goes.

Karl

On Wed, Dec 7, 2011 at 6:10 AM, Swapna Vuppala
<swapna.kollipara@gmail.com> wrote:
> Hi Karl,
>
> Thanks for the detailed reply.
>
> As per your suggestion, I tried connecting to the problematic server via
> windows using a machine that is not in the domain. On supplying the
> credentials via windows, I could connect to the server, although it did not
> list any folders. But when I tried accessing the folder of my interest via
> \\server\folder$, I could see all the sub folders. All this is when I
> connected via windows from a machine that's not joined to the domain.
>
> The behavior is same when I connect to the server via windows from a machine
> that's joined to the domain.
>
> So I assume this means that through both Kerberos and NTLM, I see the same
> behavior. Whereas ManifoldCF throws an exception when trying to define the
> repository connection.
>
> Please advise me on what I should be doing next to resolve this.
>
> Thanks and Regards,
> Swapna.
>
>
>
>
> On Tue, Dec 6, 2011 at 12:55 PM, Karl Wright <daddywri@gmail.com> wrote:
>>
>> About your capture - Michael Allen says the following:
>>
>> "Actually this has nothing to do with DFS. JCIFS does not get to the
>> point where it does DFS anything. The capture shows a vanilla
>> STATUS_LOGON_FAILURE when GLOBAL\swapna.vuppala tries to auth with
>> l-carx01.global.arup.com. So the possible causes for this are 1) the
>> account name is not valid 2) the supplied password is incorrect 3)
>> some security policy is deliberately blocking that user or particular
>> type of auth or 4) some server configuration is incompatible with
>> JCIFS. I only mention this last option because I noticed the target
>> server has security signatures disabled. That's strange. If they're
>> messing around with things like that, who knows what their clients are
>> expected to do.
>>
>> Try a Windows client that uses NTLM instead of Kerberos. Meaning try a
>> machine that is not joined to the domain so that when you try to
>> access the target it asks you for credentials at which point you can
>> test with GLOBAL\swapna.vuppala. Then it will use NTLM and you can
>> actually compare captures. If the operator doesn't have a laptop or
>> something not joined to the domain, it might be sufficient to log into
>> a workstation using machine credentials and not domain credentials.
>>
>> Also when testing JCIFS you should use a simple stand-alone program
>> like examples/ListFiles.java."
>>
>> In other words:
>> (a) Since JCIFS does not use Kerberos for authentication, you need to
>> try to log into the recalcitrant server via Windows without using
>> Kerberos to be able to do a side-by-side comparison.  Michael has some
>> ways of doing that, above.
>> (b) You may find that it doesn't work, in which case JCIFS is not
>> going to work either.
>> (c) If it *does* work, then try to generate your side-by-side
>> comparisons using a simpler example rather than ManifoldCF en toto;
>> you can see how at jcifs.samba.org, or I can help you further.
>>
>> He also mentions that there is some bizarreness on the response that
>> indicates that the server is configured in a way that he's never seen
>> before.  And believe me, Michael has seen a *lot* of strange
>> configurations...
>>
>> Hope this helps.
>> Karl
>>
>> On Mon, Nov 28, 2011 at 4:12 AM, Karl Wright <daddywri@gmail.com> wrote:
>> > That should read "properties.xml", not "properties.ini".  It looks
>> > like this page needs updating.
>> >
>> > The debug property in the XML form is:
>> >
>> > <property name="org.apache.manifoldcf.connectors" value="DEBUG"/>
>> >
>> > I don't think it will provide you with any additional information that
>> > is useful for debugging your authentication issue, however, if that is
>> > why you are looking at it.  There may be some jcifs.jar debugging
>> > switches that might be of more help, but in the end I suspect you will
>> > need a packet capture of both a successful connection (via Windows)
>> > and an unsuccessful one (via MCF).  The guy you will need to talk with
>> > after that is the jcifs author Michael Allen; I can give you his email
>> > address if you get that far.
>> >
>> > Karl
>> >
>> >
>> > On Mon, Nov 28, 2011 at 1:30 AM, Swapna Vuppala
>> > <swapna.kollipara@gmail.com> wrote:
>> >> Hi Karl,
>> >>
>> >> I was planning to debug jCIFS repository connection using WireShark and
>> >> I
>> >> came across this
>> >> https://cwiki.apache.org/CONNECTORS/debugging-connections.html
>> >> Here, I see something as add "org.apache.manifoldcf.connectors=DEBUG"
>> >> to the
>> >> properties.ini file. Is it the properties.xml file that is being
>> >> referred
>> >> here ? If not, where do I find properties.ini file ?
>> >>
>> >> Thanks and Regards,
>> >> Swapna.
>> >>
>> >> On Thu, Nov 17, 2011 at 1:31 PM, Karl Wright <daddywri@gmail.com>
>> >> wrote:
>> >>>
>> >>> See http://jcifs.samba.org/src/docs/api/overview-summary.html#scp.
>> >>> The properties jcifs.smb.lmCompatibility and
>> >>> jcifs.smb.client.useExtendedSecurity are the ones you may want to
>> >>> change.  These two properties go together so certain combinations make
>> >>> sense and others don't, so there's really only combinations you need
>> >>> but I'll need to look at what they are and get back to you later
>> >>> today.
>> >>>
>> >>> As far as setting the switches are concerned, if you are using the
>> >>> Quick Start you do this trivially by:
>> >>>
>> >>> <java> -Dxxx -Dyyy -jar start.jar
>> >>>
>> >>> If you are using the multi-process configuration, that is what the
>> >>> "defines" directory is for; you only need to create files in that
>> >>> directory with the names "jcifs.smb.lmCompatibility" and
>> >>> "jcifs.smb.client.useExtendedSecurity" containing the values you want
>> >>> to set.
>> >>>
>> >>> Karl
>> >>>
>> >>>
>> >>> On Thu, Nov 17, 2011 at 1:11 AM, Swapna Vuppala
>> >>> <swapna.kollipara@gmail.com> wrote:
>> >>> > Hi Karl,
>> >>> >
>> >>> > Am able to access the folders on the problem server through windows
>> >>> > explorer, (\\server3\Folder1). I tried couple of things with the
>> >>> > credentials
>> >>> > form, changing username, domain etc.. but I keep getting the same
>> >>> > error
>> >>> > "Couldn't connect to server: Logon failure: unknown user name or
bad
>> >>> > password"
>> >>> >
>> >>> > Can you tell me more about the -D switch you were talking of ?
>> >>> >
>> >>> > Thanks and Regards,
>> >>> > Swapna.
>> >>> >
>> >>> > On Tue, Nov 15, 2011 at 12:40 PM, Karl Wright <daddywri@gmail.com>
>> >>> > wrote:
>> >>> >>
>> >>> >> Glad you chased it down this far.
>> >>> >>
>> >>> >> First thing to try is whether you can get into the problem
server
>> >>> >> using Windows Explorer.  Obviously ManifoldCF is not going
to be
>> >>> >> able
>> >>> >> to do it if Windows can't.  If you *can* get in, then just
playing
>> >>> >> with the form of the credentials in the MCF connection might
do the
>> >>> >> trick.  Some Windows or net appliance servers are picky about
this.
>> >>> >> Try various things like leaving the domain blank and specifying
the
>> >>> >> user as "abc@domain.com", for instance. There's also a different
>> >>> >> NTLM
>> >>> >> mode you can operation jcifs in that some servers may be configured
>> >>> >> to
>> >>> >> require; this would need you to set a -D switch on the command
line
>> >>> >> to
>> >>> >> enable.
>> >>> >>
>> >>> >> Karl
>> >>> >>
>> >>> >> On Tue, Nov 15, 2011 at 12:10 AM, Swapna Vuppala
>> >>> >> <swapna.kollipara@gmail.com> wrote:
>> >>> >> > Hi Karl,
>> >>> >> >
>> >>> >> > Thanks for the input. It looks like my problem is related
to the
>> >>> >> > second
>> >>> >> > one
>> >>> >> > that you specified. One of the directories in the path
am trying
>> >>> >> > to
>> >>> >> > index is
>> >>> >> > actually redirecting to a different server. And when I
specify
>> >>> >> > this
>> >>> >> > new
>> >>> >> > server in defining the repository connection, with my
>> >>> >> > credentials,
>> >>> >> > the
>> >>> >> > connection fails with the message:  "Couldn't connect
to server:
>> >>> >> > Logon
>> >>> >> > failure: unknown user name or bad password"
>> >>> >> >
>> >>> >> > I'll look into why am not able to connect to this server.
>> >>> >> >
>> >>> >> > Thanks and Regards,
>> >>> >> > Swapna.
>> >>> >> >
>> >>> >> > On Mon, Nov 14, 2011 at 4:56 PM, Karl Wright <daddywri@gmail.com>
>> >>> >> > wrote:
>> >>> >> >>
>> >>> >> >> There's two kinds of problem you might be having.
 The first is
>> >>> >> >> intermittent, and the second is not intermittent but
would have
>> >>> >> >> something to do with specific directories.
>> >>> >> >>
>> >>> >> >> Intermittent problems might include a domain controller
that is
>> >>> >> >> not
>> >>> >> >> always accessible.  In such cases, the crawl will
proceed but
>> >>> >> >> will
>> >>> >> >> tend to fail unpredictably.  On the other hand, if
you have a
>> >>> >> >> directory that is handled by a DFS redirection, it
is possible
>> >>> >> >> that
>> >>> >> >> the redirection is indicating a new server (lets call
it
>> >>> >> >> server3)
>> >>> >> >> which may not like the precise form of your login
credentials.
>> >>> >> >>  Can
>> >>> >> >> you determine which scenario you are seeing?
>> >>> >> >>
>> >>> >> >> Karl
>> >>> >> >>
>> >>> >> >> On Mon, Nov 14, 2011 at 3:11 AM, Swapna Vuppala
>> >>> >> >> <swapna.kollipara@gmail.com> wrote:
>> >>> >> >> > Hi,
>> >>> >> >> >
>> >>> >> >> > I have been using windows share repository connection
to crawl
>> >>> >> >> > and
>> >>> >> >> > get
>> >>> >> >> > data
>> >>> >> >> > from a particular server (server 1). Its working
perfectly
>> >>> >> >> > fine.
>> >>> >> >> > However, am
>> >>> >> >> > having trouble when I try with data from another
server
>> >>> >> >> > (server
>> >>> >> >> > 2).
>> >>> >> >> >
>> >>> >> >> > When I define a repository connection of type
windows share
>> >>> >> >> > and
>> >>> >> >> > specify
>> >>> >> >> > the
>> >>> >> >> > server name (server 2) with my credentials, the
connection
>> >>> >> >> > status
>> >>> >> >> > shows
>> >>> >> >> > "Connection working". But when I run a job to
use this
>> >>> >> >> > repository
>> >>> >> >> > connection
>> >>> >> >> > and index data from a location on this server
2, I keep
>> >>> >> >> > getting
>> >>> >> >> > the
>> >>> >> >> > exception below:
>> >>> >> >> >
>> >>> >> >> > JCIFS: Possibly transient exception detected
on attempt 3
>> >>> >> >> > while
>> >>> >> >> > checking
>> >>> >> >> > if
>> >>> >> >> > file exists: Logon failure: unknown user name
or bad password.
>> >>> >> >> > jcifs.smb.SmbAuthException: Logon failure: unknown
user name
>> >>> >> >> > or
>> >>> >> >> > bad
>> >>> >> >> > password.
>> >>> >> >> >     at
>> >>> >> >> > jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:544)
>> >>> >> >> >     at jcifs.smb.SmbTransport.send(SmbTransport.java:661)
>> >>> >> >> >     at jcifs.smb.SmbSession.sessionSetup(SmbSession.java:390)
>> >>> >> >> >     at jcifs.smb.SmbSession.send(SmbSession.java:218)
>> >>> >> >> >     at jcifs.smb.SmbTree.treeConnect(SmbTree.java:176)
>> >>> >> >> >     at jcifs.smb.SmbFile.doConnect(SmbFile.java:911)
>> >>> >> >> >     at jcifs.smb.SmbFile.connect(SmbFile.java:954)
>> >>> >> >> >     at jcifs.smb.SmbFile.connect0(SmbFile.java:880)
>> >>> >> >> >     at jcifs.smb.SmbFile.queryPath(SmbFile.java:1335)
>> >>> >> >> >     at jcifs.smb.SmbFile.exists(SmbFile.java:1417)
>> >>> >> >> >     at
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileExists(SharedDriveConnector.java:2064)
>> >>> >> >> >     at
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getDocumentVersions(SharedDriveConnector.java:521)
>> >>> >> >> >     at
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:318)
>> >>> >> >> >
>> >>> >> >> > Am able to access this location from windows
explorer. What
>> >>> >> >> > else
>> >>> >> >> > should
>> >>> >> >> > I be
>> >>> >> >> > checking or what could be the reasons/factors
causing this to
>> >>> >> >> > fail
>> >>> >> >> > ?
>> >>> >> >> >
>> >>> >> >> > Thanks and Regards,
>> >>> >> >> > Swapna.
>> >>> >> >> >
>> >>> >> >
>> >>> >> >
>> >>> >
>> >>> >
>> >>
>> >>
>
>

Mime
View raw message