manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Authority Connection works unpredictably
Date Wed, 23 Nov 2011 10:55:00 GMT
I've attached a patch to the ticket so even MCF 0.3 users should be
able to apply it.

Karl

On Wed, Nov 23, 2011 at 5:40 AM, Karl Wright <daddywri@gmail.com> wrote:
> To clarify, what I think may be happening is this.
>
> (1) The Java LDAP context is keeping a socket connection to the AD controller.
> (2) The AD controller must be configured to close connections forcibly
> after a certain period of time.
> (3) The LDAP context's reconnect() operation doesn't recover from a
> socket that was closed by the server.
> (4) The authority code won't release the LDAP context until 5 idle
> minutes go by.
>
> So basically, a connection winds up in a busted state and doesn't
> recover, if the server closes the socket out from under the ldap
> connection.
>
> It's easy to fix, so I've opened a ticket (CONNECTORS-291), and will
> commit code changes to trunk shortly.  What version of MCF are you
> using?
>
> Karl
>
> On Wed, Nov 23, 2011 at 5:23 AM, Karl Wright <daddywri@gmail.com> wrote:
>> Hi Swapna,
>>
>> There should be manifoldcf log output that contains the actual stack
>> trace of the exception.  That would be very helpful; I need the line
>> numbers.
>>
>> The code is quite simple, and indicates that the LDAP server is
>> refusing a connection:
>>
>>  protected void getSession()
>>    throws ManifoldCFException
>>  {
>>    if (ctx == null)
>>    {
>>      // Calculate the ldap url first
>>      String ldapURL = "ldap://" + domainControllerName + ":389";
>>
>>      Hashtable env = new Hashtable();
>>      env.put(Context.INITIAL_CONTEXT_FACTORY,"com.sun.jndi.ldap.LdapCtxFactory");
>>      env.put(Context.SECURITY_AUTHENTICATION,authentication);
>>      env.put(Context.SECURITY_PRINCIPAL,userName);
>>      env.put(Context.SECURITY_CREDENTIALS,password);
>>
>>      //connect to my domain controller
>>      env.put(Context.PROVIDER_URL,ldapURL);
>>
>>      //specify attributes to be returned in binary format
>>      env.put("java.naming.ldap.attributes.binary","tokenGroups objectSid");
>>
>>      // Now, try the connection...
>>      try
>>      {
>>        ctx = new InitialLdapContext(env,null);
>>      }
>>      catch (AuthenticationException e)
>>      {
>>        // This means we couldn't authenticate!
>>        throw new ManifoldCFException("Authentication problem
>> authenticating admin user '"+userName+"': "+e.getMessage(),e);
>>      }
>>      catch (CommunicationException e)
>>      {
>>        // This means we couldn't connect, most likely
>>        throw new ManifoldCFException("Couldn't communicate with domain
>> controller '"+domainControllerName+"': "+e.getMessage(),e);
>>      }
>>      catch (NamingException e)
>>      {
>>        throw new ManifoldCFException(e.getMessage(),e);
>>      }
>>    }
>>    else
>>    {
>>      // Attempt to reconnect.  I *hope* this is efficient and doesn't
>> do unnecessary work.
>>      try
>>      {
>>        ctx.reconnect(null);
>>      }
>>      catch (AuthenticationException e)
>>      {
>>        // This means we couldn't authenticate!
>>        throw new ManifoldCFException("Authentication problem
>> authenticating admin user '"+userName+"': "+e.getMessage(),e);
>>      }
>>      catch (CommunicationException e)
>>      {
>>        // This means we couldn't connect, most likely
>>        throw new ManifoldCFException("Couldn't communicate with domain
>> controller '"+domainControllerName+"': "+e.getMessage(),e);
>>      }
>>      catch (NamingException e)
>>      {
>>        throw new ManifoldCFException(e.getMessage(),e);
>>      }
>>    }
>>
>>    expiration = System.currentTimeMillis() + expirationInterval;
>>
>>    try
>>    {
>>      responseLifetime = Long.parseLong(this.cacheLifetime) * 60L * 1000L;
>>      LRUsize = Integer.parseInt(this.cacheLRUsize);
>>    }
>>    catch (NumberFormatException e)
>>    {
>>      throw new ManifoldCFException("Cache lifetime or Cache LRU size
>> must be an integer: "+e.getMessage(),e);
>>    }
>>
>>  }
>>
>>
>> Your problem description indicates that it is possible that the
>> ctx.reconnect() call is failing to reconnect, but a new connection
>> works OK on your setup.  A stack trace should tell me everything.
>>
>> Thanks,
>> Karl
>>
>>
>>
>> On Wed, Nov 23, 2011 at 12:58 AM, Swapna Vuppala
>> <swapna.kollipara@gmail.com> wrote:
>>> Hi Karl,
>>>
>>> Even after reducing the max connections to 3, the connection fails abruptly
>>> for me.
>>>
>>> Currently, the domain controller am using is mapped to only one IP address,
>>> and that responds on ping, and the max connections are 3. It was working
>>> yesterday and it fails suddenly throwing different exceptions like below:
>>>
>>> Threw exception: 'Couldn't communicate with domain controller 'globalad1':
>>> null'
>>> Threw exception: 'Couldn't communicate with domain controller
>>> 'globalad1.global.arup.com': null'
>>> Threw exception: 'globalad1.global.arup.com:389; socket closed'
>>>
>>> Sometimes, it works when I change the cache lifetime parameter. What others
>>> factors do you think that can cause this to fail ?
>>>
>>> Thanks and Regards,
>>> Swapna.
>>>
>>> On Tue, Nov 22, 2011 at 11:56 AM, Swapna Vuppala
>>> <swapna.kollipara@gmail.com> wrote:
>>>>
>>>> OK.. Thanks for the information
>>>>
>>>> On Mon, Nov 21, 2011 at 6:31 PM, Karl Wright <daddywri@gmail.com> wrote:
>>>>>
>>>>> The sAMAccountName and UserPrincipalName LDAP fields were used by
>>>>> different versions of Windows at different points in time.  Some
>>>>> backwards compatibility was maintained, however Microsoft has
>>>>> apparently decided to deprecate one of them (can't remember which),
>>>>> and thus you need support for both.
>>>>>
>>>>> Karl
>>>>>
>>>>> On Mon, Nov 21, 2011 at 6:39 AM, Swapna Vuppala
>>>>> <swapna.kollipara@gmail.com> wrote:
>>>>> > Hi Karl,
>>>>> >
>>>>> > Yes, my Active Directory authority connection is configured to talk
to
>>>>> > only
>>>>> > one IP address and that particular one is responding to ping always.
>>>>> >
>>>>> > Earlier, the max connections parameter was set to 10, now I reduced
it
>>>>> > to 3.
>>>>> > Its working as of now and I'll keep checking if its going to throw
an
>>>>> > exception. Thanks a lot for the inputs.
>>>>> >
>>>>> > Also, I was wondering what the difference was between 2 options
for
>>>>> > Login
>>>>> > name AD attribute, sAMAccountName and UserPrincipalName ?
>>>>> >
>>>>> > Thanks and Regards,
>>>>> > Swapna.
>>>>> >
>>>>> > On Mon, Nov 21, 2011 at 4:57 PM, Karl Wright <daddywri@gmail.com>
>>>>> > wrote:
>>>>> >>
>>>>> >> So let me get this straight - your Active Directory authority
>>>>> >> connection is configured to talk to only one IP address?  and
that IP
>>>>> >> address responds to ping even when you are receiving an error
back
>>>>> >> from the authority connection?
>>>>> >>
>>>>> >> Another possibility is that the DC can only accept a limited
number of
>>>>> >> connections at a time. What is the max connections parameter
for your
>>>>> >> authority connection?  Try reducing it to no more than 3-4
and see if
>>>>> >> that helps.
>>>>> >>
>>>>> >> Karl
>>>>> >>
>>>>> >>
>>>>> >> On Mon, Nov 21, 2011 at 5:34 AM, Swapna Vuppala
>>>>> >> <swapna.kollipara@gmail.com> wrote:
>>>>> >> > Hi Karl,
>>>>> >> >
>>>>> >> > I think I see many domain controllers for the domain am
using. But I
>>>>> >> > see
>>>>> >> > only one IP address mapped to the domain controller name
that am
>>>>> >> > using
>>>>> >> > in
>>>>> >> > the credentials form.
>>>>> >> >
>>>>> >> > As I told you, its working sometimes and throwing exception
>>>>> >> > sometimes.
>>>>> >> > But
>>>>> >> > ping works always fine on the  domain controller name
that am using,
>>>>> >> > from
>>>>> >> > which I assume that it is not unreachable.
>>>>> >> >
>>>>> >> > Can you tell me what else I should be checking or what
other factors
>>>>> >> > could
>>>>> >> > be causing this to fail ?
>>>>> >> >
>>>>> >> > Thanks and Regards,
>>>>> >> > Swapna.
>>>>> >> >
>>>>> >> > On Thu, Nov 17, 2011 at 1:18 PM, Karl Wright <daddywri@gmail.com>
>>>>> >> > wrote:
>>>>> >> >>
>>>>> >> >> Try doing nslookup on the domain controller.  In some
larger
>>>>> >> >> companies
>>>>> >> >> there are many domain controllers all with the same
name but
>>>>> >> >> different
>>>>> >> >> IP's.  These *should* all be in synch but it may be
the case that
>>>>> >> >> they
>>>>> >> >> are not - or some of them are unreachable or offline.
 This can
>>>>> >> >> also
>>>>> >> >> be the cause of intermittent authorization failures
during
>>>>> >> >> crawling.
>>>>> >> >>
>>>>> >> >> If that is the case you have the option of setting
the local
>>>>> >> >> machine's
>>>>> >> >> /etc/hosts file to point to a couple of domain controller
instances
>>>>> >> >> that are local and in good working order, rather than
rely on DNS
>>>>> >> >> to
>>>>> >> >> find one.
>>>>> >> >>
>>>>> >> >> Karl
>>>>> >> >>
>>>>> >> >> On Thu, Nov 17, 2011 at 1:32 AM, Swapna Vuppala
>>>>> >> >> <swapna.kollipara@gmail.com> wrote:
>>>>> >> >> > Hi,
>>>>> >> >> >
>>>>> >> >> > I seem to have some problem with Authority Connection.
When I
>>>>> >> >> > define
>>>>> >> >> > an
>>>>> >> >> > Authority Connection specifying all the parameters
like Domain
>>>>> >> >> > Controller,
>>>>> >> >> > username, password etc, the connection status
shows "Connection
>>>>> >> >> > Working"
>>>>> >> >> > and
>>>>> >> >> > everything works fine, crawling and sending docs
to solr, using
>>>>> >> >> > mcf-authority-service to get only those docs that
a user has got
>>>>> >> >> > permission
>>>>> >> >> > to see etc.
>>>>> >> >> >
>>>>> >> >> > But suddenly, the connection status for the Authority
Connection
>>>>> >> >> > throws
>>>>> >> >> > an
>>>>> >> >> > exception, and when I play around the credentials
form toggling
>>>>> >> >> > Login
>>>>> >> >> > name
>>>>> >> >> > AD attribute, or changing domain controller name,
or
>>>>> >> >> > authentication ,
>>>>> >> >> > or
>>>>> >> >> > sometimes even with the same settings that threw
an exception
>>>>> >> >> > earlier,
>>>>> >> >> > the
>>>>> >> >> > status shows "Connection working" again. I cannot
define when it
>>>>> >> >> > fails
>>>>> >> >> > and
>>>>> >> >> > when it works and for what settings it works.
>>>>> >> >> >
>>>>> >> >> > Can someone help me in understanding why this
is happening and
>>>>> >> >> > what
>>>>> >> >> > needs to
>>>>> >> >> > be done to make it work always ?
>>>>> >> >> >
>>>>> >> >> > Thanks and Regards,
>>>>> >> >> > Swapna.
>>>>> >> >> >
>>>>> >> >
>>>>> >> >
>>>>> >
>>>>> >
>>>>
>>>
>>>
>>
>

Mime
View raw message