james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harmeet Bedi" <harm...@kodemuse.com>
Subject Re: Re: [Ldapd-devel] RE: Using ldapd with James
Date Fri, 02 May 2003 01:23:07 GMT
Message from "Alex Karasulu" <aok123@bellsouth.net> , key developer ldapd
project regd JNDI as backend API:

----- Original Message -----
From: <aok123@bellsouth.net>
To: <ldapd-devel@lists.sourceforge.net>;
<ldapd-devel@lists.sourceforge.net>; <james-dev@apache.org>;
<harmeet@kodemuse.com>
Sent: Thursday, May 01, 2003 11:08 AM
Subject: Re: Re: [Ldapd-devel] RE: Using ldapd with James


> Harmeet,
>
> Let me know if this gets to the James dev list.  If not please forward it
for me.
>
> Ok first let me start by talking about the James backing store or data
repository aspects of the conversation.  The James server correct me if I'm
wrong has a smtp and nntp front end.  Meaning its a newtork protocol server.
To store email and news content as well as configuration info it must manage
backing stores for this info right?  The data comes in two forms: relational
and simple bulk content.  The relational data is info that needs to be
queried and the bulk content is the body of an email message or a news item.
>
> My point with slide was to use it for content storage only.  The
relatiional information would be stored, altered and accessed via ldap.
Whether or not the ldap server is local to James, remote or bundled into the
same process is irrelavent.  Both WebDAV and LDAP operations can be wrapped
by JNDI.  JNDI is generic enough to allow both very different backing store
types to be accessed in the same way.  The relational info in the LDAP
directory stores header into and users and groups etc so that it can be
queried.  The relational info then uses an ldap referral to point to the
body or content held in the WebDAV server.  LDAP referrals can use any URL
schema. So for example I can have a referral (pointer) to an email body on a
file server by using a URL like so file://var/mail/james/ASq345nawnfsd.txt.
>
> The beauty of JNDI lies in the fact that the provider can switch without
our needing to be aware of it.  So if we look up a set of email bodies using
an LDAP query (filter) they can be transparently accessed using JNDI however
the provider would be the file system provider.  Federation and provider
switching this would makes JNDI very flexible and an ideal candidate for a
common backing store.  This is what I meant by the James, Slide and LDAPd
synergy.
>
> If LDAPd was bundled within the same process space as James then the
server side JNDI provider can be used to directly access the JDBM based
backing store without having to go through the IP stack and the LDAP line
protocol.  The server side provider directly accesses backends.  This would
mean extremely fast access to relational data for James.  If later someone
would like to keep the LDAPd server separate in another teir that's not a
problem either since JNDI is used for both local and remote access.
>
> With respect to the code your looking at - ask any questions at all.
Perhaps we can begin compiling a FAQ for developers.  Also take a look at
the documentation on the home page it should help even though it is not
absolutely complete.
>
> Snacc4J is hairy from a licensing perspective that is why we're so keen on
isolating it into a berlib provider in the message framework.  Here's where
you can get it btw: http://ldapd.sourceforge.net/dist/Snacc4J_v2.3.tgz . I
don't think ibm is exposing it anymore.  Not a very good situation but we
can switch later to a2j or another asn.1 stub compiler and berlib provider.
I'll take a look at Bouncycastle to see if it can be made into a provider.
>
> Talk to you soon,
> -Alex
>
>
>
>
>
> >
> > From: "Harmeet Bedi" <harmeet@kodemuse.com>
> > Date: 2003/04/30 Wed PM 11:57:43 EDT
> > To: <ldapd-devel@lists.sourceforge.net>
> > Subject: Re: Re: [Ldapd-devel] RE: Using ldapd with James
> >
> > Alex, thanks for the explanation. I will certainly look more closely at
> > ldap-common.
> >
> > Checking out the code to get familiar with it. It looks very well done.
I
> > hope I will be able to contribute some thing useful.
> >
> >
> > A few things from your email that I did some research on:
> > BER library. I think mozilla LDAP Java sdk is very good and may be worth
> > investing in.
> > DER library. Bouncycastle has a nicely written open source library.
> > http://www.bouncycastle.org
> >
> > > I would love to have at a minimum Avalon Framework interfaces defined
for
> > C#.
> > There is one existing at least in protoype form. I think it is hosted by
> > Apache too. I cannot find the reference but Peter Donald could know.
> > One could also trivially implement a simple Avalon container in the
spirit
> > of tweety(simple example Avalon container) for C# to just run ldapd. I
would
> > be very nice to have C++ too. I have been bitten by openldap corruption
and
> > unreliability and would be nice if there was some alternative. Willing
to
> > work on it although my C++ is a bit rusty.
> > Regd. portability. I am most worried about snacc4j and what it may
involve.
> > I could not even find a way to download it off IBM. Would snacc4j cause
> > portability issues ?
> >
> > > Note that this is a client-side server-side neutral library unlike
JNDI.
> > ...
> > > There are some amazing things that can be done with James, Slide and
> > LDAPd.
> >
> > Agreed. I think slide and that entire model of hierarchical
representation
> > is very nice. A lot of work I do is based on ideas in Slide Conext/Store
API
> > as it existed about 2 years back. Not sure what is the latest there but
I
> > did find it a lot simpler and neutral compared to JNDI. Namespaces were
> > interesting to me and that was not there in JNDI, but not sure if it is
> > useful to ldapd. atm. these ideas and your mentioning it is encouraging
to
> > me. I not sure what the store interfaces used in ldapd are, but should
know
> > more as I read more (ldapd) code. :-)
> >
> > > In terms of James, perhaps the best means to standardize access to
data is
> > through JNDI.  Relational data can be stored within LDAP and
non-relational
> >
> > FYI: This was one of the strong candidate ideas for repository
interface,
> > but the consensus was to use a more mail oriented and Java specfic API,
> > specifically Java Mail. I did favor JNDI, but majority of James dev and
> > users seem to favor Javamail so that is where James is headed. I think
it
> > should be ok. A simple well known API would be good for James esp as
> > it(James) wants to be Java specific and as of now is mail/news server
> > oriented. If you strongly feel this should be revisited please send your
> > ideas to james-dev list.
> >
> > Harmeet
> > ----- Original Message -----
> > From: <aok123@bellsouth.net>
> > To: <ldapd-devel@lists.sourceforge.net>;
<ldapd-devel@lists.sourceforge.net>
> > Cc: "James Developers List" <james-dev@jakarta.apache.org>
> > Sent: Wednesday, April 30, 2003 11:15 AM
> > Subject: Re: Re: [Ldapd-devel] RE: Using ldapd with James
> >
> >
> > > Harmeet,
> > >
> > > LDAPd has a ldapd-common project which presently holds a LDAPv3
message
> > package with hand woven interfaces.  These interfaces expose a message
(req
> > or resp) as a bean that is the result of [de]marshalling BER encoded
LDAPv3
> > protocol data units.  The package name is ldapd.common.message.  This
was
> > the package we had discussed earlier in a previous post you had put on
the
> > ldapd-devel list.
> > >
> > > The message handling (on wire part of the protocol) using a specific
BER
> > Library provider or ASN.1 stub compiler is abstracted away using the
> > provider design pattern.  Within the ldapd.common.message package is the
> > 'spi' subpackage for service provider interfaces that must be
implemented by
> > a specific BER Library provider.  The registration and use of a provider
is
> > similar in concept to the way JNDI providers are used.  By having
providers
> > implement these SPIs the message handling machinery in the ldapd-common
CVS
> > project uses SPI interfaces to do the work remaining aloof to the
> > idiosyncracies of each BER Library implementation.
> > >
> > > To end users of this message library (the client and server
developers)
> > message handling occurs using the interfaces and classes within the
> > ldapd.common.message package.  Right now we have a snacc provider CVS
> > project for IBM's snacc4J compiler and runtime BER library.  We will add
> > more to take advantage of performance and features in other BER or DER
> > libraries.
> > >
> > > So to answer your first question Harmeet I think we do have the
protocol
> > handlers independant of connection handling or backends.  The message
APIs
> > are completely independant and on their own within as expected this
> > ldapd-common subproject.  It is here for these reasons and will
eventually
> > be used by both clients and servers.
> > >
> > > Note that this is a client-side server-side neutral library unlike
JNDI.
> > Furthermore it is dedicated to LDAPv3 specifically and so it represent
> > LDAPv3 message envelopes in a type safe fashion with explicit types.
This
> > makes for way cleaner code when dealing with the parts of LDAP on the
wire
> > (network protocol oriented aspects).
> > >
> > > Eventually I would love to write a solid ASN.1 stub compiler or use a
byte
> > code generation mechanism to generate beans to [de]marshall BER encoded
> > ASN.1 structures.  It would be nice to dynamically read a ASN.1 based
> > structure specification and generate code or classes on the fly to
> > communicate the structure.  I would also like to take advantage of some
NIO
> > classes and efficiencies within 1.4 for handling buffers.  I don't think
any
> > ASN.1 stub compilers and the BER libraries really do that now.
> > >
> > > Yeah I have thought about porting ldapd to C#.  It would be very easy
to
> > do actually.  Porting the message library would be even easier.  From
you
> > http://sourceforge.net/projects/ldapserversdk/ project it looks to me
like
> > we have a lot of common objectives.  Let's pull it together - nothing
says
> > that ldapd and its components need to be written in Java only.  However
I
> > chose Java because of two things: write once run anywhere and Avalon.  I
> > would love to have at a minimum Avalon Framework interfaces defined for
C#.
> > We can talk about this some more.
> > >
> > > LDAPd BTW has created a Backend API to allow for backend pluggability
> > using a server to backend contract.  Take a look at ldapd.server.backend
or
> > go here:
> >
http://ldapd.sourceforge.net/modules/ldapd-server/design/backend-module.html
> > .  There's still alot missing from the doc so I appologize. BTW I'm
thinking
> > about revamping this somewhat to make it a bit more coherent.  I would
love
> > to have you join us to help lead these efforts.
> > >
> > > In terms of James, perhaps the best means to standardize access to
data is
> > through JNDI.  Relational data can be stored within LDAP and
non-relational
> > content can be accessed via some WebDAV JNDI provider or the JNDI
provider
> > for the file system.  The JNDI LDAP provider can be used to index into
the
> > non-relational content using referrals that can be resolved by a
separate
> > JNDI provider.  The power of federation in JNDI can hide the backing
store
> > mechanism using a common API regardless of the nature of the data. Let
me
> > give you a simple example where a JNDI LDAP provider and JNDI based
WebDAV
> > provider are used in concert:
> > >
> > > LDAP contains the users, groups and other relational peices of data.
Mail
> > or news content can be stored on disk pointed to by an LDAP referral
using a
> > webdav (http really) url scheme.  Even the file system provider can be
used
> > via a file://some/path referral.  Using the server side provider in
ldapd
> > and the webdav provider James can manage both the relational data and
the
> > content.
> > >
> > > There are some amazing things that can be done with James, Slide and
> > LDAPd.
> > >
> > > Talk to you soon,
> > > -Alex
> > >
> > >
> > >
> > >
> > > >
> > > > From: "Harmeet Bedi" <harmeet@kodemuse.com>
> > > > Date: 2003/04/30 Wed AM 02:33:12 EDT
> > > > To: <ldapd-devel@lists.sourceforge.net>
> > > > CC: "James Developers List" <james-dev@jakarta.apache.org>
> > > > Subject: Re: [Ldapd-devel] RE: Using ldapd with James
> > > >
> > > > ----- Original Message -----
> > > > From: "Alex Karasulu" <aok123@bellsouth.net>
> > > >
> > > >
> > > > > What improvements would best help with the James integration
effort?
> > > >
> > > > One question to ldapd. Are you planning, or do you already have LDAP
> > > > Protocol handler libraries from the Connection oriented and backend
> > parts ?
> > > >
> > > > It would be very useful to just be able to pick a stable protocol
> > handler
> > > > and be able to do everything else around it. Kind of like idea
expressed
> > in:
> > > > http://sourceforge.net/projects/ldapserversdk/. It was something I
> > started
> > > > more than a year back using BER libraries instead of ASN but never
got
> > > > around to releasing any code. Advantages of using BER libraries was
> > lower
> > > > footprint, easier to profile, easy to port to other languages. Some
of
> > it is
> > > > still sitting on my disk and could share if there is sufficient
interest
> > for
> > > > me to make the effort.
> > > >
> > > > How this may be relevant to at least James ?
> > > > James has its' own connection handling, own backend repositories.
One of
> > the
> > > > things in James that is troublesome is that there are multiple
backend
> > APIs.
> > > > The underlying filesystem persistence format is the same but API are
> > > > different for SMTP and NNTP Severs. It would be good to be able to
> > plugin
> > > > James persistence layer and let it evolve without being coupled with
the
> > > > protocol stack.
> > > > There are 2 other projects(non James) that I am associated with that
> > have an
> > > > interest only in the protocol handler part.
> > > >
> > > >
> > > > Would ldapd group consider having just the protocol handler part as
a
> > > > separate subproject ?
> > > >
> > > > Would ldapd group consider making the ldap protocol handler easier
to
> > port
> > > > to other languages - C++,C# ? This is not relevant to James as James
is
> > Java
> > > > specific, but it would make ldapd even more usable.
> > > >
> > > >
> > > > Cheers to the ldapd folks for the excellent work.
> > > >
> > > > Harmeet



---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org


Mime
View raw message