james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Charles <e...@apache.org>
Subject Re: What's in a store?
Date Mon, 20 Sep 2010 16:14:42 GMT
  Hi Norman,

So moving and simplifying some Store hierarchy makes sense.

Now, the important question is the alignment, or not, of the Store with 
Mailbox/MessageManager
(I talked about MailboxManager only, but I meant the Mailbox/MessageManager)

With such alignment:
1. A user could to define in spoolmanager.xml different store for 
different usage/priority/sla. For example
- locally delivered mails to jpa://mailbox/
- spam mails to jpa://mailbox/Spam (the spam folder of the user mailbox)
- relay denied mails to file://var/mail/relay-denied/
- error mails to jcr://error/
2. Current 2.3 users could simply leverage their mailstore and access 
them via POP3/IMAP without any change.

If we don't align/adapt:
1. Locally delivered mails are delivered in the unique store defined in 
spring-beans.xml .Oher mails have no access to existing and future 
mailbox-store impl.
2. James 2.3 users will have to manually migrate their 2.3 store, that 
means, pump it and inject it in jpa/jcr/....

It's true that the current MessageManager.appendMessage is not suited 
for mail delivery (it needs a mailboxSession,...).
But we could always add some methods such as 
MessageManager.appendRawMessage(Mail,...) (would not deal with Mail 
attributes,...) and let the mailbox-store implement them. The 
MailRepository.store(Mail) would simply delegate to that 
MessageManager.appendRawMessage(Mail,...)

Tks,

Eric


On 20/09/2010 14:50, Norman Maurer wrote:
> Hi Eric,
>
> comments inside.
>
> 2010/9/20 Eric Charles<eric@apache.org>:
>>   Hi,
>>
>> It's always possible to inject the mailboxmanager in mailets
>> (ToRepository),... but we would  really gain to unify/simplify the 2.3 mail
>> storage with the 3.0 one.
>>
>> In 2.3, we have such class hierarchy:
>>
>> Repository
>> +- AbstractFileRepository
>>     +- File_Persistent_Object_Repository
>>     +- File_Persistent_Stream_Repository
>>
>> MailRepository
>> +- AbstractMailRepository
>>     +- FileMailRepository
>>     +- JCRMailRepository
>>     +- JDBCMailRepository
>> +- MBoxMailRepository
>> +- SpoolRepository
>>     +- InMemorySpoolRepository
>>     +- MailStoreSpoolRepository
>>
>> Store
>> +- AbstractMailStore
>>     +- SpringMailStore
>>
>> In short, the SpringMailStore is responsible to register the
>> MailRepositories and to select them.
> Thats correct but I would more call it a hack to let it Spring load it
>   ;) The whole Store interface is just a left-over from the old days
> were we used Avalon/Excalibur for DI and component loading. I would be
> more in favor to replace the Store interface with something which is
> not such a mess..
>
>> In Mailbox projects introduced for 3.0, we have:
>>
>> MailboxManager
>> +- StoreMailboxManager
>>     +- InMemoryMailboxManager
>>     +- JCRMailboxManager
>>     +- JPAMailboxManager
>>     +- MaildirMailboxManager
>>
>> With new 3.0 implementation, we only can deliver in one-and-only-one store
>> (jpa, jcr,...).
>> We should overcome this limitation.
> Thats the question.. Do we really want to overcome this ? I'm not sure
> if the MailboxManager should replace the MailRepository for such cases
> as storing mails in
> the error processor etc. With the MailRepository we are able to store
> the whole Mail including Mail Attributes etc, thats not the case with
> MailboxManager. MailboxManager was not designed for that purpose. Its
> just a storage for Mailboxes nothing more, nothing less ;) IMHO it
> would make sense to "not merge" this twos and so only allow to
> configure one MailboxManager implementation..
>
>
>> Way to go further...:
>>
>> We should also discuss on the future way to evolve. There are different
>> options/steps to go further with that unification/simplification:
>>
>> A/ Should we consider the MaiboxManager as just another MailStore, so
>> developing a MailboxManagerMailStoreAdapter class and add the needed entries
>> to the mailstore.xml, so the existing SpringMailStore mecanic is invoked.
>>
>> B/ Or simplifying and removing the Store and MailRepository hierarchy.
>>
>> C/ Any other options... ?
>>
> See above..
>
>> This could be also thought in parallel with a discussion on UsersRepository
>> hierarchy, so we have coherent way of working within James.
>> I we go for option B/, it would be logical to also simplify the
>> UsersRepository hierarchy.
>> (Think also the DomainList has no Repository notion).
>>
> I think its a bit different. We should just Remove the UsersStore and
> VirtualUserTableStore. There is no need to support more the one
> implementation at one time
> here. This just makes things more complex.
>
>> I think we should also extract the 2.3 classes/interfaces, and place them in
>> the correct mailbox projects (also create new mailbox projects for example
>> apache-james-mailbox-file that will contain FileMailRepository,..).
>>
> +1
>
>> WDYT?
>>
>> Eric
>>
>>
>> On 6/09/2010 08:24, Eric Charles wrote:
>>>   OK, we've done a picture of the current situation.
>>>
>>> I would like to pick an simple example as strong requirement to unify the
>>> store/repository (whatever then name is) API across all projects.
>>> In spoolmanager.xml, we define respositoryPath such as db://maildb/spam.
>>> These repositoryPath are used in mailets,...
>>> Let's say I want to develop a mailet that under certain conditions stores
>>> the mail in a folder (let's say "spam" folder) of the user.
>>> If the mailbox is for example JCR, I have no standard way to define the
>>> folder path in the james-server "repository" syntax. Or I must invent one...
>>> (jcr://...? or a more generic one mailbox:///? because I don't want to
>>> change spoolmanager.xml if spring-beans.xml changes)
>>>
>>> If both are linked, we have clear conventions to implement.
>>> And we can always release any project when we want.
>>>
>>> This is why I think we should link in a way or another the james-server
>>> repository and the imap store. Not only on syntax extension, but also on
>>> implementing classes.
>>>
>>> Tks,
>>>
>>> Eric
>>>
>>> On 6/09/2010 07:49, Norman Maurer wrote:
>>>> Comments inline..
>>>>
>>>>
>>>> 2010/9/6 Eric Charles<eric@apache.org>:
>>>>>   Hi Norman,
>>>>>
>>>>> pop3server has store in its pom, but does not really need it. I will
>>>>> carefully test before commit the changes.
>>>> Should be safe to remove it then..
>>>>
>>>>> JPAUser is accessed in JPAUsersRepository (instanciated via the
>>>>> LocalUsers
>>>>> repository) while JPAMailbox is managed in JPAMailboMapper is accessed
>>>>> via
>>>>> JPAMailboxMapper. JPAUsersRepository and JPAMailboxMapper are created
in
>>>>> completely different ways.
>>>> Ok, I think the "mapper stuff" is just in imap to make implementing
>>>> different Mailbox stores easy. So its ok to have it work different.
>>>>
>>>>> I see that james-server has a whole infrastructure around "repository".
>>>>> Imap
>>>>> has not that infrastructure and rely on implementing different stores
in
>>>>> different projects. The questions are:
>>>>> 1.- must imap go to server repository infra?
>>>> Not sure..
>>>>
>>>>> 2.- must server repository infra be simplified and merged with the imap
>>>>> approach?
>>>> -1 No
>>>>
>>>>> 3.- must we leave that separated?
>>>> Have them seperate allow to have a independend release cycle. Well I
>>>> know imap had no release yet but I hope to get it done this week.
>>>>
>>>>> btw, even in james-server, all persistence does not go the same way:
>>>>> domainlist does not work completely with repository.
>>>>> If you take the JDBCDomainList, it needs a repositoryPath which is
>>>>> db://maildb/domain (I don't see directly the relation with the
>>>>> persistence.xml for the mailbox, and however, it stores in the same
>>>>> database...). However, if you take the JPADomainList, it does not need
>>>>> any
>>>>> "repository", because it's using the persistence.xml.
>>>>> I mean, the repository principle has some exceptions in james-server.
>>>>>
>>>> The RepositoryPath stuff is from the good old days. With this concept
>>>> its possible to use more then one db. Like db://maildb/domain (db =
>>>> maildb, table = domain) and db://maildb2/domain (db= maildb2, table =
>>>> domain). The whole concept is a "left-over" from using excalibur and
>>>> avalon. To be honest I don't care if we remove it at all, one db
>>>> should be enough...
>>>>
>>>>> Tks,
>>>>>
>>>>> Eric
>>>>>
>>>>>
>>>>> On 5/09/2010 20:11, Norman Maurer wrote:
>>>>>> Hi Eric,
>>>>>>
>>>>>> comments inside..
>>>>>>
>>>>>> 2010/9/5 Eric Charles<eric@apache.org>:
>>>>>>>   Hi All,
>>>>>>>
>>>>>>> The store project has the mappers and imports some classes of
mailbox
>>>>>>> (mailboxmanager,...), so 'store depends on mailbox'.
>>>>>>>
>>>>>>> To access some mailbox/subscription, you need to use the maibox
>>>>>>> project
>>>>>>> (example: imap-processor depends on mailbox, and imapProcessor
bean is
>>>>>>> injected with mailboxmanager/subscriptionManager). So far, so
good.
>>>>>>>
>>>>>>> mailbox is the interface to the mailboxes,...": I had a quick
look at
>>>>>>> the
>>>>>>> pop3server dependencies and saw that it was depending on store.
I
>>>>>>> didn't
>>>>>>> find that logical, removed it, and it was still compiling. Logical...
>>>>>>> Thinking in "what depends on what" helps much.
>>>>>> pop3server should not depend on store and only depend on mailbox.
If
>>>>>> its not the case its a bug..
>>>>>>
>>>>>>
>>>>>>> However, the other '"domain classes" (user, virutalusertable,
domain
>>>>>>> in
>>>>>>> the
>>>>>>> sense 'domain.tld') are accessed in a all different way. This
comes
>>>>>>> probably
>>>>>>> from the history (before using imap projects, server used its
own
>>>>>>> storage
>>>>>>> system which has been kept). In spring-beans.xml, you've got:
>>>>>>> - users-store: org.apache.james.container.spring.SpringUsersStore
>>>>>>> class
>>>>>>> being injected via @Resource
>>>>>>> - virtualusertable-store :
>>>>>>> org.apache.james.container.spring.SpringVirtualUserTableStore
being
>>>>>>> injected
>>>>>>> via @Resource
>>>>>>> - domainlist: XML/JDBC/JPADomainList being injected via @Resource
>>>>>>>
>>>>>>> I see there 3 differents access to the persisted domain classes
>>>>>>> (Mailbox,
>>>>>>> Message, Subscription, Domain, User, VirtualUserTable):
>>>>>>>
>>>>>>>   1. Injection of mailbox project (MailboxManager, MessageManager,
>>>>>>>      SubscriptionManger,...) beans
>>>>>>>   2. Injection of spring-common project (SpringUsersStore,
>>>>>>>      SpringVirtualUserTableStore) beans
>>>>>>>   3. Injection of core-function project (XML/JDBC/JPADomainList)
beans
>>>>>>>
>>>>>>> If you persist everything in a database, accesses for JPAUser
is done
>>>>>>> completely differently compared to the JPAMailbox for example.
>>>>>>>
>>>>>> Could you explain whats the dfference ?
>>>>>>
>>>>>>> Coming to the point, should we:
>>>>>>>
>>>>>>>   1. Continue to live with heterogeneous access the persistence
layer?
>>>>>>>   2.   Have uniform access to persistence for all services and
all
>>>>>>>      domains? In this case, a store project would be a good name,
but
>>>>>>>   is
>>>>>>> already taken... we could always rename the existing one...
>>>>>>>
>>>>>>> WDYT?
>>>>>>>
>>>>>>> Eric
>>>>>>>
>>>>>>> PS: I didn't even talked of the AbstractFileRepository
>>>>>>>
>>>>>>>
>>>>>> The whole *Store stuff is just a pita (james-server).. The idea with
>>>>>> its was that you can configure for example different UserRepositories
>>>>>> and just lookup the right one via the UsersStore. So its really
>>>>>> flexible. However the loading of the classes + injecting the right
>>>>>> dependencies is really complicated with this. I'm open to new ideas
to
>>>>>> make it more easy :)
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>> Bye,
>>>> Norman
>>>>
>>>> ---------------------------------------------------------------------
> Bye,
> Norman
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message