james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Charles <e...@apache.org>
Subject Re: What's in a store?
Date Thu, 30 Sep 2010 09:23:27 GMT
Hi Norman,

After rethinking about that, I think you are right to keep the 2 stores 
mechanic separated.
The 2.3 should be used to store raw mails that is not really related to 
a mailbox.
The new mailbox/message manager is the mailbox and should not be 
impacted by something else.

To deliver spam mail in the spam folder of the user's mailbox, I will 
propose a custom mailet.

Tks,

Eric


On 20/09/2010 18:14, Eric Charles wrote:
>  Hi Norman,
>
> So moving and simplifying some Store hierarchy makes sense.
>
> Now, the important question is the alignment, or not, of the Store 
> with Mailbox/MessageManager
> (I talked about MailboxManager only, but I meant the 
> Mailbox/MessageManager)
>
> With such alignment:
> 1. A user could to define in spoolmanager.xml different store for 
> different usage/priority/sla. For example
> - locally delivered mails to jpa://mailbox/
> - spam mails to jpa://mailbox/Spam (the spam folder of the user mailbox)
> - relay denied mails to file://var/mail/relay-denied/
> - error mails to jcr://error/
> 2. Current 2.3 users could simply leverage their mailstore and access 
> them via POP3/IMAP without any change.
>
> If we don't align/adapt:
> 1. Locally delivered mails are delivered in the unique store defined 
> in spring-beans.xml .Oher mails have no access to existing and future 
> mailbox-store impl.
> 2. James 2.3 users will have to manually migrate their 2.3 store, that 
> means, pump it and inject it in jpa/jcr/....
>
> It's true that the current MessageManager.appendMessage is not suited 
> for mail delivery (it needs a mailboxSession,...).
> But we could always add some methods such as 
> MessageManager.appendRawMessage(Mail,...) (would not deal with Mail 
> attributes,...) and let the mailbox-store implement them. The 
> MailRepository.store(Mail) would simply delegate to that 
> MessageManager.appendRawMessage(Mail,...)
>
> Tks,
>
> Eric
>
>
> On 20/09/2010 14:50, Norman Maurer wrote:
>> Hi Eric,
>>
>> comments inside.
>>
>> 2010/9/20 Eric Charles<eric@apache.org>:
>>>   Hi,
>>>
>>> It's always possible to inject the mailboxmanager in mailets
>>> (ToRepository),... but we would  really gain to unify/simplify the 
>>> 2.3 mail
>>> storage with the 3.0 one.
>>>
>>> In 2.3, we have such class hierarchy:
>>>
>>> Repository
>>> +- AbstractFileRepository
>>>     +- File_Persistent_Object_Repository
>>>     +- File_Persistent_Stream_Repository
>>>
>>> MailRepository
>>> +- AbstractMailRepository
>>>     +- FileMailRepository
>>>     +- JCRMailRepository
>>>     +- JDBCMailRepository
>>> +- MBoxMailRepository
>>> +- SpoolRepository
>>>     +- InMemorySpoolRepository
>>>     +- MailStoreSpoolRepository
>>>
>>> Store
>>> +- AbstractMailStore
>>>     +- SpringMailStore
>>>
>>> In short, the SpringMailStore is responsible to register the
>>> MailRepositories and to select them.
>> Thats correct but I would more call it a hack to let it Spring load it
>>   ;) The whole Store interface is just a left-over from the old days
>> were we used Avalon/Excalibur for DI and component loading. I would be
>> more in favor to replace the Store interface with something which is
>> not such a mess..
>>
>>> In Mailbox projects introduced for 3.0, we have:
>>>
>>> MailboxManager
>>> +- StoreMailboxManager
>>>     +- InMemoryMailboxManager
>>>     +- JCRMailboxManager
>>>     +- JPAMailboxManager
>>>     +- MaildirMailboxManager
>>>
>>> With new 3.0 implementation, we only can deliver in one-and-only-one 
>>> store
>>> (jpa, jcr,...).
>>> We should overcome this limitation.
>> Thats the question.. Do we really want to overcome this ? I'm not sure
>> if the MailboxManager should replace the MailRepository for such cases
>> as storing mails in
>> the error processor etc. With the MailRepository we are able to store
>> the whole Mail including Mail Attributes etc, thats not the case with
>> MailboxManager. MailboxManager was not designed for that purpose. Its
>> just a storage for Mailboxes nothing more, nothing less ;) IMHO it
>> would make sense to "not merge" this twos and so only allow to
>> configure one MailboxManager implementation..
>>
>>
>>> Way to go further...:
>>>
>>> We should also discuss on the future way to evolve. There are different
>>> options/steps to go further with that unification/simplification:
>>>
>>> A/ Should we consider the MaiboxManager as just another MailStore, so
>>> developing a MailboxManagerMailStoreAdapter class and add the needed 
>>> entries
>>> to the mailstore.xml, so the existing SpringMailStore mecanic is 
>>> invoked.
>>>
>>> B/ Or simplifying and removing the Store and MailRepository hierarchy.
>>>
>>> C/ Any other options... ?
>>>
>> See above..
>>
>>> This could be also thought in parallel with a discussion on 
>>> UsersRepository
>>> hierarchy, so we have coherent way of working within James.
>>> I we go for option B/, it would be logical to also simplify the
>>> UsersRepository hierarchy.
>>> (Think also the DomainList has no Repository notion).
>>>
>> I think its a bit different. We should just Remove the UsersStore and
>> VirtualUserTableStore. There is no need to support more the one
>> implementation at one time
>> here. This just makes things more complex.
>>
>>> I think we should also extract the 2.3 classes/interfaces, and place 
>>> them in
>>> the correct mailbox projects (also create new mailbox projects for 
>>> example
>>> apache-james-mailbox-file that will contain FileMailRepository,..).
>>>
>> +1
>>
>>> WDYT?
>>>
>>> Eric
>>>
>>>
>>> On 6/09/2010 08:24, Eric Charles wrote:
>>>>   OK, we've done a picture of the current situation.
>>>>
>>>> I would like to pick an simple example as strong requirement to 
>>>> unify the
>>>> store/repository (whatever then name is) API across all projects.
>>>> In spoolmanager.xml, we define respositoryPath such as 
>>>> db://maildb/spam.
>>>> These repositoryPath are used in mailets,...
>>>> Let's say I want to develop a mailet that under certain conditions 
>>>> stores
>>>> the mail in a folder (let's say "spam" folder) of the user.
>>>> If the mailbox is for example JCR, I have no standard way to define 
>>>> the
>>>> folder path in the james-server "repository" syntax. Or I must 
>>>> invent one...
>>>> (jcr://...? or a more generic one mailbox:///? because I don't want to
>>>> change spoolmanager.xml if spring-beans.xml changes)
>>>>
>>>> If both are linked, we have clear conventions to implement.
>>>> And we can always release any project when we want.
>>>>
>>>> This is why I think we should link in a way or another the 
>>>> james-server
>>>> repository and the imap store. Not only on syntax extension, but 
>>>> also on
>>>> implementing classes.
>>>>
>>>> Tks,
>>>>
>>>> Eric
>>>>
>>>> On 6/09/2010 07:49, Norman Maurer wrote:
>>>>> Comments inline..
>>>>>
>>>>>
>>>>> 2010/9/6 Eric Charles<eric@apache.org>:
>>>>>>   Hi Norman,
>>>>>>
>>>>>> pop3server has store in its pom, but does not really need it. I will
>>>>>> carefully test before commit the changes.
>>>>> Should be safe to remove it then..
>>>>>
>>>>>> JPAUser is accessed in JPAUsersRepository (instanciated via the
>>>>>> LocalUsers
>>>>>> repository) while JPAMailbox is managed in JPAMailboMapper is 
>>>>>> accessed
>>>>>> via
>>>>>> JPAMailboxMapper. JPAUsersRepository and JPAMailboxMapper are 
>>>>>> created in
>>>>>> completely different ways.
>>>>> Ok, I think the "mapper stuff" is just in imap to make implementing
>>>>> different Mailbox stores easy. So its ok to have it work different.
>>>>>
>>>>>> I see that james-server has a whole infrastructure around 
>>>>>> "repository".
>>>>>> Imap
>>>>>> has not that infrastructure and rely on implementing different 
>>>>>> stores in
>>>>>> different projects. The questions are:
>>>>>> 1.- must imap go to server repository infra?
>>>>> Not sure..
>>>>>
>>>>>> 2.- must server repository infra be simplified and merged with 
>>>>>> the imap
>>>>>> approach?
>>>>> -1 No
>>>>>
>>>>>> 3.- must we leave that separated?
>>>>> Have them seperate allow to have a independend release cycle. Well I
>>>>> know imap had no release yet but I hope to get it done this week.
>>>>>
>>>>>> btw, even in james-server, all persistence does not go the same way:
>>>>>> domainlist does not work completely with repository.
>>>>>> If you take the JDBCDomainList, it needs a repositoryPath which is
>>>>>> db://maildb/domain (I don't see directly the relation with the
>>>>>> persistence.xml for the mailbox, and however, it stores in the same
>>>>>> database...). However, if you take the JPADomainList, it does not

>>>>>> need
>>>>>> any
>>>>>> "repository", because it's using the persistence.xml.
>>>>>> I mean, the repository principle has some exceptions in 
>>>>>> james-server.
>>>>>>
>>>>> The RepositoryPath stuff is from the good old days. With this concept
>>>>> its possible to use more then one db. Like db://maildb/domain (db =
>>>>> maildb, table = domain) and db://maildb2/domain (db= maildb2, table =
>>>>> domain). The whole concept is a "left-over" from using excalibur and
>>>>> avalon. To be honest I don't care if we remove it at all, one db
>>>>> should be enough...
>>>>>
>>>>>> Tks,
>>>>>>
>>>>>> Eric
>>>>>>
>>>>>>
>>>>>> On 5/09/2010 20:11, Norman Maurer wrote:
>>>>>>> Hi Eric,
>>>>>>>
>>>>>>> comments inside..
>>>>>>>
>>>>>>> 2010/9/5 Eric Charles<eric@apache.org>:
>>>>>>>>   Hi All,
>>>>>>>>
>>>>>>>> The store project has the mappers and imports some classes
of 
>>>>>>>> mailbox
>>>>>>>> (mailboxmanager,...), so 'store depends on mailbox'.
>>>>>>>>
>>>>>>>> To access some mailbox/subscription, you need to use the
maibox
>>>>>>>> project
>>>>>>>> (example: imap-processor depends on mailbox, and imapProcessor

>>>>>>>> bean is
>>>>>>>> injected with mailboxmanager/subscriptionManager). So far,
so 
>>>>>>>> good.
>>>>>>>>
>>>>>>>> mailbox is the interface to the mailboxes,...": I had a quick

>>>>>>>> look at
>>>>>>>> the
>>>>>>>> pop3server dependencies and saw that it was depending on
store. I
>>>>>>>> didn't
>>>>>>>> find that logical, removed it, and it was still compiling.

>>>>>>>> Logical...
>>>>>>>> Thinking in "what depends on what" helps much.
>>>>>>> pop3server should not depend on store and only depend on 
>>>>>>> mailbox. If
>>>>>>> its not the case its a bug..
>>>>>>>
>>>>>>>
>>>>>>>> However, the other '"domain classes" (user, virutalusertable,

>>>>>>>> domain
>>>>>>>> in
>>>>>>>> the
>>>>>>>> sense 'domain.tld') are accessed in a all different way.
This 
>>>>>>>> comes
>>>>>>>> probably
>>>>>>>> from the history (before using imap projects, server used
its own
>>>>>>>> storage
>>>>>>>> system which has been kept). In spring-beans.xml, you've
got:
>>>>>>>> - users-store: org.apache.james.container.spring.SpringUsersStore
>>>>>>>> class
>>>>>>>> being injected via @Resource
>>>>>>>> - virtualusertable-store :
>>>>>>>> org.apache.james.container.spring.SpringVirtualUserTableStore

>>>>>>>> being
>>>>>>>> injected
>>>>>>>> via @Resource
>>>>>>>> - domainlist: XML/JDBC/JPADomainList being injected via @Resource
>>>>>>>>
>>>>>>>> I see there 3 differents access to the persisted domain classes
>>>>>>>> (Mailbox,
>>>>>>>> Message, Subscription, Domain, User, VirtualUserTable):
>>>>>>>>
>>>>>>>>   1. Injection of mailbox project (MailboxManager, MessageManager,
>>>>>>>>      SubscriptionManger,...) beans
>>>>>>>>   2. Injection of spring-common project (SpringUsersStore,
>>>>>>>>      SpringVirtualUserTableStore) beans
>>>>>>>>   3. Injection of core-function project 
>>>>>>>> (XML/JDBC/JPADomainList) beans
>>>>>>>>
>>>>>>>> If you persist everything in a database, accesses for JPAUser

>>>>>>>> is done
>>>>>>>> completely differently compared to the JPAMailbox for example.
>>>>>>>>
>>>>>>> Could you explain whats the dfference ?
>>>>>>>
>>>>>>>> Coming to the point, should we:
>>>>>>>>
>>>>>>>>   1. Continue to live with heterogeneous access the persistence

>>>>>>>> layer?
>>>>>>>>   2.   Have uniform access to persistence for all services
and all
>>>>>>>>      domains? In this case, a store project would be a good

>>>>>>>> name, but
>>>>>>>>   is
>>>>>>>> already taken... we could always rename the existing one...
>>>>>>>>
>>>>>>>> WDYT?
>>>>>>>>
>>>>>>>> Eric
>>>>>>>>
>>>>>>>> PS: I didn't even talked of the AbstractFileRepository
>>>>>>>>
>>>>>>>>
>>>>>>> The whole *Store stuff is just a pita (james-server).. The idea

>>>>>>> with
>>>>>>> its was that you can configure for example different 
>>>>>>> UserRepositories
>>>>>>> and just lookup the right one via the UsersStore. So its really
>>>>>>> flexible. However the loading of the classes + injecting the
right
>>>>>>> dependencies is really complicated with this. I'm open to new

>>>>>>> ideas to
>>>>>>> make it more easy :)
>>>>>>>
>>>>>>> ---------------------------------------------------------------------

>>>>>>>
>>>>> Bye,
>>>>> Norman
>>>>>
>>>>> ---------------------------------------------------------------------
>> Bye,
>> Norman
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Mime
View raw message