james-server-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pepijn de Vos <pepijnde...@yahoo.com>
Subject Re: NoSQL backend for James
Date Sun, 16 Oct 2011 16:24:47 GMT
Thanks man, really appreciated.

I too think my ModSeq impl looks good, but incrementing the modSeq 4 times in parallel, I
noticed that

the value is incremented only 2-3 times.
_rev is incremented even less.
additional documents get created.

What happens is that the same mailbox object gets passed to all invocations of nextModSeq.
This means that incrementing them in one increments it everywhere. Normally this would only
create conflicts in all but one, but because this mailbox does not have an id yet, it creates
a new mailbox for every conflict.

So I assume this is not a problem in practice? Because after your fix, all created mailboxes
actually do have an id. I'll test that later, and then try to figure out JUnit.

Pepijn

On Oct 16, 2011, at 4:23 PM, Manuel Carrasco Moñino wrote:

> On Sat, Oct 15, 2011 at 5:40 PM, Pepijn de Vos <pepijndevos@yahoo.com> wrote:
>> So, the result and questions for today:
>> 
>> https://github.com/pepijndevos/james-couchdb
>> 
>> What is serialVersionUID?
> 
> It is a good practice to define it for serializable classes
> http://www.javapractices.com/topic/TopicAction.do?Id=45
> 
>> 
>> Is UidValidity something else than what is provided by CouchDbUidProvider?
> 
> It's a number that changes when all uids become invalid. The server
> cannot update this number during a session.
> http://www.afterlogic.com/mailbee-net/docs/MailBee.ImapMail.Imap.UidValidity.html
> 
>> 
>> I had a look at the InMemory test, and it seems to fail on remainders of the InMemory
implementation(mentions of ConcurrentHashmap). I'm not sure what it tests, but I think it's
more than just CouchDbMailboxManager.
>> 
>> Today I tried to implement ModSeq and Uid. They work fine until you introduce concurrency.
I'd be glad if someone would take a look at nextModSeq.
> 
> Your implementation looks good.
> 
>> 
>> I'll try to write a test for it tomorrow. The thing is that the test would need the
database, right? How do you handle that?
> 
> Difficult since it seems there is no way to install couchdb and start
> it from maven.
> Right now tests should be ignored when ddbb is not running (see
> maildir tests which are omitted in windows systems)
> I've added in the setup a call to empty the 'james' database when the
> test run, also I have added  initStandardDesignDocument() to
> initialize database.
> Be careful before run the test if you do not want the ddbb be re-created.
> 
> Happy coding.
> - Manolo
> 
>> 
>> Pepijn
>> 
>> On Oct 15, 2011, at 11:41 AM, Pepijn de Vos wrote:
>> 
>>> Thanks! :)
>>> 
>>> I have no idea what you did with the generics though. I read an article about
them, and thought I understood them.
>>> 
>>> Basically they are just statically checked casts, right? So what does it mean
to write X implements Y<Z>?
>>> 
>>> The tests are just leftovers from InMemory. I'm not doing TDD, but I rather see
tests as a frozen REPL. I write code, test it in the REPL, if I find myself repeating pieces
in the REPL, I make them into a test. I didn't get to the stage where I could even compile
anything.
>>> 
>>> Pepijn
>>> 
>>> On Oct 14, 2011, at 8:14 PM, Manuel Carrasco Moñino wrote:
>>> 
>>>> You are right, you already sent the github link.
>>>> 
>>>> I've checked out the code and I've made some changes to make it
>>>> compile, also I have changed id signature in messages and mailboxes to
>>>> String since couchdb uses string. I have hardcoded a user and password
>>>> in the Utils class, it should go in the configuration files though.
>>>> 
>>>> I just have sent you a pull request
>>>> https://github.com/manolo/james-couchdb/commit/2b6b0ebce5c78198b14d1e7ec05c178d56482919
>>>> 
>>>> The unique Test in the tree do no pass, I think you can start fixing
>>>> the test so as it should be easier to follow the code taking a look to
>>>> tests.
>>>> 
>>>> - Manolo
>>>> 
>>>> 
>>>> On Thu, Oct 13, 2011 at 9:54 PM, Pepijn de Vos <pepijndevos@yahoo.com>
wrote:
>>>>> I added the github link to the beautifully formatted original email,
but I think Google mistreated my message.
>>>>> 
>>>>> Her it is again: https://github.com/pepijndevos/james-couchdb
>>>>> 
>>>>> Pepijn
>>>>> 
>>>>> On Oct 13, 2011, at 9:28 PM, Manuel Carrasco Moñino wrote:
>>>>> 
>>>>>> Hi Pepjin, could be possible to share your code anywhere, so as I
>>>>>> could checkout it and take a look?
>>>>>> 
>>>>>> Don't worry about if the code is ok or not, I think github could
be
>>>>>> ok, but you could send a compressed file via email or whatever you
>>>>>> prefer.
>>>>>> 
>>>>>> - Manolo
>>>>>> 
>>>>>> 
>>>>>> On Thu, Oct 13, 2011 at 8:12 PM, Pepijn de Vos <pepijndevos@yahoo.com>
wrote:
>>>>>>> Okay, I put up the result for the day.
>>>>>>> I made a CouchDbMailbox with Ektorp @annotations.
>>>>>>> I made an attempt to make the MailboxMapper, but I got stuck
at the <type> casting sugar which I don't grok. My IDE keeps complaining it can't resolve
the incompatible types, while both are just Mailboxes.
>>>>>>> I stuffed the CouchDB connection in a class, not happy with it.
>>>>>>> I'm not sure how to implement the findMailboxWithPathLike and
hasChildren methods.
>>>>>>> Any help appreciated, especially with the... <> things.
list() is the only one that's red wiggly lines, the others are just unchecked casts. I've
been adding random casts and <> left and right.
>>>>>>> 
>>>>>>> https://github.com/pepijndevos/james-couchdb
>>>>>>> 
>>>>>>> Pepijn
>>>>>>> 
>>>>>>> On Oct 13, 2011, at 6:23 PM, Pepijn de Vos wrote:
>>>>>>> 
>>>>>>>> More questions. I started hacking!
>>>>>>>> 
>>>>>>>> I'm going with Ektorp. I figured out most of it, I think.
Except that I don't understand the configuration. It does have a Spring module. Any pointers
on how to organize the config and connections?
>>>>>>>> 
>>>>>>>> What does findMailboxWithPathLike do? The implementations
seem to do weird things with regexes. Preferably I make that into a nice CouchDB view. CouchDB
can't do fulltext search. As far as I can tell, the IMAP RFC doesn't say anything about it.
>>>>>>>> 
>>>>>>>> To have custom message and mailbox classes, do I need to
do anything else besides subclassing the corresponding *Manager class to return one?
>>>>>>>> 
>>>>>>>> I'm getting there!
>>>>>>>> 
>>>>>>>> Pepijn
>>>>>>>> 
>>>>>>>> On Oct 12, 2011, at 9:35 PM, Manuel Carrasco Moñino wrote:
>>>>>>>> 
>>>>>>>>> Actually Ektorp is not a full implementation of JPA,
but it provides a
>>>>>>>>> JPA like API with support to many of its annotations
etc.
>>>>>>>>> 
>>>>>>>>> Anyway, based on my experience, Ektorp simplifies the
access from java
>>>>>>>>> to couchdb and the bootstrap of couchdb, so as theoretically
when
>>>>>>>>> James starts the first time, the database, views, design,
mapreduce,
>>>>>>>>> etc should be created.
>>>>>>>>> 
>>>>>>>>> - Manolo
>>>>>>>>> 
>>>>>>>>> On Wed, Oct 12, 2011 at 1:05 PM, Pepijn de Vos <pepijndevos@yahoo.com>
wrote:
>>>>>>>>>> Ektorp seems nice, but I'm more comfortable just
using something that resembles the HTTP API, since I'm not familiar with JPA. Haven't decided
yet.
>>>>>>>>>> 
>>>>>>>>>> Pepijn
>>>>>>>>>> 
>>>>>>>>>> On Oct 11, 2011, at 5:08 PM, Manuel Carrasco Moñino
wrote:
>>>>>>>>>> 
>>>>>>>>>>> Hi Pepijn
>>>>>>>>>>> 
>>>>>>>>>>> Which java library are you considering to use
to connect with couchdb?
>>>>>>>>>>> I'm using [1] ektorp and makes really easy to
map domain models.
>>>>>>>>>>> 
>>>>>>>>>>> - Manolo
>>>>>>>>>>> 
>>>>>>>>>>> [1] http://www.ektorp.org/reference_documentation.html#d0e532
>>>>>>>>>>> 
>>>>>>>>>>> On Mon, Oct 10, 2011 at 10:44 PM, Pepijn de Vos
<pepijndevos@yahoo.com> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks a lot.
>>>>>>>>>>>> 
>>>>>>>>>>>> On Oct 10, 2011, at 8:24 PM, Ioan Eugen Stan
wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Have patience. You will need it if you
wish to complete something.
>>>>>>>>>>>>> Patience and perseverance or else you'll
be just another one who
>>>>>>>>>>>>> tried.
>>>>>>>>>>>> 
>>>>>>>>>>>> I don't expect to have it finished by the
end of the week, but if I'm still completely clueless by then, it's just not worth the effort.
>>>>>>>>>>>> I don't have the ambition to become a James
commiter or even a Java dev, I just thought it would be nice to use CouchDB for my application.
>>>>>>>>>>>> Somewhere is a point where pragmatism beats
learning. There isn't any technical reason why I can't use JPA.
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Can I just copy an existing one and
rename stuff? In other words, how are the modules glued into the whole? How does the server
know which class to load? It's not in the pom.xml, afaict.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Not sure what you mean by that. It uses
dependency injection provided
>>>>>>>>>>>>> by Spring framework (and soon Guice)
to inject object references into
>>>>>>>>>>>>> other objects at runtime.
>>>>>>>>>>>> 
>>>>>>>>>>>> Ah, dependency injection. *googles* So just
the fact that I implement the interface is enough to @autowire it into James?
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> The sample config is gone btw: http://james.apache.org/server/3/config-mailbox.html
>>>>>>>>>>>>>> Do I inherit tests as well? I would
imagine that a lot of tests are common to all mailbox implementations.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I think this is because the configuration
changed and now it's spring
>>>>>>>>>>>>> based, and more modular. I see you are
very ambitious but I sense you
>>>>>>>>>>>>> have a lot of catching up. James is complex
so give it time, if you
>>>>>>>>>>>>> expect too much from yourself and fail
you will probably be too
>>>>>>>>>>>>> disappointed.
>>>>>>>>>>>> Yea, I read a Java book long ago, never did
any big projects with it.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Make a public repo, commit something
and ask if you get stuck. I will
>>>>>>>>>>>>> try to help when/if I can. I suggest
you start with simple
>>>>>>>>>>>>> implementation that passes some unit
tests.
>>>>>>>>>>>> 
>>>>>>>>>>>> So If I take any mailbox impl, put it in
a separate repo, will it work? All sorts of things refer to the parent pom. I'll put something
on github once I figure it out. I think it'll work out once I get to the point where I can
write some code.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> See for example the unit tests I did
for Mailbox interface in HBase
>>>>>>>>>>>>> implementation:
>>>>>>>>>>>>> http://svn.apache.org/repos/asf/james/mailbox/trunk/hbase/src/test/java/org/apache/james/mailbox/hbase/mail/model/HBaseMailboxTest.java
>>>>>>>>>>>> What I mean with inheriting tests is that
these all look very generic. They look like they could test any mailbox implementation.
>>>>>>>>>>>> 
>>>>>>>>>>>> Pepijn
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Ioan Eugen Stan
>>>>>>>>>>>>> http://ieugen.blogspot.com/
>>>>>>>>>>>>> 
>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>>>>>>>>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>>>>>>>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>>>>>>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>>>>>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>>>>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>>>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>> 
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>> 
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>> 
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message