lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From robert engels <>
Subject Re: Back Compatibility
Date Tue, 22 Jan 2008 20:59:54 GMT
One more example on this. A lot of work was done on transaction  
support. I would argue that this falls way short of what is needed,  
since there is no XA transaction support. Since the lucene index  
(unless stored in an XA db) is a separate resource, it really needs  
XA support in order to be consistent with the other resources.

All of the transaction work that has been performed only guarantees  
that barring a physical hardware failure the lucene index can be  
opened and used at a known state.  This index though is probably not  
consistent with the other resources.

All that was done is that we can now guarantee that the index is  
consistent at SOME point in time.

Given the work that was done, we are probably closer to adding XA  
support, but I think this would be much easier if the concept of a  
transaction was made first class through the API (and then XA  
transactions need to be supported).

On Jan 22, 2008, at 2:49 PM, robert engels wrote:

> I don't think group C is interested in bug fixes. I just don't see  
> how Lucene is at all useful if the users are encountering any bug -  
> so they either don't use that feature, or they have already  
> developed a work-around (or they have patched the code in a way  
> that avoids the bug, yet is specific to their environment).
> For example, I think the NFS work (bugs, fixes, etc.) was quite  
> substantial. I think the actual number of people trying to use NFS  
> is probably very low - as the initial implementation had so many  
> problems (and IMO is not a very good solution for distributed  
> indexes anyway). So all the work in trying to make NFS work  
> "correctly" behind the scenes may have been inefficient, since a  
> more direct, yet major fix may have solved the problem better (like  
> distributed server support, not shared index access).
> I just think that trying to maintain API compatibility through  
> major releases is a bad idea. Leads to bloat, and complex code -  
> both internal and external.  In order to achieve great gains in  
> usability and/or performance in a mature product like Lucene almost  
> certainly requires massive changes to the processes, algorithms and  
> structures, and the API should change as well to reflect this.
> On Jan 22, 2008, at 2:30 PM, Chris Hostetter wrote:
>> : If they are " no longer actively developing the portion of the  
>> code that's
>> : broken, aren't seeking the new feature, etc", and they stay back  
>> on old
>> : versions... isn't that exactly what we want? They can stay on  
>> the old version,
>> : and new application development uses the newer version.
>> This basically mirrors a philosophy that is rising in the Perl
>> community evangelized by (a really smart dude named chromatic) ...
>> "why are we worry about the effect of upgrades on users who don't  
>> upgrade?"
>> The problem is not all users are created equal and not all users  
>> upgrade
>> for the same reasons or at the same time...
>> Group A: If someone is paranoid about upgrading, and is still running
>> lucene1.4.3 because they are afraid if they upgrade their app will  
>> break
>> and they don't want to deal with it; they don't care about known  
>> bugs in
>> lucene1.4.3, as long as those bugs haven't impacted them yet -- these
>> people aren't going to care wether we add a bunch of new methods to
>> interfaces, or remove a bunch of public methods from arbitrary  
>> releases,
>> because they are never going to see them.  They might do a total  
>> rewrite
>> of their project later, and they'll worry about it then (when they  
>> have
>> lots of time and QA resources)
>> Group: B: At the other extreme, are the "free-spirited" developers  
>> (god i
>> hate that that the word "agile" has been co-opted) who are always  
>> eager to
>> upgrade to get the latest bells and whistles, and don't mind making
>> changes to code and recompiling everytime they upgrades -- just as  
>> long as
>> there are some decent docs on what to change.
>> Croup: C: In the middle is a larg group of people who are  
>> interested in
>> upgrading, who want bug fixes, are willing to write new code to take
>> advantage of new features, in some cases are even willing to make
>> small or medium changes their code to get really good performance
>> improvements ... but they don't have a lot of time or energy to  
>> constantly
>> rewrite big chunks of their app.  For these people, knowing that  
>> they can
>> "drop in" the new version and it will work is a big reason why  
>> there are
>> willing to upgrade, and why they are willing to spend soem time
>> tweaking code to take advantage of the new features and the new
>> performacne enhaced APIs -- becuase they don't have to spend a lot  
>> of time
>> just to get the app working as well as it was before.
>> To draw an analogy...
>> Group A will stand in one place for a really long time no matter  
>> how easy
>> the path is.  Once in a great while they will decide to march forward
>> dozens of miles in one big push, but only once they feel they have
>> adequate resources to make the entire trip at once.
>> Group B likes to frolic, and will happily take two sptens backward  
>> and
>> then 3 steps forward every day.
>> Group C will walk forward with you at a steady pace, and  
>> occasionally even
>> take a step back before moving forward, but only if the path is  
>> clear and
>> not very steap.
>> : I bet, if you did a poll of all Lucene users, you would find a  
>> majority of
>> : them still only run 1.4.3, or maybe 1.9. Even with 2.0, 2.3, or  
>> 3.0, that is
>> : still going to be the case.
>> That's probably true, but a nice perk of our current backwards
>> compatibility commitments is that when people pop up asking questions
>> about 1.4.3, we can give them like "upgrading to 2.0.0 solves your
>> problem" and that advice isn't a death sentence -- the steps to move
>> forward are small and easy.
>> I look at things the way things like Maven v1 vs v2 worked out,  
>> and how
>> that fractured the community for a long time (as far as i can tell  
>> it's
>> still pretty fractured) because the path from v1 to v2 was so  
>> steep and
>> involved backtracking so much and i worry that if we make changes  
>> to our
>> "copatibility pledge" that don't allow for an even forward walk,  
>> we'll
>> wind up with a heavily fractured community.
>> -Hoss
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message