buildr-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Spiewak <djspie...@gmail.com>
Subject Re: Welcome to Rebase Hell!
Date Sun, 01 Mar 2009 07:02:57 GMT
Actually, I don't think that it is a restriction that Apache has placed on
their SVN server (AFAIK).  The way that git-svn works is it grabs every
single change set back to the beginning of time.  This means a really, *
really* huge checkout from the server, all at once.  While this is not
strictly a problem for SVN, it does pose some issues when SVN is being run
through mod_dav.  Basically, mod_svn_dav (or mod_svn, whichever it is) keeps
accruing objects in memory as the checkout progresses.  At some point, the
process handling your HTTP request actually crashes because it runs out of
heap space.

I ran into this issue at my previous job, though it would manifest itself as
a PROPSET error on svn checkout.  Because we had one particularly huge
changeset, I was never able to do a full checkout of all revisions from
scratch.  Using the svn client, things were at least passable (since you
could repeatedly "svn up"), but it wasn't pretty.  The really depressing
thing is that this is a known issue.  The mod_dav folks say that it's SVN's
fault, while the SVN folks keep bouncing the problem back.  I've also heard
it said that no one would ever run into this issue when using SVN
"properly", so it's not a serious concern.  :-S  The officially "workaround"
is to just allocate more memory to httpd.

I'm not sure why this wouldn't affect incubator, but it may be partially due
to the fact that you performed your initial clone of the Buildr repo at a
much earlier point than this.  Alternatively, there may be some other
factors in play here beyond annoying bugs associated with SVN+HTTPS.

Daniel

On Sun, Mar 1, 2009 at 12:40 AM, Assaf Arkin <arkin@intalio.com> wrote:

> On Sat, Feb 28, 2009 at 8:52 PM, Daniel Spiewak <djspiewak@gmail.com>
> wrote:
>
> > Those of you following development progress using Git are probably
> starting
> > to notice that the classical "Vic Master" is no longer the all-knowing
> > source of data.  Actually, Assaf's GitHub fork has become the more
> > trustworthy one.  This is because upon its exit from incubation, Buildr
> > gets
> > to move its SVN repository to a new URL.  This is good for the project,
> but
> > bad for the Git forks since git-svn stores the URL information in its
> > commit
> > messages.
> >
> > The solution of course is to re-clone from SVN, which I assume exactly
> what
> > Assaf did.  There result is a repository which contains all of the same
> SVN
> > commits as Vic's, but different messages and very different SHA1
> revisions,
> > meaning that Git has a much harder time merging between the two.  I
> > discovered this when I attempted to merge Assaf's latest changes with my
> > master (forked from Vic's).  57 conflicts later (all petty, little issues
> > unrelated to my additions), I finally had a working master with the
> latest
> > commits.  Unfortunately, when I cloned Assaf's repository directly and
> > attempted to merge back some of my changes, it became very apparent that
> I
> > would need to fix the issue in a more scientific manner.
> >
> > Long story short, the solution is to rebase all of your branches onto
> > Assaf's master.  I did this by finding the exact commit where I diverged
> > from vic (I had it tagged, actually) as well as the corresponding commit
> in
> > Assaf's master.  These commits I tagged "branch-point" and
> > "new-branch-point", respectively.  Then for each branch, I did something
> > like the following:
> >
> > git checkout scala-joint-compilation
> > git rebase new-branch-point
> >
> > Once this was done, I went back to my master and performed a similar
> > procedure:
> >
> > git checkout master
> > git rebase -s ours new-branch-point
> >
> > This effectively wiped out all of my changes in that branch (it's
> possible
> > that some commits may remain if you try it, but none did in my case).
>  Once
> > this was done, I went and picked through my origin/master log to see what
> I
> > was missing.  This meant re-merging all of my branches:
> >
> > git merge scala-joint-compilation
> > git merge clojure
> > ...
> >
> > Also, I had to cherry-pick a few commits that I had done on master (like
> > four or five):
> >
> > git cherry-pick all-your-ant-base
> > ...
> >
> > Once this was done, I pushed the result back to GitHub:
> >
> > git push -f origin
> >
> > The one caveat to this approach is I had all of my changes on numerous
> > separate branches (for patching reasons).  All of these branches were
> > branched off of the same point on vic/master.  Since there hadn't been
> any
> > merging *between* the branches (only onto master), it was fairly easy to
> > just rebase these branches onto the new trunk (I only had three conflicts
> > in
> > the whole process, all easily resolved).  Just judging by GitHub, not
> many
> > people are managing their repositories in this fashion.  However, this
> does
> > mean that you could be able to just rebase without the "-s ours" on your
> > master and come to the same result.
> >
> > The point is that you will need to perform some conniptions of this sort
> in
> > order to fix your repositories, otherwise your changes will remain
> > incompatible with the Buildr mainline trunk: you won't be able to
> (easily)
> > merge assaf/master, and he won't be able to (easily) pull from you.
> >
> > Incidentally, if anyone has a *better* way of doing this (particularly
> one
> > where the entire master history doesn't get wiped out), I'm all ears!  I
> do
> > still have the unmerged repository sitting in Time Machine, so I'm
> > perfectly
> > willing to roll-forward a copy and try again if the result turns out to
> be
> > more correct.
>
>
> Thanks.
>
>
> AFAIK it's not possible to git svn clone directly from svn.apache.org due
> to
> some weird restriction they placed on the SVN server, it will just keep
> git-svn hanging forever. Somehow that doesn't affect incubator projects, or
> svn.eu.apache.org, although my starting point was Jukka's unofficial but
> somewhat official git mirror[1].
>
>
> When you git log, check the git-svn-id:
>
> commit a3ab30a66a092bf730950bd95a1394253ebd2f39
>
> Author: Assaf Arkin <assaf@apache.org>
>
> Date:   Fri Feb 27 22:24:50 2009 +0000
>
>
> >     Fixed RDoc 2.3/2.4 conflict on rake setup.
>
>
>
>    git-svn-id:
>
> https://svn.eu.apache.org/repos/asf/buildr/trunk@74872213f79535-47bb-0310-9956-ffa450edef68
>
>
> If two repositories use a different URL -- look for http vs https,
> svn.eu.apache.org vs svn.apache.org, asf/buildr vs asf/incubator/buildr --
> Git considers them distinct trees (branches). Any time you merge, Git will
> have to merge the entire history of these two branches, leading to a lot of
> conflicts.
>
> So if you have branches with changes, follow Daniel's instructions. If you
> don't, you can still use the -s ours strategy to "switch" from one branch
> to
> another.
>
> Until Apache comes with a better solution, I'm going to keep my
> repository synchronized against
> https://svn.eu.apache.org/repos/asf/buildr.
> I suggest you all do the same, that way we have the same history and can
> easily merge with each other.
>
> Assaf
>
> [1] http://jukka.zitting.name/git/
>
>
>
> >
> >
> > Daniel
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message