On Mon, Aug 8, 2011 at 12:49 PM, Jon Stafford <Jon.Stafford@complyserve.com> wrote:

I'm trying to understand why subversion isn't just storing diffs of some binary files.† It looks like itís taking up more space than it needs to.

At length the issue is described here: http://stackoverflow.com/q/6917505/277208

The more summarized version is...

I understand that subversion stores files differentially (compressed), even for binary files.† And subversion uses Xdelta to help with this.

I can do a standalone xdelta on my 50MB zip files:

xdelta3.0z.x86-64.exe -e -s v1_path\data.zip v2_path\data.zip v1v2_delta.file

and get a nicely small v1v2_delta.file which is about 3MB.

But when I check successive versions of the data.zip into a subversion repository (FSFS, version 1.6.11) I get two ~50MB files in db\revs\0

I appreciate minimizing disk usage isn't the only thing, e.g. performance may be more important.† Being able to configure this would be useful in my situation.

I've spent a long time trying to figure out what is going on and not got very far.† I guess this behaviour is probably deliberate - it would be really interesting to have an idea of different factors deciding subversion behaving in this way.


There are no knobs you can turn, and SVN internally does not have any heuristics it goes through where it decides not to produce deltas. †It always produces deltas, even when it is a waste of CPU to do so.

However, the fsfs backend also uses a technique called "skip-deltas" which you can read about here:

http://svn.apache.org/repos/asf/subversion/trunk/notes/skip-deltas

Have you only done two commits of this file? †Reading the above note, it looks like the first two revisions of the file are stored as full revisions.

--†
Thanks

Mark Phippard
http://markphip.blogspot.com/