mina-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frederic Bregier <fred.breg...@free.fr>
Subject New project on starting block: OpenLSD using MINA
Date Sun, 13 May 2007 14:56:21 GMT
Hi all,
I am using MINA for one year now and I finally arrived at a point
where my project is reaching a level that start to satisfy me a little.

I post this not to make some publicity on my project but in case
Trustin wants to add it as one of the project that use MINA or some
benchmarks (at the end of this mail) review.

I have not finishing everything and of course I am sure that some
parts could be better, but I think it was time to open it to the community.
Of course I am open to any comment ! I do that on my spare time
so obviously it is not perfect... ;-)

This project, named OpenLSD, is about a framework for
legal document archiving and specifically for huge amount of files,
which stands for Open Legacy Storage Document.
You can find some infor on it on sourceforge and/or on
my web site : http://openlsd.free.fr/en/OpenLSDandCo.html

My employer (Ministery of Finance in France) has, like a lot
of administrations, some problems with electronic archiving:
- How to archive in electronic form about hundred of TeraBytes
  or even thousand of Terabytes
- How to be able to access them in a secure and in a performant way
  (well, Mina came here just at the right time)
- How to enable to adapt the code as easy as possible to new
   problem (my next focus will be OpenLSM for Open Legacy
   Storage Mail for email legal archiving)
- How to ensure security on storage (crypto on files for instance)

So I decided to develop something that could answear to the
various problems that we have to face.
It is currently starting its integration in the Ministery IT.
Of course, it should be also possible to use it for smaller storage
like departemental archive.

The project is not finished (will it be one day ?).
Today there is an example of a simple application (simple business)
based on this framework, using Oracle as database, MINA as network
framework and NIO also on filesystem access.
The next steps will be to correct MySQL support (my compagny has to use
Oracle so the priority on this one but I merge regularly the capabilities),
to enable cache support (read and write), to enable multiple OpenLSD
server (mirroring by OpenLSD for security and performance aspect),
to enable PostGreSQL also (I use JDBC but some optimization can be done
with each database).

Last but not least, my benchmark can be resume as follow :
- Import of 10 KB documents at a rate of 1420 files/second
     (so 1420 messages by second using MINA as framework between
    1 server and 8 MINA clients)
    The main time is in the file handling not in the MINA's part.
    The server was a 8 CPU Power5 AIX 5.3 IBM server using
    the IBM JVM 1.5.
- Retrieve 1000 documents by second through network (3 Tomcat servers
    using servlet with MINA client connected to OpenLSD MINA server)
    with 88 Mbits of bandwith (fully using MINA support) with a mean time
    of 0,012 seconds by document
    The Tomcat servers are 2-Xeon CPU 64bits using Linux (Suse 10),
    IBM JDK 1.5 and Tomcat 5.5. The LSD Server (storage and MINA server)
    was on a 2 CPU Power5 AIX 5.3 IBM server using the IBM JVM 1.5.
- Another simple test (without Database persistence) shows that I was 
able to
    get 2500 documents of 10KB by second using 1 Tomcat server with
    each servlet connecting to OpenLSD server using MINA, so
    400 Mbits of bandwith.
    The Tomcat server is a 2-Xeon CPU 64bits using Linux (Suse 10),
    IBM JDK 1.5 and Tomcat 5.5. The LSD Server (storage and MINA server)
    was on a 2 CPU Power5 AIX 5.3 IBM server using the IBM JVM 1.5.

Other benchmarks were done but are not relative to MINA, so I just put
it here to be compelete:
- Import in the same condition but in crypted mode (files are save using 
a key)
    gives 1290 files / second. (same condition as normal import)
- Check of consistency database/filesystem is done at 2400 files/second
    when MD5 computation is done (50 MB/s or 400 Mbits on a 2Gb SAN)
- Check of consistency database/filesystem is done at 30 000 files/second
    without MD5 computation is done (80 MB/s or 640 Mbits on a 2Gb SAN)

Just to be complete, the version of MINA is the trunk one.

I hope other projects will come on and on using MINA, a great piece of 
Thank you all, since MINA is powerful thanks to all its users.


View raw message