From dev-return-55151-apmail-directory-dev-archive=directory.apache.org@directory.apache.org Mon Jan 30 09:26:14 2017 Return-Path: X-Original-To: apmail-directory-dev-archive@www.apache.org Delivered-To: apmail-directory-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9D14519677 for ; Mon, 30 Jan 2017 09:26:14 +0000 (UTC) Received: (qmail 89005 invoked by uid 500); 30 Jan 2017 09:26:14 -0000 Delivered-To: apmail-directory-dev-archive@directory.apache.org Received: (qmail 88952 invoked by uid 500); 30 Jan 2017 09:26:14 -0000 Mailing-List: contact dev-help@directory.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Apache Directory Developers List" Delivered-To: mailing list dev@directory.apache.org Received: (qmail 88942 invoked by uid 99); 30 Jan 2017 09:26:13 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 30 Jan 2017 09:26:13 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 7D3D118586A for ; Mon, 30 Jan 2017 09:26:13 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.473 X-Spam-Level: * X-Spam-Status: No, score=1.473 tagged_above=-999 required=6.31 tests=[DKIM_ADSP_CUSTOM_MED=0.001, NML_ADSP_CUSTOM_MED=1.2, RCVD_IN_DNSWL_LOW=-0.7, SPF_SOFTFAIL=0.972] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id Z7A6phltcEB6 for ; Mon, 30 Jan 2017 09:26:12 +0000 (UTC) Received: from zmcc-5-mx.zmailcloud.com (zmcc-5-mx.zmailcloud.com [192.198.93.228]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id D8BE75FC2F for ; Mon, 30 Jan 2017 09:26:11 +0000 (UTC) Received: from zmcc-5-mta-1.zmailcloud.com (127.37.197.104.bc.googleusercontent.com [104.197.37.127]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by zmcc-5-mx.zmailcloud.com (Postfix) with ESMTPS id A35DB520210 for ; Mon, 30 Jan 2017 04:26:05 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zmcc-5-mta-1.zmailcloud.com (Postfix) with ESMTP id 568ADC11BD for ; Mon, 30 Jan 2017 03:26:05 -0600 (CST) Received: from zmcc-5-mta-1.zmailcloud.com ([127.0.0.1]) by localhost (zmcc-5-mta-1.zmailcloud.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id TtnhPH0aG5Y2 for ; Mon, 30 Jan 2017 03:26:04 -0600 (CST) Received: from localhost (localhost [127.0.0.1]) by zmcc-5-mta-1.zmailcloud.com (Postfix) with ESMTP id D6EC3C2A2C for ; Mon, 30 Jan 2017 03:26:04 -0600 (CST) X-Virus-Scanned: amavisd-new at zmcc-5-mta-1.zmailcloud.com Received: from zmcc-5-mta-1.zmailcloud.com ([127.0.0.1]) by localhost (zmcc-5-mta-1.zmailcloud.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id xLJVp212Byrz for ; Mon, 30 Jan 2017 03:26:04 -0600 (CST) Received: from MacBook-Pro.local (LFbn-1-822-212.w86-246.abo.wanadoo.fr [86.246.56.212]) by zmcc-5-mta-1.zmailcloud.com (Postfix) with ESMTPSA id 89D38C11BD for ; Mon, 30 Jan 2017 03:26:04 -0600 (CST) To: dev@directory.apache.org References: Subject: Re: [Mavibot] Update From: =?UTF-8?Q?Emmanuel_L=c3=a9charny?= Message-ID: <0ed28f95-d464-a1e3-e8da-6b9e5755eeac@gmail.com> Date: Mon, 30 Jan 2017 10:26:00 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Hi ! End of the week-end... FTR here is what looks like the insert operation : try ( WriteTransaction writeTxn = recordManager.beginWriteTransaction() ) { btree.insert( writeTxn, 1L, "1" ); btree.insert( writeTxn, 4L, "4" ); btree.insert( writeTxn, 2L, "2" ); btree.insert( writeTxn, 3L, "3" ); btree.insert( writeTxn, 5L, "5" ); } As you can see, we use a try-with-resource, where the transaction is automatically committed, which makes it really easy for the users. Of course, one can abort the transaction at any moment, and the auto-commit will jst do nothing in this case. The extra advantage is that it's harder to forget to close a transaction :-) Of course, it's still possible to explicitely commit the transaction, if needed. One thing that came to my mind is that we don't necessarily need to detect that we had a crash. Currently, the idea was to save the current state (ie, the reference to the n-1 revision offsets in the RecordManagerHeader - the first page in the file -) when we start a txn, and to replace it with an updated version when we are done with the txn. If we have a crash in the middle, on restart, we will detect that the recordManagerHeader is in an intermediary state, and we will be able to scan the full fine to retrieve unlinked pages - in any case, read and write are stll possile duting this recovery phase, which is ran concurrently -. That comes with an extra write for every single txn. My take is that it's probably less expensive to do this scan on every start, no matter what, because this is a one-time expense, and this effort is not impacting reads and writes anyway. That would save one extra write per write transaction. In any case, the data will always be consistant, no matter what. More next week ! -- Emmanuel Lecharny Symas.com directory.apache.org