hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From N Keywal <nkey...@gmail.com>
Subject Re: batch insert performance
Date Sun, 27 May 2012 17:17:45 GMT
Hi,

What version are you using?
On trunk, put(Put) and put(List<Put>) calls the same code, so I would
expect  comparable performances when autoflush it set to false.

However, with 250K small puts you may have the gc playing a role.

What are the results if you do the inserts with 50 times 5K rows?

N.

On Sun, May 27, 2012 at 1:58 AM, Faruk Berksöz <fberksoz@gmail.com> wrote:
> Codes and their results :
>
>    Code Nmr List List Size Code Avarage Elapsed Time (sec)  1 List<Put>
> batchAllRows; 250.000   table.setAutoFlush(false);
>  for (Put mRow : batchAllRows) {
>   table.put(mRow);
>  }
>  table.flushCommits(); 27       2 List<Put> batchAllRows; 250.000
> table.setAutoFlush(false);
>    table.put(batchAllRows);
>    table.flushCommits(); 103     3 List<Row> batchAllRows; 250.000
> table.setAutoFlush(false);
>  Object[] results = new Object[batchAllRows.size()];
>  table.batch(batchAllRows, results);
>  //table.batch(batchAllRows) ; /* already tried */
>  table.flushCommits(); 105
> ---------- Forwarded message ----------
> From: Faruk Berksöz <fberksoz@gmail.com>
> Date: 2012/5/27
> Subject: batch insert performance
> To: user@hbase.apache.org
>
>
> Hi, HBase users,
>
> I have 250.000 Rows in a list.
> I want to insert all rows in HTable as soon as possible.
> I have 3 different Code and 3 different elapsed time.
> Why HTable.batch(List<? extends Row> actions, Object[] results) and
> HTable.put(List<Put>
> puts) methods 4 times slower than 1.Code which inserts records to htable in
> a simple loop ?
> Codes and their results :
>
>
>
>
>
>
> Faruk

Mime
View raw message