hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From y_823...@tsmc.com
Subject Re: Does HBASE allow multiple clients write to the same table?
Date Mon, 06 Jul 2009 03:38:00 GMT
Hi Nitay,

By counting the table records, I found it's ok to put data by multiple
clients until the warning occured.

In order to improve my put speed, I create a thread with HTable.
Something wrong with my codes?


class Thread2HBase extends Thread {

              public void saveData2HBase(ResultSet rs,String
TableName,String KeyColumn,String FamColumn,Vector<String> ColumnList){
            HTable  table = new HTable(this.config, TableName);
             // table.setAutoFlush(false);
             table.setWriteBufferSize(1024*1024*12);
            //===================================================
            Put p ;
            String tmp="";
            while (rs.next()) {
                  p = new Put(Bytes.toBytes(rs.getString(KeyColumn)));
                  Enumeration<String> columns =ColumnList.elements();
                  while(columns.hasMoreElements()){
                      String column = (String)columns.nextElement();
                      if (rs.getString(column)==null){
                        tmp="";
                     }else{
                        tmp=rs.getString(column);
                     }
                     p.add(Bytes.toBytes(FamColumn),
Bytes.toBytes(column),Bytes.toBytes(tmp));
                   }
                   table.put(p);
             }
             table.close();
      }

}


                                                                                         
                                                            
                      Nitay                                                              
                                                            
                      <nitayj@gmail.com        To:      hbase-user@hadoop.apache.org  
                                                               
                      >                        cc:      (bcc: Y_823910/TSMC)          
                                                               
                                               Subject: Re: Does HBASE allow multiple clients
write to the same table?                                
                      2009/07/03 02:10                                                   
                                                            
                      PM                                                                 
                                                            
                      Please respond to                                                  
                                                            
                      hbase-user                                                         
                                                            
                                                                                         
                                                            
                                                                                         
                                                            




Hi Fleming,

That warning is harmless. It should recover after such things. If you ever
see some fatal problems (crashes, data loss, corruption, etc) after such a
thing let us know.

HBase should handle multiple clients writing to the same table fine. You
will get better performance if they are writing to different regions (i.e.
random data), as then the load is split between multiple region serving
nodes.

-n

On Thu, Jul 2, 2009 at 9:28 PM, <y_823910@tsmc.com> wrote:

> Hi there,
>
> Now I use HTable to put my oracle table data to HBASE.
> In order to enhance the speed, I create a few HTable clients to put the
> data.
> It occured only some WARN like "WARN zookeeper.ClientCnxn: Exception
> closing session",
> not fetal error,evently I found there's no data increase in HBASE.
> Any HBASE setting about the client connection limitation?
> Or HBASE doesn't allow multiple clients write to the same table.
> Any suggestion?
> Thank you
>
> Fleming
>
>
---------------------------------------------------------------------------
>                                                         TSMC PROPERTY
>  This email communication (and any attachments) is proprietary
information
>  for the sole use of its
>  intended recipient. Any unauthorized review, use or distribution by
anyone
>  other than the intended
>  recipient is strictly prohibited.  If you are not the intended
recipient,
>  please notify the sender by
>  replying to this email, and then delete this email and any copies of it
>  immediately. Thank you.
>
>
---------------------------------------------------------------------------
>
>
>
>




 --------------------------------------------------------------------------- 
                                                         TSMC PROPERTY       
 This email communication (and any attachments) is proprietary information   
 for the sole use of its                                                     
 intended recipient. Any unauthorized review, use or distribution by anyone  
 other than the intended                                                     
 recipient is strictly prohibited.  If you are not the intended recipient,   
 please notify the sender by                                                 
 replying to this email, and then delete this email and any copies of it     
 immediately. Thank you.                                                     
 --------------------------------------------------------------------------- 




Mime
View raw message