phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sumit Nigam <sumit_o...@yahoo.com>
Subject Re: simple commands that mutate a large number of rows
Date Tue, 15 Sep 2015 15:12:24 GMT
Hi James,
Is it right to assume that with auto-commit set to true, the mutate maxSize being exceeded
error would not occur? This should be because now server side does the commit automatically
when the batch size is reached/ buffered.
Thanks,Sumit
      From: James Taylor <jamestaylor@apache.org>
 To: "user@phoenix.apache.org" <user@phoenix.apache.org> 
 Sent: Tuesday, September 15, 2015 7:37 PM
 Subject: Re: simple commands that mutate a large number of rows
   
That config setting (phoenix.mutate.maxSiize) is just a safety valve to prevent out of memory
errors and may be set to whatever you like. However, if you're going to just turn around and
do a commit after running your upsert statement, performance will improve if you turn on auto
commit instead (connection.setAutoCommit(true)), as that'll prevent double buffering and enable
execution completely on the server side when possible.
Thanks,James

On Tuesday, September 15, 2015, James Heather <james.heather@mendeley.com> wrote:



  I found today that I can't execute this:
 
 UPSERT INTO loadtest.testing (id, firstname, lastname) SELECT NEXT VALUE FOR loadtest.testing_id_seq,
firstname, lastname FROM loadtest.testing


 when the table has more than 500,000 rows in it ("MutationState size of 512000 is bigger
than max allowed size of 500000").
 
 Is there a reason for the restriction, or a way of lifting it so that things get processed
in batches?
 
 Does this also mean that if I had a large table, and I wanted to (say) null out one of the
columns, I wouldn't be able to do it in one hit because I'd be changing too many rows?
 
 James
  


   
Mime
View raw message