phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Batyrshin (Jira)" <j...@apache.org>
Subject [jira] [Created] (PHOENIX-5462) Region Server crash with: Added a key not lexically larger than previous
Date Tue, 03 Sep 2019 11:37:00 GMT
Alex Batyrshin created PHOENIX-5462:
---------------------------------------

             Summary: Region Server crash with: Added a key not lexically larger than previous
                 Key: PHOENIX-5462
                 URL: https://issues.apache.org/jira/browse/PHOENIX-5462
             Project: Phoenix
          Issue Type: Bug
    Affects Versions: 4.14.2
            Reporter: Alex Batyrshin


Region-Server crash with "Added a key not lexically larger than previous" after table update
with Phoenix client.

This bug also was created for HBase - https://issues.apache.org/jira/browse/HBASE-22862

Looks like mutations was added in reverse order or something like this.

Our table and pseudo code for table updates:
{code:java}
CREATE TABLE IF NOT EXISTS TBL_TABLE_CODE (
    "c" VARCHAR NOT NULL PRIMARY KEY,
    "d"."apd" TIMESTAMP,
    "d"."emd" TIMESTAMP,
    "d"."prid" VARCHAR,
    "d"."o" VARCHAR,
    "d"."elr" UNSIGNED_TINYINT,
    "d"."st" UNSIGNED_TINYINT
    ...
);
CREATE INDEX "IDX_TBL_TABLE_CODE_O" ON "TBL_TABLE_CODE" ("d"."o", "d"."emd") INCLUDE( "d"."elr",
"apd", "d"."st", ... );
CREATE INDEX "IDX_TBL_TABLE_CODE_PRID" ON "TBL_TABLE_CODE" ("d"."prid", "d"."emd") INCLUDE(
"d"."elr", "d"."apd", "d"."st", ...);

for (int i=0; i < batches.size; i += 1) {
  for (int j = 0; j < batches[i].size; j += 1 ) {
    UPSERT INTO TBL_TABLE_CODE VALUES (?, ?, ? ...);
  }
  commit;
}
{code}
Env:
HBase-1.4.10
Phoenix-4.14.2-HBase-1.4

Exception:
{code:java}
2019-08-15 18:02:21,777 FATAL [MemStoreFlusher.0] regionserver.HRegionServer: ABORTING region
server prod006,60020,1565873610692: Replay of WAL required. Forcing server shutdown
org.apache.hadoop.hbase.DroppedSnapshotException: region: TBL_TABLE_CODE,\x0904606203097821slG=sPD,1563070299676.5110b3395ca64a51cea99c6572a4c3d9.
       at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2675)
       at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2352)
       at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2314)
       at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2200)
       at org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:2125)
       at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:512)
       at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:482)
       at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:76)
       at org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:264)
       at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Added a key not lexically larger than previous. Current cell
= \x0901820448218>wGavb'/d:elr/1565881054828/DeleteColumn/vlen=0/seqid=44456567, lastCell
= \x0901820448218>wGavb'/d:elr/1565881054828/Put/vlen=1/seqid=44457770
       at org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:204)
       at org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:279)
       at org.apache.hadoop.hbase.io.hfile.HFileWriterV3.append(HFileWriterV3.java:87)
       at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.append(StoreFile.java:1127)
       at org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:139)
       at org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:75)
       at org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1003)
       at org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2523)
       at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2622)
       ... 9 more
{code}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Mime
View raw message