phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3507) HBase secure wal log cash with Phoenix secondary index
Date Thu, 24 Nov 2016 11:37:58 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15693043#comment-15693043
] 

Anoop Sam John commented on PHOENIX-3507:
-----------------------------------------

Copied from HBase issue
"I believe I got what is the issue
AbstractProtobufLogWriter#initAfterHeader will do Codec create and will consider the config
'hbase.regionserver.wal.codec'. But SecureProtobufLogWriter, which is in use here, override
initAfterHeader() and that in turn creates SecureWALCellCodec only. The even if the config
says to use a diff Codec, that is not at all getting used. In Pheonix this Codec might be
having some extra logic to handle some special KVs. There was an IndexKV usage in the past..
Not sure abt cur code base of Phoenix.
So HBase side issue is this - Config 'hbase.regionserver.wal.codec' is discarded when wal
encryption is ON."
Ping [~apurtell]..  With WAL encryption, we intended not to support  custom wal cell codec?
 Or else we should make SecureWALCellCodec limited private and document that when encryption
is ON, the custom wal codec impl has to extend this NOT WALCellCodec.

> HBase secure wal log cash with Phoenix secondary index
> ------------------------------------------------------
>
>                 Key: PHOENIX-3507
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3507
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.8.0
>         Environment: Hadoop 2.7.3
> HBase 1.2.4
> Phoenix 4.8.1
> JDK 1.8.0_112
>            Reporter: SonixLegend
>             Fix For: 4.9.0
>
>
> When I used wal.encryption and phoenix secondary indexes, I got the error message
> 2016-11-24 17:11:20,361 INFO [regionserver/host-172-31-0-68/172.31.0.68:16020.logRoller]
wal.FSHLog: Rolled WAL /hbase/WALs/host-172-31-0-68,16020,1479976404182/host-172-31-0-68%2C16020%2C1479976404182.default.1479978679455
with entries=0, filesize=242 B; new WAL /hbase/WALs/host-172-31-0-68,16020,1479976404182/host-172-31-0-68%2C16020%2C1479976404182.default.1479978680140
> 2016-11-24 17:11:20,363 INFO [regionserver/host-172-31-0-68/172.31.0.68:16020.logRoller]
wal.FSHLog: Archiving hdfs://data/hbase/WALs/host-172-31-0-68,16020,1479976404182/host-172-31-0-68%2C16020%2C1479976404182.default.1479978679455
to hdfs://data/hbase/oldWALs/host-172-31-0-68%2C16020%2C1479976404182.default.1479978679455
> 2016-11-24 17:11:20,654 WARN [regionserver/host-172-31-0-68/172.31.0.68:16020.append-pool1-t1]
wal.FSHLog: Append sequenceId=9, requesting roll of WAL
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.util.Bytes.toInt(Bytes.java:801)
> at org.apache.hadoop.hbase.util.Bytes.toInt(Bytes.java:788)
> at org.apache.hadoop.hbase.KeyValue.getKeyLength(KeyValue.java:1324)
> at org.apache.hadoop.hbase.KeyValue.getTagsLength(KeyValue.java:1638)
> at org.apache.hadoop.hbase.regionserver.wal.SecureWALCellCodec$EncryptedKvEncoder.write(SecureWALCellCodec.java:199)
> at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.append(ProtobufLogWriter.java:122)
> at org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1932)
> at org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1794)
> at org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1704)
> at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> 2016-11-24 17:11:20,810 WARN [regionserver/host-172-31-0-68/172.31.0.68:16020.logRoller]
wal.FSHLog: Failed sync-before-close but no outstanding appends; closing WAL: org.apache.hadoop.hbase.regionserver.wal.DamagedWALException:
Append sequenceId=9, requesting roll of WAL
> And my hbase-site.xml is containing the settings.
> <property>
> <name>hbase.regionserver.wal.encryption</name>
> <value>true</value>
> </property>
> <property>
> <name>hbase.regionserver.hlog.reader.impl</name>
> <value>org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogReader</value>
> </property>
> <property>
> <name>hbase.regionserver.hlog.writer.impl</name>
> <value>org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogWriter</value>
> </property>
> <property>
> <name>hbase.regionserver.wal.codec</name>
> <value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value>
> </property>
> And my phoenix table sql
> CREATE SCHEMA TEST;
> CREATE TABLE TEST.TEST (ID BIGINT NOT NULL PRIMARY KEY, TEST VARCHAR);
> UPSERT INTO TEST.TEST (ID, TEST) VALUES (1, 'test');
> CREATE INDEX TEST_INDEX ON TEST.TEST (TEST);



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message