phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ankit Singhal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3649) After PHOENIX-3271 higher memory consumption on RS leading to OOM/abort on immutable index creation with multiple regions on single RS
Date Wed, 01 Mar 2017 09:25:45 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889837#comment-15889837
] 

Ankit Singhal commented on PHOENIX-3649:
----------------------------------------

bq. Does PHOENIX-3271 set the time stamp of the distributed upsert to the time stamp of when
the query was started/compiled? We'd want to pass the time stamp over from the client so that
we're consistent across all region servers. If the time stamp is set correctly, then ImmutableIndexIT#testCreateIndexDuringUpsertSelect
should be ok.
No, we don't pass the timestamp of compilation as I thought it was needed only to cap the
query to not read the new data but with read isolation, we should not need this right? or
do you want updates to go at client timestamp even if SCN is also not set? we can't run UPSERT
SELECT on the server for immutable tables having indexes because Index maintenance of immutable
is still handled at the client.

bq. Otherwise, if it's not working for immutable tables, I'd expect it's not working for mutable
tables either
Yes, there will be the same problem if the mutable index is created during Upsert Select on
the table.

But currently also we have this problem right when the batch is sent to the server with index
maintainers (in cache or with mutations) then Index created during that time will not get
the updates on the fly. see PHOENIX-3583.

> After PHOENIX-3271 higher memory consumption on RS leading to OOM/abort on immutable
index creation with multiple regions on single RS
> --------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-3649
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3649
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.9.0
>            Reporter: Mujtaba Chohan
>            Assignee: Ankit Singhal
>            Priority: Blocker
>             Fix For: 4.9.1, 4.10.0
>
>         Attachments: PHOENIX-3649.patch, PHOENIX-3649_v1.patch
>
>
> *Configuration*
> hbase-0.98.23 standalone
> Heap 5GB
> *When*
> Verified that this happens after PHOENIX-3271 Distribute UPSERT SELECT across cluster.
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=accd4a276d1085e5d1069caf93798d8f301e4ed6
> To repro
> {noformat}
> CREATE TABLE INDEXED_TABLE (HOST CHAR(2) NOT NULL,DOMAIN VARCHAR NOT NULL, FEATURE VARCHAR
NOT NULL,DATE DATE NOT NULL,USAGE.CORE BIGINT,USAGE.DB BIGINT,STATS.ACTIVE_VISITOR INTEGER
CONSTRAINT PK PRIMARY KEY (HOST, DOMAIN, FEATURE, DATE)) IMMUTABLE_ROWS=true,MAX_FILESIZE=30485760
> {noformat}
> Upsert 2M rows (CSV is available at https://goo.gl/OsTSKB) that will create ~4 regions
on a single RS and then create index with data present
> {noformat}
> CREATE INDEX idx5 ON INDEXED_TABLE (CORE) INCLUDE (DB,ACTIVE_VISITOR)
> {noformat}
> From RS log
> {noformat}
> 2017-02-02 13:29:06,899 WARN  [rs,51371,1486070044538-HeapMemoryChore] regionserver.HeapMemoryManager:
heapOccupancyPercent 0.97875696 is above heap occupancy alarm watermark (0.95)
> 2017-02-02 13:29:18,198 INFO  [SessionTracker] server.ZooKeeperServer: Expiring session
0x15a00ad4f300001, timeout of 10000ms exceeded
> 2017-02-02 13:29:18,231 WARN  [JvmPauseMonitor] util.JvmPauseMonitor: Detected pause
in JVM or host machine (eg GC): pause of approximately 10581ms
> GC pool 'ParNew' had collection(s): count=4 time=139ms
> 2017-02-02 13:29:19,669 FATAL [RS:0;rs:51371-EventThread] regionserver.HRegionServer:
ABORTING region server rs,51371,1486070044538: regionserver:51371-0x15a00ad4f300001, quorum=localhost:2181,
baseZNode=/hbase regionserver:51371-0x15a00ad4f300001 received expired from ZooKeeper, aborting
> {noformat}
> Prior to the change index creation succeeds with as little as 2GB heap.
> [~ankit@apache.org]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message