phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Taylor <jamestay...@apache.org>
Subject Re: Dropping local index on large table caused crash of region servers
Date Thu, 05 Jan 2017 18:37:40 GMT
It should not be necessary to increase the RPC timeout when dropping an
index with the renew lease changes we made (Phoenix 4.7.0+ and HBase
1.1.3+). If it still is, please file a JIRA.

Samarth - can you confirm the version requirements?

Also - take a look at PHOENIX-3262. It'd be an interesting approach to
dropping a local index.

Thanks,
James

On Thu, Jan 5, 2017 at 10:25 AM, rajeshbabu@apache.org <
chrajeshbabu32@gmail.com> wrote:

> Dropping local index index issues scan and in the coprocessors we prepare
> index data and write back to same table.  To avoid the timeout you can
> increase scanner timeout/rpc timeout and phoenix client timeout values as
> well.
>
> If you have many guideposts then we might issue many parallel scans to the
> same RS which might have added lot of overhead.
> Try dropping the index with less number of guideposts on the table.
>
> Actually during major compaction we can automatically skip writing back the
> deleted index details so we can just drop meta data. I am checking this
> improvement. Raised https://issues.apache.org/jira/browse/PHOENIX-3566 for
> the same.
>
> Thanks,
> Rajeshbabu.
>
> On Thu, Jan 5, 2017 at 7:16 PM, 김영우 (YoungWoo Kim) <ywkim@apache.org>
> wrote:
>
> > Hi,
> >
> > I've faced crash of RS when I'm dropping local index from very large
> table.
> > I have several tables with billions of row and the tables are hosted on
> > HBase 1.2.4 and Phoenix 4.9.0
> >
> > Each region server reported 'gc pause' from their logs and it's strange
> to
> > me because, the log messages are popped up when I drop local indexes.
> > eventually, my query failed with timeout. any other operations like
> > insert/bulkload data or querying data works fine.
> >
> > I'm not sure but dropping local indexes from large table is expensive. Is
> > there any workaround for this?
> >
> > Thanks,
> > Youngwoo
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message