hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Hbase REST returns invalid status code
Date Wed, 08 Feb 2017 02:52:21 GMT
Which release of hbase are you using ?

I was searching using the snippet from AsyncProcess and found HBASE-14431.

FYI

On Tue, Feb 7, 2017 at 6:29 PM, Akshat Mahajan <amahajan@brightedge.com>
wrote:

> We are seeing exceptions where Hbase REST is returning an invalid status
> code on writes, causing our Python processes to error out on trying to make
> a write (the following error is thrown by the Python starbase library, but
> is not caused by starbase directly):
>
> `requests.exceptions.ConnectionError: ('Connection aborted.',
> BadStatusLine("''",))`
>
> There are no traces nor errors reported in the REST log at the time of the
> exception. We know that this is consistent, as we retry thrice with
> exponential backoff (2 seconds initially).
>
> Our REST threads are allowed to grow dynamically from 100 to 9000 at
> worst; our connections typically do not exceed these values, and did not at
> the time.
>
> Are there circumstances where Hbase REST will deliberately refuse to honor
> a connection or respond with invalid codes? Or should I continue looking
> for more environmental reasons? Note that at the time REST was documenting
> wait on a number of AsyncProcesses on several tasks after having recently
> closed a table - not sure how useful that information is. A log snippet at
> the time from Hbase REST is below:
>
> ```
> 2017-02-07 17:07:35,312 INFO  [196541187@qtp-1054818236-2084]
> client.HBaseAdmin: Started disable of urls_table_50007_20170207170255
> 2017-02-07 17:07:43,294 INFO  [196541187@qtp-1054818236-2084]
> client.HBaseAdmin: Disabled urls_table_50007_20170207170255
> 2017-02-07 17:07:44,093 INFO  [196541187@qtp-1054818236-2084]
> client.HBaseAdmin: Deleted urls_table_50007_20170207170255
> 2017-02-07 17:07:53,238 INFO  [115866407@qtp-1054818236-2339]
> client.HBaseAdmin: Started disable of urls_table_50001_20170207170502
> 2017-02-07 17:07:59,953 INFO  [115866407@qtp-1054818236-2339]
> client.HBaseAdmin: Disabled urls_table_50001_20170207170502
> 2017-02-07 17:08:02,128 INFO  [115866407@qtp-1054818236-2339]
> client.HBaseAdmin: Deleted urls_table_50001_20170207170502
> 2017-02-07 17:08:04,585 INFO  [328489809@qtp-1054818236-2207]
> client.HBaseAdmin: Started disable of urls_table_50023_20170207170455
> 2017-02-07 17:08:04,791 INFO  [1227614359@qtp-1054818236-647]
> client.HBaseAdmin: Started disable of urls_table_50028_20170207170311
> 2017-02-07 17:08:05,664 INFO  [1252180146@qtp-1054818236-143]
> client.HBaseAdmin: Started disable of urls_table_50036_20170207170458
> 2017-02-07 17:08:07,783 INFO  [328489809@qtp-1054818236-2207]
> client.HBaseAdmin: Disabled urls_table_50023_20170207170455
> 2017-02-07 17:08:14,066 INFO  [1474938506@qtp-1054818236-2246]
> client.AsyncProcess: #1361530, waiting for some tasks to finish. Expected
> max=0, tasksInProgress=7
> 2017-02-07 17:08:14,167 INFO  [865578926@qtp-1054818236-1712]
> client.AsyncProcess: #1361562, waiting for some tasks to finish. Expected
> max=0, tasksInProgress=7
> 2017-02-07 17:08:14,267 INFO  [1276865244@qtp-1054818236-1407]
> client.AsyncProcess: #1361556, waiting for some tasks to finish. Expected
> max=0, tasksInProgress=7
> 2017-02-07 17:08:14,267 INFO  [1368235409@qtp-1054818236-315]
> client.AsyncProcess: #1361671, waiting for some tasks to finish. Expected
> max=0, tasksInProgress=7
> ```
>
> Akshat
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message