hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ganesh V <gan...@gmail.com>
Subject Hourly performance degradation on HBase cluster
Date Fri, 11 Mar 2016 23:02:20 GMT
Hello-

I am evaluating HBase for storage of time-series data and see periodic
degradation of the cluster happening at the start of the hour, every hour,
for about 10mins. I see "WALProcedureStoreSyncThread" (wal rolling?) in the
logs during this time like so:

2016-03-01 06:09:35,151 INFO  [WALProcedureStoreSyncThread]
wal.WALProcedureStore: Remove log:
hdfs://hdmaster01:54310/hbase/MasterProcWALs/state-00000000000000000009.log

In my tests, I'm pushing 1million data points (each about 200bytes in size)
into the cluster each minute at the same time using 5000 threads. I am also
querying 1000 of these data points each minute to test latency, errors, and
performance. The cluster runs absolutely fine except for those ~10mins at
the start of every hour when both reads and writes are affected. Could
someone suggest which part of HBase needs tuning? Thanks!


-Ganesh

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message