drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andries Engelbrecht <aengelbre...@maprtech.com>
Subject Re: Drill 0.9 slowness/hanging
Date Tue, 12 May 2015 00:14:16 GMT
You set up the Storage plugin to use the loopback NFS to read the file, which is not the best
way to do it.

Why not use the MapR-FS or HDFS plug ins , this is a setup that uses the MapR-FS plugin which
will be considerably better than the loopback NFS for Drill


> {
>  "type": "file",
>  "enabled": true,
>  "connection": “maprfs:///",
>  "workspaces": {
>    "schema1": {
>      "location": "/user/theUser/test",
>      "writable": true,
>      "defaultInputFormat": null
>    }
>  },


—Andries


On May 11, 2015, at 5:07 PM, Minnow Noir <minnownoir@gmail.com> wrote:

> Hanifi:  The web UI is extremely slow, and the style sheets aren't loading,
> but here is what I see for "show databases"
> 
> Major FragmentMinor Fragments ReportingFirst StartLast StartFirst EndLast
> Endtmintavgtmaxlast updatelast progressmemmax 00-xx-xx1 /
> 10.213s0.213s0.670s0.670s0.457s0.457s0.457s12:15:3012:15:303MB
>  Major Fragment: 00-xx-xx
> <http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#fragment-0>
>  Minor FragmentHostStartEndTotal TimeMax RecordsMax BatchesLast UpdateLast
> ProgressPeak MemoryState 00-00-xxip-10-101-197-197.ec2.internal0.213s0.670s
> 0.457s6112:15:3012:15:303MBFINISHED
> Operator Profiles
>  Overview
> <http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-overview>
>  OperatorTypeSetup (min)Setup (avg)Setup (max)Process (min)Process
> (avg)Process
> (max)Wait (min)Wait (avg)Wait (max)Mem (avg)Mem (max) 00-xx-00SCREEN0.000s
> 0.000s0.000s0.000s0.000s0.000s0.000s0.000s0.000s-- 00-xx-01PROJECT0.422s
> 0.422s0.422s0.001s0.001s0.001s0.000s0.000s0.000s-- 00-xx-02
> INFO_SCHEMA_SUB_SCAN0.000s0.000s0.000s0.020s0.020s0.020s0.000s0.000s0.000s
> 17MB17MB
>   00-xx-00 - SCREEN
> <http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-0-0>
>  Minor FragmentSetupProcessWaitMax BatchesMax RecordsPeak Mem 00-00-00
> 0.000s0.000s0.000s16-
>   00-xx-01 - PROJECT
> <http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-0-1>
>  Minor FragmentSetupProcessWaitMax BatchesMax RecordsPeak Mem 00-00-01
> 0.422s0.001s0.000s16-
>   00-xx-02 - INFO_SCHEMA_SUB_SCAN
> <http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-0-2>
> Minor FragmentSetupProcessWaitMax BatchesMax RecordsPeak Mem
> 00-00-020.000s0.020s0.000s0017MB
> 
> Andries:  That storage plugin is very simple.  I've obfuscated some
> proprietary info.  (Note that the web UI was slow/hanging before I added
> the storage plugin.)
> 
> {
>  "type": "file",
>  "enabled": true,
>  "connection": "file:///",
>  "workspaces": {
>    "schema1": {
>      "location": "/mapr/theClusterName/user/theUser/test",
>      "writable": true,
>      "defaultInputFormat": null
>    }
>  },
>  "formats": {
>    "csv": {
>      "type": "text",
>      "extensions": [
>        "csv"
>      ],
>      "delimiter": ","
>    },
>    "tsv": {
>      "type": "text",
>      "extensions": [
>        "tsv"
>      ],
>      "delimiter": "\t"
>    },
>    "parquet": {
>      "type": "parquet"
>    }
>  }
> }
> 
> Unfortunately, because the Web UI is having issues, it is presenting an
> unstyled page with no disable links for the storage plugin so I can't
> disable it. With the web UI not working, I don't know how to delete the
> plugin.
> 
> Abdel: There are no errors in the logs.
> 
> 
> On Mon, May 11, 2015 at 7:22 PM, Hanifi Gunes <hgunes@maprtech.com> wrote:
> 
>> I would be interested in knowing where the time has been spent. Can you
>> inspect query profile from web ui for `show databases` and let us know how
>> the profile looks like?
>> 
>> On Mon, May 11, 2015 at 4:10 PM, Abdel Hakim Deneche <
>> adeneche@maprtech.com>
>> wrote:
>> 
>>> any errors in the logs ?
>>> 
>>> On Mon, May 11, 2015 at 4:03 PM, Minnow Noir <minnownoir@gmail.com>
>> wrote:
>>> 
>>>> Yes.  The bit and server are idle, with no jobs of any sort running on
>>>> them.
>>>> 
>>>> I just did started sqlline and it happened again.  I had to cancel the
>>> show
>>>> databases command after 30s.
>>>> 
>>>> sqlline version 1.1.6
>>>> 0: jdbc:drill:zk=theServer:5181> show databases;
>>>> +-------------+
>>>> | SCHEMA_NAME |
>>>> +-------------+
>>>> | INFORMATION_SCHEMA |
>>>> | cp.default  |
>>>> | dfs.default |
>>>> | dfs.root    |
>>>> | dfs.tmp     |
>>>> | test.default |
>>>> | test.schema1 |
>>>> | sys         |
>>>> +-------------+
>>>> 8 rows selected (29.264 seconds)
>>>> 
>>>> On Mon, May 11, 2015 at 6:57 PM, Hanifi Gunes <hgunes@maprtech.com>
>>> wrote:
>>>> 
>>>>> Can you confirm if this happens even when the particular 0.9 bit is
>>> idle?
>>>>> 
>>>>> -Hanifi
>>>>> 
>>>>> On Mon, May 11, 2015 at 3:53 PM, Minnow Noir <minnownoir@gmail.com>
>>>> wrote:
>>>>> 
>>>>>> Experiencing some slowness after upgrading one of our servers from
>>> 0.8
>>>> to
>>>>>> 0.9.  In particular, despite restarting the Drillbit, and even the
>>>>> server,
>>>>>> several times, the Drill web UI and sqlline are both so slow as to
>> be
>>>>>> unusable.
>>>>>> 
>>>>>> For example, doing a show databases hangs indefinitely, until I
>>> CTRL-C
>>>>> the
>>>>>> command (e.g., after a minute).
>>>>>> 
>>>>>> show databases;
>>>>>> +-------------+
>>>>>> | SCHEMA_NAME |
>>>>>> +-------------+
>>>>>> | INFORMATION_SCHEMA |
>>>>>> | cp.default  |
>>>>>> | dfs.default |
>>>>>> | dfs.root    |
>>>>>> | dfs.tmp     |
>>>>>> | test.default |
>>>>>> | test.schema1 |
>>>>>> | sys         |
>>>>>> +-------------+
>>>>>> 8 rows selected (59.044 seconds)
>>>>>> 
>>>>>> I tried changing schemas, but had to cancel after waiting several
>>>>> minutes.
>>>>>> 
>>>>>> use test.schema1;
>>>>>> +------------+------------+
>>>>>> |     ok     |  summary   |
>>>>>> +------------+------------+
>>>>>> | true       | Default schema changed to 'test.schema1' |
>>>>>> +------------+------------+
>>>>>> 1 row selected (208.089 seconds)
>>>>>> 
>>>>>> There are no errors or obvious issues in the logs. The server has
>>>> plenty
>>>>> of
>>>>>> disk, CPU, and RAM free.  It's running a recent version of MAPR
>>> Hadoop
>>>> on
>>>>>> CentOS 6.5 on EC2.  It feels like something is hanging
>>>>>> 
>>>>>> Note: This is just after starting sqlline; no actual work, which
>>> might
>>>>> have
>>>>>> slowed things down, has happened yet.
>>>>>> 
>>>>>> This is what's running from a Drill perspective:
>>>>>> 
>>>>>> ps aux | grep drill
>>>>>> ec2-user 24662  0.0  0.0 106100  1328 pts/0    S    12:29   0:00
>> bash
>>>>>> bin/drillbit.sh internal_start drillbit
>>>>>> ec2-user 24697  0.3  2.3 6486240 730568 pts/0  Sl   12:29   0:36
>>>>>> /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/bin/java
>>>>>> -Dlog.path=/opt/mapr/drill/drill-0.9.0/logs/drillbit.log -Xms1G
>>> -Xmx4G
>>>>>> -XX:MaxDirectMemorySize=8G -XX:MaxPermSize=512M
>>>>>> -XX:ReservedCodeCacheSize=1G -ea
>>>>>> -Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf
>>>>>> -Dzookeeper.sasl.client=false -XX:+CMSClassUnloadingEnabled
>>>>>> -XX:+UseConcMarkSweepGC -cp
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>> /opt/mapr/drill/drill-0.9.0/conf:/opt/mapr/drill/drill-0.9.0/jars/*:/opt/mapr/drill/drill-0.9.0/jars/ext/*:/opt/mapr/drill/drill-0.9.0/jars/3rdparty/*:/opt/mapr/drill/drill-0.9.0/jars/classb/*
>>>>>> org.apache.drill.exec.server.Drillbit
>>>>>> 
>>>>>> Drill has been started thusly:
>>>>>> 
>>>>>> bin/sqlline -u jdbc:drill:zk=theServer:5181
>>>>>> 
>>>>>> Any idea how I could troubleshoot this?
>>>>>> 
>>>>>> Anyone else experiencing slowness after upgrading?
>>>>>> 
>>>>>> Thank you
>>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> 
>>> Abdelhakim Deneche
>>> 
>>> Software Engineer
>>> 
>>>  <http://www.mapr.com/>
>>> 
>>> 
>>> Now Available - Free Hadoop On-Demand Training
>>> <
>>> 
>> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
>>>> 
>>> 
>> 


Mime
View raw message