drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andries Engelbrecht <aengelbre...@maprtech.com>
Subject Re: Drill 0.9 slowness/hanging
Date Tue, 12 May 2015 00:26:28 GMT
instead of file:///  use maprfs:///  (I assume it is a mapr cluster)

If you do that you don’t need to use /mapr/clustername in front of the location path, but
you can rather just directly point to the maprfs directory.

See the sample I posted vs the storage plug in you had configured.

The default DFS plugin should also show you the correct config info.

—Andries

On May 11, 2015, at 5:21 PM, Minnow Noir <minnownoir@gmail.com> wrote:

> I'm not sure I understand what you mean, Andries.  What field and value are
> you suggesting that I change?  If there's a better way to do this, I would
> like to do it the right way, even if it's not the issue in question.
> 
> Thanks
> 
> On Mon, May 11, 2015 at 8:14 PM, Andries Engelbrecht <
> aengelbrecht@maprtech.com> wrote:
> 
>> You set up the Storage plugin to use the loopback NFS to read the file,
>> which is not the best way to do it.
>> 
>> Why not use the MapR-FS or HDFS plug ins , this is a setup that uses the
>> MapR-FS plugin which will be considerably better than the loopback NFS for
>> Drill
>> 
>> 
>>> {
>>> "type": "file",
>>> "enabled": true,
>>> "connection": “maprfs:///",
>>> "workspaces": {
>>>   "schema1": {
>>>     "location": "/user/theUser/test",
>>>     "writable": true,
>>>     "defaultInputFormat": null
>>>   }
>>> },
>> 
>> 
>> —Andries
>> 
>> 
>> On May 11, 2015, at 5:07 PM, Minnow Noir <minnownoir@gmail.com> wrote:
>> 
>>> Hanifi:  The web UI is extremely slow, and the style sheets aren't
>> loading,
>>> but here is what I see for "show databases"
>>> 
>>> Major FragmentMinor Fragments ReportingFirst StartLast StartFirst EndLast
>>> Endtmintavgtmaxlast updatelast progressmemmax 00-xx-xx1 /
>>> 10.213s0.213s0.670s0.670s0.457s0.457s0.457s12:15:3012:15:303MB
>>> Major Fragment: 00-xx-xx
>>> <
>> http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#fragment-0
>>> 
>>> Minor FragmentHostStartEndTotal TimeMax RecordsMax BatchesLast
>> UpdateLast
>>> ProgressPeak MemoryState
>> 00-00-xxip-10-101-197-197.ec2.internal0.213s0.670s
>>> 0.457s6112:15:3012:15:303MBFINISHED
>>> Operator Profiles
>>> Overview
>>> <
>> http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-overview
>>> 
>>> OperatorTypeSetup (min)Setup (avg)Setup (max)Process (min)Process
>>> (avg)Process
>>> (max)Wait (min)Wait (avg)Wait (max)Mem (avg)Mem (max)
>> 00-xx-00SCREEN0.000s
>>> 0.000s0.000s0.000s0.000s0.000s0.000s0.000s0.000s-- 00-xx-01PROJECT0.422s
>>> 0.422s0.422s0.001s0.001s0.001s0.000s0.000s0.000s-- 00-xx-02
>>> 
>> INFO_SCHEMA_SUB_SCAN0.000s0.000s0.000s0.020s0.020s0.020s0.000s0.000s0.000s
>>> 17MB17MB
>>>  00-xx-00 - SCREEN
>>> <
>> http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-0-0
>>> 
>>> Minor FragmentSetupProcessWaitMax BatchesMax RecordsPeak Mem 00-00-00
>>> 0.000s0.000s0.000s16-
>>>  00-xx-01 - PROJECT
>>> <
>> http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-0-1
>>> 
>>> Minor FragmentSetupProcessWaitMax BatchesMax RecordsPeak Mem 00-00-01
>>> 0.422s0.001s0.000s16-
>>>  00-xx-02 - INFO_SCHEMA_SUB_SCAN
>>> <
>> http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-0-2
>>> 
>>> Minor FragmentSetupProcessWaitMax BatchesMax RecordsPeak Mem
>>> 00-00-020.000s0.020s0.000s0017MB
>>> 
>>> Andries:  That storage plugin is very simple.  I've obfuscated some
>>> proprietary info.  (Note that the web UI was slow/hanging before I added
>>> the storage plugin.)
>>> 
>>> {
>>> "type": "file",
>>> "enabled": true,
>>> "connection": "file:///",
>>> "workspaces": {
>>>   "schema1": {
>>>     "location": "/mapr/theClusterName/user/theUser/test",
>>>     "writable": true,
>>>     "defaultInputFormat": null
>>>   }
>>> },
>>> "formats": {
>>>   "csv": {
>>>     "type": "text",
>>>     "extensions": [
>>>       "csv"
>>>     ],
>>>     "delimiter": ","
>>>   },
>>>   "tsv": {
>>>     "type": "text",
>>>     "extensions": [
>>>       "tsv"
>>>     ],
>>>     "delimiter": "\t"
>>>   },
>>>   "parquet": {
>>>     "type": "parquet"
>>>   }
>>> }
>>> }
>>> 
>>> Unfortunately, because the Web UI is having issues, it is presenting an
>>> unstyled page with no disable links for the storage plugin so I can't
>>> disable it. With the web UI not working, I don't know how to delete the
>>> plugin.
>>> 
>>> Abdel: There are no errors in the logs.
>>> 
>>> 
>>> On Mon, May 11, 2015 at 7:22 PM, Hanifi Gunes <hgunes@maprtech.com>
>> wrote:
>>> 
>>>> I would be interested in knowing where the time has been spent. Can you
>>>> inspect query profile from web ui for `show databases` and let us know
>> how
>>>> the profile looks like?
>>>> 
>>>> On Mon, May 11, 2015 at 4:10 PM, Abdel Hakim Deneche <
>>>> adeneche@maprtech.com>
>>>> wrote:
>>>> 
>>>>> any errors in the logs ?
>>>>> 
>>>>> On Mon, May 11, 2015 at 4:03 PM, Minnow Noir <minnownoir@gmail.com>
>>>> wrote:
>>>>> 
>>>>>> Yes.  The bit and server are idle, with no jobs of any sort running
on
>>>>>> them.
>>>>>> 
>>>>>> I just did started sqlline and it happened again.  I had to cancel
the
>>>>> show
>>>>>> databases command after 30s.
>>>>>> 
>>>>>> sqlline version 1.1.6
>>>>>> 0: jdbc:drill:zk=theServer:5181> show databases;
>>>>>> +-------------+
>>>>>> | SCHEMA_NAME |
>>>>>> +-------------+
>>>>>> | INFORMATION_SCHEMA |
>>>>>> | cp.default  |
>>>>>> | dfs.default |
>>>>>> | dfs.root    |
>>>>>> | dfs.tmp     |
>>>>>> | test.default |
>>>>>> | test.schema1 |
>>>>>> | sys         |
>>>>>> +-------------+
>>>>>> 8 rows selected (29.264 seconds)
>>>>>> 
>>>>>> On Mon, May 11, 2015 at 6:57 PM, Hanifi Gunes <hgunes@maprtech.com>
>>>>> wrote:
>>>>>> 
>>>>>>> Can you confirm if this happens even when the particular 0.9
bit is
>>>>> idle?
>>>>>>> 
>>>>>>> -Hanifi
>>>>>>> 
>>>>>>> On Mon, May 11, 2015 at 3:53 PM, Minnow Noir <minnownoir@gmail.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>>> Experiencing some slowness after upgrading one of our servers
from
>>>>> 0.8
>>>>>> to
>>>>>>>> 0.9.  In particular, despite restarting the Drillbit, and
even the
>>>>>>> server,
>>>>>>>> several times, the Drill web UI and sqlline are both so slow
as to
>>>> be
>>>>>>>> unusable.
>>>>>>>> 
>>>>>>>> For example, doing a show databases hangs indefinitely, until
I
>>>>> CTRL-C
>>>>>>> the
>>>>>>>> command (e.g., after a minute).
>>>>>>>> 
>>>>>>>> show databases;
>>>>>>>> +-------------+
>>>>>>>> | SCHEMA_NAME |
>>>>>>>> +-------------+
>>>>>>>> | INFORMATION_SCHEMA |
>>>>>>>> | cp.default  |
>>>>>>>> | dfs.default |
>>>>>>>> | dfs.root    |
>>>>>>>> | dfs.tmp     |
>>>>>>>> | test.default |
>>>>>>>> | test.schema1 |
>>>>>>>> | sys         |
>>>>>>>> +-------------+
>>>>>>>> 8 rows selected (59.044 seconds)
>>>>>>>> 
>>>>>>>> I tried changing schemas, but had to cancel after waiting
several
>>>>>>> minutes.
>>>>>>>> 
>>>>>>>> use test.schema1;
>>>>>>>> +------------+------------+
>>>>>>>> |     ok     |  summary   |
>>>>>>>> +------------+------------+
>>>>>>>> | true       | Default schema changed to 'test.schema1' |
>>>>>>>> +------------+------------+
>>>>>>>> 1 row selected (208.089 seconds)
>>>>>>>> 
>>>>>>>> There are no errors or obvious issues in the logs. The server
has
>>>>>> plenty
>>>>>>> of
>>>>>>>> disk, CPU, and RAM free.  It's running a recent version of
MAPR
>>>>> Hadoop
>>>>>> on
>>>>>>>> CentOS 6.5 on EC2.  It feels like something is hanging
>>>>>>>> 
>>>>>>>> Note: This is just after starting sqlline; no actual work,
which
>>>>> might
>>>>>>> have
>>>>>>>> slowed things down, has happened yet.
>>>>>>>> 
>>>>>>>> This is what's running from a Drill perspective:
>>>>>>>> 
>>>>>>>> ps aux | grep drill
>>>>>>>> ec2-user 24662  0.0  0.0 106100  1328 pts/0    S    12:29
  0:00
>>>> bash
>>>>>>>> bin/drillbit.sh internal_start drillbit
>>>>>>>> ec2-user 24697  0.3  2.3 6486240 730568 pts/0  Sl   12:29
  0:36
>>>>>>>> /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/bin/java
>>>>>>>> -Dlog.path=/opt/mapr/drill/drill-0.9.0/logs/drillbit.log
-Xms1G
>>>>> -Xmx4G
>>>>>>>> -XX:MaxDirectMemorySize=8G -XX:MaxPermSize=512M
>>>>>>>> -XX:ReservedCodeCacheSize=1G -ea
>>>>>>>> -Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf
>>>>>>>> -Dzookeeper.sasl.client=false -XX:+CMSClassUnloadingEnabled
>>>>>>>> -XX:+UseConcMarkSweepGC -cp
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>> /opt/mapr/drill/drill-0.9.0/conf:/opt/mapr/drill/drill-0.9.0/jars/*:/opt/mapr/drill/drill-0.9.0/jars/ext/*:/opt/mapr/drill/drill-0.9.0/jars/3rdparty/*:/opt/mapr/drill/drill-0.9.0/jars/classb/*
>>>>>>>> org.apache.drill.exec.server.Drillbit
>>>>>>>> 
>>>>>>>> Drill has been started thusly:
>>>>>>>> 
>>>>>>>> bin/sqlline -u jdbc:drill:zk=theServer:5181
>>>>>>>> 
>>>>>>>> Any idea how I could troubleshoot this?
>>>>>>>> 
>>>>>>>> Anyone else experiencing slowness after upgrading?
>>>>>>>> 
>>>>>>>> Thank you
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> 
>>>>> Abdelhakim Deneche
>>>>> 
>>>>> Software Engineer
>>>>> 
>>>>> <http://www.mapr.com/>
>>>>> 
>>>>> 
>>>>> Now Available - Free Hadoop On-Demand Training
>>>>> <
>>>>> 
>>>> 
>> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
>>>>>> 
>>>>> 
>>>> 
>> 
>> 


Mime
View raw message