Hanifi: The web UI is extremely slow, and the style sheets aren't loading,
but here is what I see for "show databases"
Major FragmentMinor Fragments ReportingFirst StartLast StartFirst EndLast
Endtmintavgtmaxlast updatelast progressmemmax 00-xx-xx1 /
10.213s0.213s0.670s0.670s0.457s0.457s0.457s12:15:3012:15:303MB
Major Fragment: 00-xx-xx
<http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#fragment-0>
Minor FragmentHostStartEndTotal TimeMax RecordsMax BatchesLast UpdateLast
ProgressPeak MemoryState 00-00-xxip-10-101-197-197.ec2.internal0.213s0.670s
0.457s6112:15:3012:15:303MBFINISHED
Operator Profiles
Overview
<http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-overview>
OperatorTypeSetup (min)Setup (avg)Setup (max)Process (min)Process
(avg)Process
(max)Wait (min)Wait (avg)Wait (max)Mem (avg)Mem (max) 00-xx-00SCREEN0.000s
0.000s0.000s0.000s0.000s0.000s0.000s0.000s0.000s-- 00-xx-01PROJECT0.422s
0.422s0.422s0.001s0.001s0.001s0.000s0.000s0.000s-- 00-xx-02
INFO_SCHEMA_SUB_SCAN0.000s0.000s0.000s0.020s0.020s0.020s0.000s0.000s0.000s
17MB17MB
00-xx-00 - SCREEN
<http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-0-0>
Minor FragmentSetupProcessWaitMax BatchesMax RecordsPeak Mem 00-00-00
0.000s0.000s0.000s16-
00-xx-01 - PROJECT
<http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-0-1>
Minor FragmentSetupProcessWaitMax BatchesMax RecordsPeak Mem 00-00-01
0.422s0.001s0.000s16-
00-xx-02 - INFO_SCHEMA_SUB_SCAN
<http://174.129.247.50:8047/profiles/2aaf002d-0351-e983-b9a1-51da33885d87#operator-0-2>
Minor FragmentSetupProcessWaitMax BatchesMax RecordsPeak Mem
00-00-020.000s0.020s0.000s0017MB
Andries: That storage plugin is very simple. I've obfuscated some
proprietary info. (Note that the web UI was slow/hanging before I added
the storage plugin.)
{
"type": "file",
"enabled": true,
"connection": "file:///",
"workspaces": {
"schema1": {
"location": "/mapr/theClusterName/user/theUser/test",
"writable": true,
"defaultInputFormat": null
}
},
"formats": {
"csv": {
"type": "text",
"extensions": [
"csv"
],
"delimiter": ","
},
"tsv": {
"type": "text",
"extensions": [
"tsv"
],
"delimiter": "\t"
},
"parquet": {
"type": "parquet"
}
}
}
Unfortunately, because the Web UI is having issues, it is presenting an
unstyled page with no disable links for the storage plugin so I can't
disable it. With the web UI not working, I don't know how to delete the
plugin.
Abdel: There are no errors in the logs.
On Mon, May 11, 2015 at 7:22 PM, Hanifi Gunes <hgunes@maprtech.com> wrote:
> I would be interested in knowing where the time has been spent. Can you
> inspect query profile from web ui for `show databases` and let us know how
> the profile looks like?
>
> On Mon, May 11, 2015 at 4:10 PM, Abdel Hakim Deneche <
> adeneche@maprtech.com>
> wrote:
>
> > any errors in the logs ?
> >
> > On Mon, May 11, 2015 at 4:03 PM, Minnow Noir <minnownoir@gmail.com>
> wrote:
> >
> > > Yes. The bit and server are idle, with no jobs of any sort running on
> > > them.
> > >
> > > I just did started sqlline and it happened again. I had to cancel the
> > show
> > > databases command after 30s.
> > >
> > > sqlline version 1.1.6
> > > 0: jdbc:drill:zk=theServer:5181> show databases;
> > > +-------------+
> > > | SCHEMA_NAME |
> > > +-------------+
> > > | INFORMATION_SCHEMA |
> > > | cp.default |
> > > | dfs.default |
> > > | dfs.root |
> > > | dfs.tmp |
> > > | test.default |
> > > | test.schema1 |
> > > | sys |
> > > +-------------+
> > > 8 rows selected (29.264 seconds)
> > >
> > > On Mon, May 11, 2015 at 6:57 PM, Hanifi Gunes <hgunes@maprtech.com>
> > wrote:
> > >
> > > > Can you confirm if this happens even when the particular 0.9 bit is
> > idle?
> > > >
> > > > -Hanifi
> > > >
> > > > On Mon, May 11, 2015 at 3:53 PM, Minnow Noir <minnownoir@gmail.com>
> > > wrote:
> > > >
> > > > > Experiencing some slowness after upgrading one of our servers from
> > 0.8
> > > to
> > > > > 0.9. In particular, despite restarting the Drillbit, and even the
> > > > server,
> > > > > several times, the Drill web UI and sqlline are both so slow as to
> be
> > > > > unusable.
> > > > >
> > > > > For example, doing a show databases hangs indefinitely, until I
> > CTRL-C
> > > > the
> > > > > command (e.g., after a minute).
> > > > >
> > > > > show databases;
> > > > > +-------------+
> > > > > | SCHEMA_NAME |
> > > > > +-------------+
> > > > > | INFORMATION_SCHEMA |
> > > > > | cp.default |
> > > > > | dfs.default |
> > > > > | dfs.root |
> > > > > | dfs.tmp |
> > > > > | test.default |
> > > > > | test.schema1 |
> > > > > | sys |
> > > > > +-------------+
> > > > > 8 rows selected (59.044 seconds)
> > > > >
> > > > > I tried changing schemas, but had to cancel after waiting several
> > > > minutes.
> > > > >
> > > > > use test.schema1;
> > > > > +------------+------------+
> > > > > | ok | summary |
> > > > > +------------+------------+
> > > > > | true | Default schema changed to 'test.schema1' |
> > > > > +------------+------------+
> > > > > 1 row selected (208.089 seconds)
> > > > >
> > > > > There are no errors or obvious issues in the logs. The server has
> > > plenty
> > > > of
> > > > > disk, CPU, and RAM free. It's running a recent version of MAPR
> > Hadoop
> > > on
> > > > > CentOS 6.5 on EC2. It feels like something is hanging
> > > > >
> > > > > Note: This is just after starting sqlline; no actual work, which
> > might
> > > > have
> > > > > slowed things down, has happened yet.
> > > > >
> > > > > This is what's running from a Drill perspective:
> > > > >
> > > > > ps aux | grep drill
> > > > > ec2-user 24662 0.0 0.0 106100 1328 pts/0 S 12:29 0:00
> bash
> > > > > bin/drillbit.sh internal_start drillbit
> > > > > ec2-user 24697 0.3 2.3 6486240 730568 pts/0 Sl 12:29 0:36
> > > > > /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/bin/java
> > > > > -Dlog.path=/opt/mapr/drill/drill-0.9.0/logs/drillbit.log -Xms1G
> > -Xmx4G
> > > > > -XX:MaxDirectMemorySize=8G -XX:MaxPermSize=512M
> > > > > -XX:ReservedCodeCacheSize=1G -ea
> > > > > -Djava.security.auth.login.config=/opt/mapr/conf/mapr.login.conf
> > > > > -Dzookeeper.sasl.client=false -XX:+CMSClassUnloadingEnabled
> > > > > -XX:+UseConcMarkSweepGC -cp
> > > > >
> > > > >
> > > >
> > >
> >
> /opt/mapr/drill/drill-0.9.0/conf:/opt/mapr/drill/drill-0.9.0/jars/*:/opt/mapr/drill/drill-0.9.0/jars/ext/*:/opt/mapr/drill/drill-0.9.0/jars/3rdparty/*:/opt/mapr/drill/drill-0.9.0/jars/classb/*
> > > > > org.apache.drill.exec.server.Drillbit
> > > > >
> > > > > Drill has been started thusly:
> > > > >
> > > > > bin/sqlline -u jdbc:drill:zk=theServer:5181
> > > > >
> > > > > Any idea how I could troubleshoot this?
> > > > >
> > > > > Anyone else experiencing slowness after upgrading?
> > > > >
> > > > > Thank you
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> >
> > Abdelhakim Deneche
> >
> > Software Engineer
> >
> > <http://www.mapr.com/>
> >
> >
> > Now Available - Free Hadoop On-Demand Training
> > <
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > >
> >
>
|