drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nawfal osman <nawfal.os...@corporate-groupe.com>
Subject RE: Soucis Drill
Date Tue, 28 Sep 2021 16:11:20 GMT
Hi

Please find atatched the json profile of the query


Have you made any changes to Drill memory options such as planner.memory.max_query_memory_per_node?

Answer : yes  we use   this parameter sometimes we ran queries  with the parameter '3' and
some other query with  '28'


Kinds Regards :
________________________________
De : James Turton <dzamo@apache.org>
Envoyé : mardi 28 septembre 2021 14:52
À : user@drill.apache.org <user@drill.apache.org>; nawfal osman <nawfal.osman@corporate-groupe.com>
Cc : Zakaria Rahoui <zakaria.rahoui@corporate-groupe.com>; Amine Elmekki <amine.elmekki@corporate-groupe.com>
Objet : Re: Soucis Drill

Plenty of direct memory there, 50Gb on every Drillbit, far in excess of the modest 134Mb that
Drill says it can't get.  Can you share a JSON profile from an execution of this query (Web
UI -> Profiles -> Completed queries -> (choose query) -> Full JSON Profile?

Have you made any changes to Drill memory options such as planner.memory.max_query_memory_per_node?

On 2021/09/28 13:55, nawfal osman wrote:
Hi

Please notice  we  have  10 nodes (mapr11-mapr15 )  ( mapr31-35)

please find below the output of the command : free -h of the node mapr15 :
[mapr@mapr15 conf]$ free -h
              total        used        free      shared  buff/cache   available
Mem:           125G         97G        3.1G        1.3G         25G         24G
Swap:          117G         54G



Find below the output of the command svc :
[mapr@mapr15 conf]$ maprcli node list -columns svc
2021-09-28 12:42:30,6130 INFO:Unable to set F_SETPIPE_SZ  vmsplice 406 err = 1   10.148.0.73:7222


hostname  service                                                                        
                                                                                         
              ip
mapr11    cldb,data-access-gateway,mastgateway,nodemanager,kibana,drill-bits,collectd,hoststats,fluentd,fileserver,nfs,gateway,apiserver
                                                         10.148.0.44,10.32.243.44
mapr12    data-access-gateway,fileserver,elasticsearch,mastgateway,nodemanager,kibana,drill-bits,collectd,hoststats,fluentd,grafana,resourcemanager,historyserver,nfs,opentsdb,gateway,apiserver
 10.148.0.45,10.32.243.45
mapr13    httpfs,hs2,data-access-gateway,fileserver,mastgateway,nodemanager,kibana,hcat,drill-bits,hoststats,fluentd,hivemeta,grafana,hue,nfs,gateway,apiserver
                                  10.148.0.46,10.32.243.46
mapr14    cldb,mastgateway,nodemanager,drill-bits,hbasethrift,oozie,collectd,hoststats,fluentd,hbaserest,fileserver,nfs,apiserver
                                                                10.148.0.47
mapr15    fileserver,mastgateway,nodemanager,drill-bits,collectd,hoststats,fluentd,hbaserest,nfs,apiserver
                                                                                       10.148.0.48,10.32.243.48
mapr31    fileserver,mastgateway,nodemanager,drill-bits,collectd,hoststats,fluentd,hbaserest,nfs,apiserver
                                                                                       10.148.0.70
mapr32    fileserver,mastgateway,nodemanager,drill-bits,collectd,hoststats,fluentd,hbaserest,nfs,apiserver
                                                                                       10.148.0.71
mapr33    fileserver,mastgateway,nodemanager,drill-bits,collectd,hoststats,fluentd,hbaserest,nfs,apiserver
                                                                                       10.148.0.72
mapr34    cldb,mastgateway,nodemanager,drill-bits,collectd,hoststats,fluentd,hbaserest,fileserver,nfs,apiserver
                                                                                  10.148.0.73
mapr35    data-access-gateway,fileserver,mastgateway,nodemanager,drill-bits,collectd,hoststats,fluentd,hbaserest,nfs,apiserver


For the memory setting of drill ; please find  attached the conf of all nodes



Kind Regards;


Mime
View raw message