spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject Re: How to run spark shell using YARN
Date Wed, 14 Mar 2018 18:37:57 GMT
Hi,

I am not using emr. And yes I restarted several times.

On Wed, Mar 14, 2018 at 6:35 AM, Anthony, Olufemi <
Olufemi.Anthony@capitalone.com> wrote:

> After you updated your yarn-site.xml  file, did you restart the YARN
> resource manager ?
>
>
>
> https://aws.amazon.com/premiumsupport/knowledge-
> center/restart-service-emr/
>
>
>
> Femi
>
>
>
> *From: *kant kodali <kanth909@gmail.com>
> *Date: *Wednesday, March 14, 2018 at 6:16 AM
> *To: *Femi Anthony <femibyte@gmail.com>
> *Cc: *vermanurag <anurag.verma@fnmathlogic.com>, "user @spark" <
> user@spark.apache.org>
> *Subject: *Re: How to run spark shell using YARN
>
>
>
> 16GB RAM.  AWS m4.xlarge. It's a three node cluster and I only have YARN
> and  HDFS running. Resources are barely used however I believe there is
> something in my config that is preventing YARN to see that I have good
> amount of resources I think (thats my guess I never worked with YARN
> before). My mapred-site.xml is empty. Do I even need this? if so, what
> should I set it to?
>
>
>
> On Wed, Mar 14, 2018 at 2:46 AM, Femi Anthony <femibyte@gmail.com> wrote:
>
> What's the hardware configuration of the box you're running on i.e. how
> much memory does it have ?
>
>
>
> Femi
>
>
>
> On Wed, Mar 14, 2018 at 5:32 AM, kant kodali <kanth909@gmail.com> wrote:
>
> Tried this
>
>
>
>  ./spark-shell --master yarn --deploy-mode client --executor-memory 4g
>
>
>
> Same issue. Keeps going forever..
>
>
>
> 18/03/14 09:31:25 INFO Client:
>
> client token: N/A
>
> diagnostics: N/A
>
> ApplicationMaster host: N/A
>
> ApplicationMaster RPC port: -1
>
> queue: default
>
> start time: 1521019884656
>
> final status: UNDEFINED
>
> tracking URL: http://ip-172-31-0-54:8088/proxy/application_
> 1521014458020_0004/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__ip-2D172-2D31-2D0-2D54-3A8088_proxy_application-5F1521014458020-5F0004_&d=DwMFaQ&c=pLULRYW__RtkwsQUPxJVDGboCTdgji3AcHNJU0BpTJE&r=yGeUxkUZBNPLfjlLWOxq59qm8G85KrtO5kZzZS4Mb6Mram0KPWstdXkCzdil9aYa&m=oOFBWIVhH_T4NwkrNL0SyXQhhsGyETZUeedvYi0AVd0&s=Eq5RVldAnerufWd7pgeydUZWtdXr2XJoEncqgUV5McE&e=>
>
> user: centos
>
>
>
> 18/03/14 09:30:08 INFO Client: Application report for
> application_1521014458020_0003 (state: ACCEPTED)
>
> 18/03/14 09:30:09 INFO Client: Application report for
> application_1521014458020_0003 (state: ACCEPTED)
>
> 18/03/14 09:30:10 INFO Client: Application report for
> application_1521014458020_0003 (state: ACCEPTED)
>
> 18/03/14 09:30:11 INFO Client: Application report for
> application_1521014458020_0003 (state: ACCEPTED)
>
> 18/03/14 09:30:12 INFO Client: Application report for
> application_1521014458020_0003 (state: ACCEPTED)
>
> 18/03/14 09:30:13 INFO Client: Application report for
> application_1521014458020_0003 (state: ACCEPTED)
>
> 18/03/14 09:30:14 INFO Client: Application report for
> application_1521014458020_0003 (state: ACCEPTED)
>
> 18/03/14 09:30:15 INFO Client: Application report for
> application_1521014458020_0003 (state: ACCEPTED)
>
>
>
> On Wed, Mar 14, 2018 at 2:03 AM, Femi Anthony <femibyte@gmail.com> wrote:
>
> Make sure you have enough memory allocated for Spark workers, try
> specifying executor memory as follows:
>
> --executor-memory <memory>
>
> to spark-submit.
>
>
>
> On Wed, Mar 14, 2018 at 3:25 AM, kant kodali <kanth909@gmail.com> wrote:
>
> I am using spark 2.3.0 and hadoop 2.7.3.
>
>
>
> Also I have done the following and restarted all. But I still
> see ACCEPTED: waiting for AM container to be allocated, launched and
> register with RM. And i am unable to spawn spark-shell.
>
>
>
> editing $HADOOP_HOME/etc/hadoop/capacity-scheduler.xml and change the
> following property value from 0.1 to something higher. I changed to 0.5
> (50%)
>
> <property>
>
>     <name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
>
>     <value>0.5</value>
>
>     <description>
>
>         Maximum percent of resources in the cluster which can be used to run application
masters i.e. controls number of concurrent running applications.
>
>     </description>
>
> </property>
>
> You may have to allocate more memory to YARN by editing yarn-site.xml by
> updating the following property:
>
> <property>
>
>     <name>yarn.nodemanager.resource.memory-mb</name>
>
>     <value>8192</value>
>
> </property>
>
> https://stackoverflow.com/questions/45687607/waiting-
> for-am-container-to-be-allocated-launched-and-register-with-rm
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__stackoverflow.com_questions_45687607_waiting-2Dfor-2Dam-2Dcontainer-2Dto-2Dbe-2Dallocated-2Dlaunched-2Dand-2Dregister-2Dwith-2Drm&d=DwMFaQ&c=pLULRYW__RtkwsQUPxJVDGboCTdgji3AcHNJU0BpTJE&r=yGeUxkUZBNPLfjlLWOxq59qm8G85KrtO5kZzZS4Mb6Mram0KPWstdXkCzdil9aYa&m=oOFBWIVhH_T4NwkrNL0SyXQhhsGyETZUeedvYi0AVd0&s=i8R5_ASmKyL_OccyAC0AtMDz7VWncp0UO27XuXBnfXs&e=>
>
>
>
>
>
>
>
> On Wed, Mar 14, 2018 at 12:12 AM, kant kodali <kanth909@gmail.com> wrote:
>
> any idea?
>
>
>
> On Wed, Mar 14, 2018 at 12:12 AM, kant kodali <kanth909@gmail.com> wrote:
>
> I set core-site.xml, hdfs-site.xml, yarn-site.xml  as per this website
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__dwbi.org_etl_bigdata_183-2Dsetup-2Dhadoop-2Dcluster&d=DwMFaQ&c=pLULRYW__RtkwsQUPxJVDGboCTdgji3AcHNJU0BpTJE&r=yGeUxkUZBNPLfjlLWOxq59qm8G85KrtO5kZzZS4Mb6Mram0KPWstdXkCzdil9aYa&m=oOFBWIVhH_T4NwkrNL0SyXQhhsGyETZUeedvYi0AVd0&s=hHisUoN8gj2IyS6_ZjQUvCOHUzfLc5jAAFftyskPWag&e=>
> and these are the only three files I changed Do I need to set or change
> anything in mapred-site.xml (As of now I have not
> touched mapred-site.xml)?
>
>
>
> when I do yarn -node -list -all I can see both node manager and resource
> managers are running fine.
>
>
>
> But when I run spark-shell --master yarn --deploy-mode client
>
>
>
>
>
> it just keeps looping forever and never stops with the following messages
>
>
>
> 18/03/14 07:07:47 INFO Client: Application report for
> application_1521011212550_0001 (state: ACCEPTED)
>
> 18/03/14 07:07:48 INFO Client: Application report for
> application_1521011212550_0001 (state: ACCEPTED)
>
> 18/03/14 07:07:49 INFO Client: Application report for
> application_1521011212550_0001 (state: ACCEPTED)
>
> 18/03/14 07:07:50 INFO Client: Application report for
> application_1521011212550_0001 (state: ACCEPTED)
>
> 18/03/14 07:07:51 INFO Client: Application report for
> application_1521011212550_0001 (state: ACCEPTED)
>
> 18/03/14 07:07:52 INFO Client: Application report for
> application_1521011212550_0001 (state: ACCEPTED)
>
>
>
> when I go to RM UI I see this
>
>
>
> ACCEPTED: waiting for AM container to be allocated, launched and register
> with RM.
>
>
>
>
>
>
>
>
>
> On Mon, Mar 12, 2018 at 7:16 PM, vermanurag <anurag.verma@fnmathlogic.com>
> wrote:
>
> This does not look like Spark error. Looks like yarn has not been able to
> allocate resources for spark driver. If you check resource manager UI you
> are likely to see this as spark application waiting for resources. Try
> reducing the driver node memory and/ or other bottlenecks based on what you
> see in the resource manager UI.
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dspark-2Duser-2Dlist.1001560.n3.nabble.com_&d=DwMFaQ&c=pLULRYW__RtkwsQUPxJVDGboCTdgji3AcHNJU0BpTJE&r=yGeUxkUZBNPLfjlLWOxq59qm8G85KrtO5kZzZS4Mb6Mram0KPWstdXkCzdil9aYa&m=oOFBWIVhH_T4NwkrNL0SyXQhhsGyETZUeedvYi0AVd0&s=g7By6jrvjF4WMSbJXbFkISCgGC7y3KhCmQjGov1Op60&e=>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>
>
>
>
>
>
>
>
>
>
> --
>
> http://www.femibyte.com/twiki5/bin/view/Tech/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.femibyte.com_twiki5_bin_view_Tech_&d=DwMFaQ&c=pLULRYW__RtkwsQUPxJVDGboCTdgji3AcHNJU0BpTJE&r=yGeUxkUZBNPLfjlLWOxq59qm8G85KrtO5kZzZS4Mb6Mram0KPWstdXkCzdil9aYa&m=oOFBWIVhH_T4NwkrNL0SyXQhhsGyETZUeedvYi0AVd0&s=VNsO5HYu4dkggm7IKXxlyliIsY7ruyup_KdC2JtoLAQ&e=>
>
> http://www.nextmatrix.com
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.nextmatrix.com&d=DwMFaQ&c=pLULRYW__RtkwsQUPxJVDGboCTdgji3AcHNJU0BpTJE&r=yGeUxkUZBNPLfjlLWOxq59qm8G85KrtO5kZzZS4Mb6Mram0KPWstdXkCzdil9aYa&m=oOFBWIVhH_T4NwkrNL0SyXQhhsGyETZUeedvYi0AVd0&s=3QEiubsB-BsOFzspftggKQwytEpZpI4b34Jc4qhrVSE&e=>
>
> "Great spirits have always encountered violent opposition from mediocre
> minds." - Albert Einstein.
>
>
>
>
>
>
>
> --
>
> http://www.femibyte.com/twiki5/bin/view/Tech/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.femibyte.com_twiki5_bin_view_Tech_&d=DwMFaQ&c=pLULRYW__RtkwsQUPxJVDGboCTdgji3AcHNJU0BpTJE&r=yGeUxkUZBNPLfjlLWOxq59qm8G85KrtO5kZzZS4Mb6Mram0KPWstdXkCzdil9aYa&m=oOFBWIVhH_T4NwkrNL0SyXQhhsGyETZUeedvYi0AVd0&s=VNsO5HYu4dkggm7IKXxlyliIsY7ruyup_KdC2JtoLAQ&e=>
>
> http://www.nextmatrix.com
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.nextmatrix.com&d=DwMFaQ&c=pLULRYW__RtkwsQUPxJVDGboCTdgji3AcHNJU0BpTJE&r=yGeUxkUZBNPLfjlLWOxq59qm8G85KrtO5kZzZS4Mb6Mram0KPWstdXkCzdil9aYa&m=oOFBWIVhH_T4NwkrNL0SyXQhhsGyETZUeedvYi0AVd0&s=3QEiubsB-BsOFzspftggKQwytEpZpI4b34Jc4qhrVSE&e=>
>
> "Great spirits have always encountered violent opposition from mediocre
> minds." - Albert Einstein.
>
>
>
> ------------------------------
>
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>

Mime
View raw message