spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject Re: How to run spark shell using YARN
Date Wed, 14 Mar 2018 10:15:31 GMT
16GB RAM.  AWS m4.xlarge. It's a three node cluster and I only have YARN
and  HDFS running. Resources are barely used however I believe there is
something in my config that is preventing YARN to see that I have good
amount of resources I think (thats my guess I never worked with YARN
before). My mapred-site.xml is empty. Do I even need this? if so, what
should I set it to?

On Wed, Mar 14, 2018 at 2:46 AM, Femi Anthony <femibyte@gmail.com> wrote:

> What's the hardware configuration of the box you're running on i.e. how
> much memory does it have ?
>
> Femi
>
> On Wed, Mar 14, 2018 at 5:32 AM, kant kodali <kanth909@gmail.com> wrote:
>
>> Tried this
>>
>>  ./spark-shell --master yarn --deploy-mode client --executor-memory 4g
>>
>>
>> Same issue. Keeps going forever..
>>
>>
>> 18/03/14 09:31:25 INFO Client:
>>
>> client token: N/A
>>
>> diagnostics: N/A
>>
>> ApplicationMaster host: N/A
>>
>> ApplicationMaster RPC port: -1
>>
>> queue: default
>>
>> start time: 1521019884656
>>
>> final status: UNDEFINED
>>
>> tracking URL: http://ip-172-31-0-54:8088/proxy/application_1521014458020_
>> 0004/
>>
>> user: centos
>>
>>
>> 18/03/14 09:30:08 INFO Client: Application report for
>> application_1521014458020_0003 (state: ACCEPTED)
>>
>> 18/03/14 09:30:09 INFO Client: Application report for
>> application_1521014458020_0003 (state: ACCEPTED)
>>
>> 18/03/14 09:30:10 INFO Client: Application report for
>> application_1521014458020_0003 (state: ACCEPTED)
>>
>> 18/03/14 09:30:11 INFO Client: Application report for
>> application_1521014458020_0003 (state: ACCEPTED)
>>
>> 18/03/14 09:30:12 INFO Client: Application report for
>> application_1521014458020_0003 (state: ACCEPTED)
>>
>> 18/03/14 09:30:13 INFO Client: Application report for
>> application_1521014458020_0003 (state: ACCEPTED)
>>
>> 18/03/14 09:30:14 INFO Client: Application report for
>> application_1521014458020_0003 (state: ACCEPTED)
>>
>> 18/03/14 09:30:15 INFO Client: Application report for
>> application_1521014458020_0003 (state: ACCEPTED)
>>
>> On Wed, Mar 14, 2018 at 2:03 AM, Femi Anthony <femibyte@gmail.com> wrote:
>>
>>> Make sure you have enough memory allocated for Spark workers, try
>>> specifying executor memory as follows:
>>>
>>> --executor-memory <memory>
>>>
>>> to spark-submit.
>>>
>>> On Wed, Mar 14, 2018 at 3:25 AM, kant kodali <kanth909@gmail.com> wrote:
>>>
>>>> I am using spark 2.3.0 and hadoop 2.7.3.
>>>>
>>>> Also I have done the following and restarted all. But I still
>>>> see ACCEPTED: waiting for AM container to be allocated, launched and
>>>> register with RM. And i am unable to spawn spark-shell.
>>>>
>>>> editing $HADOOP_HOME/etc/hadoop/capacity-scheduler.xml and change the
>>>> following property value from 0.1 to something higher. I changed to 0.5
>>>> (50%)
>>>>
>>>> <property>
>>>>     <name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
>>>>     <value>0.5</value>
>>>>     <description>
>>>>         Maximum percent of resources in the cluster which can be used to
run application masters i.e. controls number of concurrent running applications.
>>>>     </description>
>>>> </property>
>>>>
>>>> You may have to allocate more memory to YARN by editing yarn-site.xml
>>>> by updating the following property:
>>>>
>>>> <property>
>>>>     <name>yarn.nodemanager.resource.memory-mb</name>
>>>>     <value>8192</value>
>>>> </property>
>>>>
>>>> https://stackoverflow.com/questions/45687607/waiting-for-am-
>>>> container-to-be-allocated-launched-and-register-with-rm
>>>>
>>>>
>>>>
>>>> On Wed, Mar 14, 2018 at 12:12 AM, kant kodali <kanth909@gmail.com>
>>>> wrote:
>>>>
>>>>> any idea?
>>>>>
>>>>> On Wed, Mar 14, 2018 at 12:12 AM, kant kodali <kanth909@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I set core-site.xml, hdfs-site.xml, yarn-site.xml  as per this
>>>>>> website <https://dwbi.org/etl/bigdata/183-setup-hadoop-cluster>
and
>>>>>> these are the only three files I changed Do I need to set or change
>>>>>> anything in mapred-site.xml (As of now I have not touched mapred-site.xml)?
>>>>>>
>>>>>> when I do yarn -node -list -all I can see both node manager and
>>>>>> resource managers are running fine.
>>>>>>
>>>>>> But when I run spark-shell --master yarn --deploy-mode client
>>>>>>
>>>>>>
>>>>>> it just keeps looping forever and never stops with the following
>>>>>> messages
>>>>>>
>>>>>> 18/03/14 07:07:47 INFO Client: Application report for
>>>>>> application_1521011212550_0001 (state: ACCEPTED)
>>>>>> 18/03/14 07:07:48 INFO Client: Application report for
>>>>>> application_1521011212550_0001 (state: ACCEPTED)
>>>>>> 18/03/14 07:07:49 INFO Client: Application report for
>>>>>> application_1521011212550_0001 (state: ACCEPTED)
>>>>>> 18/03/14 07:07:50 INFO Client: Application report for
>>>>>> application_1521011212550_0001 (state: ACCEPTED)
>>>>>> 18/03/14 07:07:51 INFO Client: Application report for
>>>>>> application_1521011212550_0001 (state: ACCEPTED)
>>>>>> 18/03/14 07:07:52 INFO Client: Application report for
>>>>>> application_1521011212550_0001 (state: ACCEPTED)
>>>>>>
>>>>>> when I go to RM UI I see this
>>>>>>
>>>>>> ACCEPTED: waiting for AM container to be allocated, launched and
>>>>>> register with RM.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Mar 12, 2018 at 7:16 PM, vermanurag <
>>>>>> anurag.verma@fnmathlogic.com> wrote:
>>>>>>
>>>>>>> This does not look like Spark error. Looks like yarn has not
been
>>>>>>> able to
>>>>>>> allocate resources for spark driver. If you check resource manager
>>>>>>> UI you
>>>>>>> are likely to see this as spark application waiting for resources.
>>>>>>> Try
>>>>>>> reducing the driver node memory and/ or other bottlenecks based
on
>>>>>>> what you
>>>>>>> see in the resource manager UI.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>>>>>>>
>>>>>>> ------------------------------------------------------------
>>>>>>> ---------
>>>>>>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> http://www.femibyte.com/twiki5/bin/view/Tech/
>>> http://www.nextmatrix.com
>>> "Great spirits have always encountered violent opposition from mediocre
>>> minds." - Albert Einstein.
>>>
>>
>>
>
>
> --
> http://www.femibyte.com/twiki5/bin/view/Tech/
> http://www.nextmatrix.com
> "Great spirits have always encountered violent opposition from mediocre
> minds." - Albert Einstein.
>

Mime
View raw message