spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel de Oliveira Mantovani <daniel.oliveira.mantov...@gmail.com>
Subject Re: How to submit a job via REST API?
Date Fri, 18 Dec 2020 17:55:37 GMT
In my opinion this should be part of the official documentation. Amazing
work Zhou Yang.

On Wed, Nov 25, 2020 at 5:45 AM Zhou Yang <zhouyang.zy@outlook.com> wrote:

> Hi all,
>
> I found the solution through the source code. Appending the —conf k-v into
> `sparkProperties` work.
> For example:
>
> ./spark-submit \
> —conf foo=bar \
> xxx
>
> equals to
>
> {
> “xxx” : “yyy”,
> “sparkProperties” : {
> “foo": "bar"
> }
> }
>
> Thanks for your reply.
>
> 2020年11月25日 下午3:55,vaquar khan <vaquar.khan@gmail.com> 写道:
>
> Hi Yang,
>
> Please find following link
>
>
> https://stackoverflow.com/questions/63677736/spark-application-as-a-rest-service/63678337#63678337
>
> Regards,
> Vaquar khan
>
> On Wed, Nov 25, 2020 at 12:40 AM Sonal Goyal <sonalgoyal4@gmail.com>
> wrote:
>
>> You should be able to supply the --conf and its values as part of appArgs
>> argument
>>
>> Cheers,
>> Sonal
>> Nube Technologies <http://www.nubetech.co/>
>> Join me at
>> Data Con LA Oct 23 | Big Data Conference Europe. Nov 24 | GIDS AI/ML Dec
>> 3
>>
>>
>>
>>
>> On Tue, Nov 24, 2020 at 11:31 AM Dennis Suhari <
>> d.suhari@icloud.com.invalid> wrote:
>>
>>> Hi Yang,
>>>
>>> I am using Livy Server for submitting jobs.
>>>
>>> Br,
>>>
>>> Dennis
>>>
>>>
>>>
>>> Von meinem iPhone gesendet
>>>
>>> Am 24.11.2020 um 03:34 schrieb Zhou Yang <zhouyang.zy@outlook.com>:
>>>
>>> 
>>> Dear experts,
>>>
>>> I found a convenient way to submit job via Rest API at
>>> https://gist.github.com/arturmkrtchyan/5d8559b2911ac951d34a#file-submit_job-sh
>>> .
>>> But I did not know whether can I append `—conf` parameter like what I
>>> did in spark-submit. Can someone can help me with this issue?
>>>
>>> *Regards, Yang*
>>>
>>>
>
> --
> Regards,
> Vaquar Khan
> +1 -224-436-0783
> Greater Chicago
>
>
>

-- 

--
Daniel Mantovani

Mime
View raw message