spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Priority queue in spark
Date Tue, 17 Mar 2015 09:30:35 GMT

What was the JIRA that Alessandro opened ?

Thanks


> On Mar 16, 2015, at 10:34 PM, Mark Hamstra <mark@clearstorydata.com> wrote:
> 
> http://apache-spark-developers-list.1001551.n3.nabble.com/Job-priority-td10076.html#a10079
> 
>> On Mon, Mar 16, 2015 at 10:26 PM, abhi <abhishekhar@gmail.com> wrote:
>> If i understand correctly , the above document creates pool for priority which is
static in nature and has to be defined before submitting the job . .in my scenario each generated
task can have different priority.
>> 
>> Thanks,
>> Abhi
>> 
>> 
>>> On Mon, Mar 16, 2015 at 9:48 PM, twinkle sachdeva <twinkle.sachdeva@gmail.com>
wrote:
>>> Hi,
>>> 
>>> Maybe this is what you are looking for : http://spark.apache.org/docs/1.2.0/job-scheduling.html#fair-scheduler-pools
>>> 
>>> Thanks,
>>> 
>>>> On Mon, Mar 16, 2015 at 8:15 PM, abhi <abhishekhar@gmail.com> wrote:
>>>> Hi 
>>>> Current all the jobs in spark gets submitted using queue . i have a requirement
where submitted job will generate another set of jobs with some priority , which should again
be submitted to spark cluster based on priority ? Means job with higher priority should be
executed first,    Is it feasible  ? 
>>>> 
>>>> Any help is appreciated ?
>>>> 
>>>> Thanks,
>>>> Abhi
> 

Mime
View raw message