spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ayan guha <guha.a...@gmail.com>
Subject Re: How more than one spark job can write to same partition in the parquet file
Date Thu, 12 Dec 2019 06:36:41 GMT
We partitioned data logically for 2 different jobs...in our use case based
on geography...

On Thu, 12 Dec 2019 at 3:39 pm, Chetan Khatri <chetan.opensource@gmail.com>
wrote:

> Thanks, If you can share alternative change in design. I would love to
> hear from you.
>
> On Wed, Dec 11, 2019 at 9:34 PM ayan guha <guha.ayan@gmail.com> wrote:
>
>> No we faced problem with that setup.
>>
>> On Thu, 12 Dec 2019 at 11:14 am, Chetan Khatri <
>> chetan.opensource@gmail.com> wrote:
>>
>>> Hi Spark Users,
>>> would that be possible to write to same partition to the parquet file
>>> through concurrent two spark jobs with different spark session.
>>>
>>> thanks
>>>
>> --
>> Best Regards,
>> Ayan Guha
>>
> --
Best Regards,
Ayan Guha

Mime
View raw message