spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lec ssmi <shicheng31...@gmail.com>
Subject Re: how to disable replace HDFS checkpoint location in structured streaming in spark3.0.1
Date Wed, 14 Oct 2020 01:29:24 GMT
sorry, the mail title  is a little problematic. "How to disable or
replace ......"

lec ssmi <shicheng31604@gmail.com> 于2020年10月14日周三 上午9:27写道:

> I have written a demo using spark3.0.0, and the location where the
> checkpoint file  is saved has been explicitly specified   like
>>
>>     stream.option("checkpointLocation","file:///C:\\Users\\Administrator
>> \\Desktop\\test")
>
> But the app still throws an   exception about the HDFS file system.
> Is it not possible to specify the local file system as a checkpoint
> location now?
>

Mime
View raw message