Hi:

I am working on a realtime application using spark structured streaming (v 2.2.1). The application reads data from kafka and if there is a failure, I would like to ignore the checkpoint.  Is there any configuration to just read from last kafka offset after a failure and ignore any offset checkpoints ?

Also, I believe that the checkpoint also saves state and will continue to aggregations after recovery.  Is there any way to ignore checkpointed state ?

Also, is there a way to selectively save state or offset checkpoint only ?

Thanks