flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink] sjwiesman commented on issue #10979: [FLINK-15814][connectors] Log warning when StreamingFileSink is used with bulk output formats and an ambiguous S3 scheme
Date Fri, 31 Jan 2020 14:18:00 GMT
sjwiesman commented on issue #10979: [FLINK-15814][connectors] Log warning when StreamingFileSink
is used with bulk output formats and an ambiguous S3 scheme
URL: https://github.com/apache/flink/pull/10979#issuecomment-580751185
 
 
   Hey @kl0u 
   
   I will generalize this for all output types. 
   
   > One point I can find is that the checkpointed above will throw an exception at the
TaskManager while yours will fail at the client, which is more desirable. The reason we did
not put it there is because the StreamingFileSink is shared across all FileSystems and we
did not want to put FS specific code in it. But if this was the reason for the change, then
this PR should change to throw the exception in any case.
   
   I agree with the intention and think this was the correct original choice. But I have been
working with a large number of users that have messed this up and I think in this case it
is more important to be pragmatic and throw / Log a clear message. 
   
   If it's ok with you, I will update this PR to throw on 's3p' and log on 's3' for both row
and bulk formats. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message