beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [beam] branch master updated: Warn on BigQuery Job failures
Date Fri, 11 Jun 2021 00:46:20 GMT
This is an automated email from the ASF dual-hosted git repository.

altay pushed a commit to branch master
in repository

The following commit(s) were added to refs/heads/master by this push:
     new 883db5c  Warn on BigQuery Job failures
     new e3b6468  Merge pull request #14722 from ryanmcdowell/patch-1
883db5c is described below

commit 883db5cdd466b45b1ae02686de8349a9a64806b9
Author: Ryan McDowell <>
AuthorDate: Tue May 4 12:30:55 2021 -0500

    Warn on BigQuery Job failures
    Currently if a BigQuery job were to fail because of a data issue, the failures are logged
at INFO level. This makes it challenging for users to debug because through the runner (e.g.
Dataflow) everything appears fine. Logging at WARN or ERROR will give users more indication
that something is wrong with the pipeline.
 .../main/java/org/apache/beam/sdk/io/gcp/bigquery/  | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/
index 13d84bb..c7cda81 100644
--- a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/
+++ b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/
@@ -251,7 +251,7 @@ public class BigQueryHelpers {
           case FAILED:
             String oldJobId = currentJobId.getJobId();
             currentJobId = BigQueryHelpers.getRetryJobId(currentJobId, lookupJob).jobId;
+            LOG.warn(
                 "Load job {} failed, {}: {}. Next job id {}",
                 shouldRetry() ? "will retry" : "will not retry",

View raw message