sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veena Basavaraj (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-1856) Sqoop2: Handling failures ( Row and Field level ) in Sqoop
Date Mon, 08 Dec 2014 17:26:13 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-1856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Veena Basavaraj updated SQOOP-1856:
-----------------------------------
    Description: 
Skipping corrupted rows in Sqoop 

What is the proposed strategy for handling such scenarios in batch transfer?
Probably one of the below ..
1. Skip/ignore and still continue for good records
2. just bail out once we have a bad record?
3. have a threshold of how many bad rows we can tolerate? that is configurable.


>From Anand Iyer

{quote}
Sqoop is the most obvious place for the functionality discussed in this thread. But at some
point, we should start think about adding similar functionality (Policy Driven SLAs and Data
Validation) ....

{quote}




  was:
Skipping corrupted rows in Sqoop 

What is the proposed strategy for handling such scenarios in batch transfer?
Probably one of the below ..
1. Skip/ignore and still continue for good records
2. just bail out once we have a bad record?
3. have a threshold of how many bad rows we can tolerate? that is configurable.


Anand Iyer
1:25 AM (7 hours ago)

Sqoop is the most obvious place for the functionality discussed in this thread. But at some
point, we should start think about adding similar functionality (Policy Driven SLAs and Data
Validation) across all our tools.





> Sqoop2: Handling failures ( Row and Field level ) in Sqoop
> ----------------------------------------------------------
>
>                 Key: SQOOP-1856
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1856
>             Project: Sqoop
>          Issue Type: Improvement
>            Reporter: Veena Basavaraj
>
> Skipping corrupted rows in Sqoop 
> What is the proposed strategy for handling such scenarios in batch transfer?
> Probably one of the below ..
> 1. Skip/ignore and still continue for good records
> 2. just bail out once we have a bad record?
> 3. have a threshold of how many bad rows we can tolerate? that is configurable.
> From Anand Iyer
> {quote}
> Sqoop is the most obvious place for the functionality discussed in this thread. But at
some point, we should start think about adding similar functionality (Policy Driven SLAs and
Data Validation) ....
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message