falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Johannes Mayer (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (FALCON-2302) Same partition gets exported over and over
Date Fri, 11 Aug 2017 11:41:00 GMT

    [ https://issues.apache.org/jira/browse/FALCON-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16123224#comment-16123224
] 

Johannes Mayer edited comment on FALCON-2302 at 8/11/17 11:40 AM:
------------------------------------------------------------------

I think i found the issue: in the coordinator.xml the input-event instance is hardcoded:

{code:java}
    <input-events>
        <data-in name="export-input" dataset="export-dataset">
            <instance>2017-05-17T10:30Z</instance>
        </data-in>
    </input-events>
{code}

where it should be something like: 
${coord:current(0)}

Please refer to: 
[Oozie Documentation - CoordinatorSpec|https://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a6.8.5_coord:dataInPartitionMinString_name_String_partition_EL_function],
see the [example|https://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#HCatPigExampleTwo]



was (Author: joha0123):
I think i found the issue: in the coordinator.xml the input-event instance is hardcoded:

{code:java}
    <input-events>
        <data-in name="export-input" dataset="export-dataset">
            <instance>2017-05-17T10:30Z</instance>
        </data-in>
    </input-events>
{code}

where it should be something like: 
${coord:current(0)}

Please refer to: 
[Oozie Documentation - CoordinatorSpec|https://oozie.apache.org/docs/4.2.0/CoordinatorFunctionalSpec.html#a6.8.5_coord:dataInPartitionMinString_name_String_partition_EL_function]




> Same partition gets exported over and over
> ------------------------------------------
>
>                 Key: FALCON-2302
>                 URL: https://issues.apache.org/jira/browse/FALCON-2302
>             Project: Falcon
>          Issue Type: Bug
>    Affects Versions: 0.10
>         Environment: Hortonworks Data Platform 2.6.1
>            Reporter: Johannes Mayer
>            Priority: Critical
>         Attachments: coordinator.xml, oozie_job_configuration.xml, OracleFeed.xml, workflow.xml
>
>
> I want to export a HCat/Hive table to Oracle. The table is partitioned by year month
and day. As exprected, for everyday a oozie workflow is generated. but when I look at the
> {code:java}
> sqoopCommand
> {code}
> I can see *it always exports the same partition* (partition of feed start).
> The expected behaviour is, that for every day partition a workflow should be generated
that exports that given partition.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message