falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shwetha G S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-697) Messaging improvements
Date Wed, 10 Sep 2014 06:56:28 GMT

    [ https://issues.apache.org/jira/browse/FALCON-697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14128155#comment-14128155

Shwetha G S commented on FALCON-697:

    private String[] getFeedPaths() throws IOException {

        if (isFalconEntityTopic()) {
            LOG.debug("Returning instance paths for Falcon Topic: " + context.getOutputFeedInstancePaths());
            return new String[]{context.getOutputFeedInstancePaths(), };
   //read from instances.csv as list of paths

So, if its falcon topic, we always read instances from context.getOutputFeedInstancePaths().
context.getOutputFeedInstancePaths() is empty for retention. So, we don't send retention jms
message to falcon topic.

In case of retention jms message for user topic, we read the paths from instances.csv and
getFeedPaths() returns n paths and buildMessageList() builds message for every path. So, we
end up sending 1 message per path for all user messages. This applies to replication and process
as well

I verified this from post processing logs as well

> Messaging improvements
> ----------------------
>                 Key: FALCON-697
>                 URL: https://issues.apache.org/jira/browse/FALCON-697
>             Project: Falcon
>          Issue Type: Improvement
>            Reporter: Shwetha G S
> 1. In eviction, we send 1 message for every path deleted to user topic. We can send 1
message with list of paths instead. I guess this applies to replication and process as well
> 2. In eviction, we don't send any jms message to falcon topic. Some listeners may be
interested in this as well

This message was sent by Atlassian JIRA

View raw message