falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peeyush Bishnoi <bpeey...@yahoo.co.in>
Subject Re: Falcon cluster mirroring
Date Tue, 29 Sep 2015 09:21:28 GMT
Hi Mihai,
Looking to the error which you have got, it looks like that Falcon could not able to validate
the execute services of BigInsights. 
To mirror the data from BigInsights (source) to HDP sandbox(target), you can try to use Falcon
HDFS replication recipe. To use Falcon HDFS replication recipe, run it on target i.e HDP
sandbox.1. Define cluster entity with services from HDP sandbox (target) and submit on target.2.
In hdfs-replication.properties of recipe , specify source HDFS address with input data path
from BigInsights and target HDFS address with output data path. Also in properties, specify
that you want to run recipe on target for attribute "falcon.recipe.cluster.name".
For more information on usage of HDFS replication recipe, please refer following doc:http://falcon.apache.org/Recipes.html 

I hope this will help you to mirror your data.

Please let us know, if you have any questions.

     On Tuesday, 29 September 2015 7:37 AM, Mihai Popa <MPopa@nextpathway.com> wrote:


I've been trying to setup mirroring between two clusters.

One runs BigInsights 3.0.2, the other Hortonworks 2.3.

I'm running Falcon on the Hortonworks cluster (which is actually just a sandbox).

I have no trouble defining the target cluster (on the localhost), but cannot define the source
cluster (on the remote BigInsights install).

The problem is with the "execute" interface; I've tried a series of ports starting with 9000
which is the default namenode, 9001, 50030.

<interface type='execute' endpoint='bivm.ibm.com:50030' version='2.2.0'/>

(I do have a DNS entry for bivm.ibm.com)

The error that I get is:

org.apache.falcon.entity.parser.ValidationException: Invalid Execute server or port: bivm.ibm.com:50030

It seems like this version of BIgInsights does not come with Yarn, and from what I see, the
execute interface tries to connect to yarn's port.

Any insight (pun intended) is greatly appreciated.

thank you in advance

mihai popa

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message