From user-return-1220-apmail-hama-user-archive=hama.apache.org@hama.apache.org Fri Feb 14 19:49:25 2014 Return-Path: X-Original-To: apmail-hama-user-archive@www.apache.org Delivered-To: apmail-hama-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5010D106E7 for ; Fri, 14 Feb 2014 19:49:25 +0000 (UTC) Received: (qmail 72184 invoked by uid 500); 14 Feb 2014 19:49:24 -0000 Delivered-To: apmail-hama-user-archive@hama.apache.org Received: (qmail 72126 invoked by uid 500); 14 Feb 2014 19:49:22 -0000 Mailing-List: contact user-help@hama.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hama.apache.org Delivered-To: mailing list user@hama.apache.org Received: (qmail 72109 invoked by uid 99); 14 Feb 2014 19:49:21 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Feb 2014 19:49:21 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [155.207.1.51] (HELO hermes8.ccf.auth.gr) (155.207.1.51) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Feb 2014 19:49:14 +0000 Received: from [192.168.1.5] (dsl-aauwti.dyn.edudsl.gr [37.32.179.166]) (authenticated bits=0) by hermes8.ccf.auth.gr (8.14.3/8.14.3/Debian-9.1ubuntu1) with ESMTP id s1EJmqsO009145 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Fri, 14 Feb 2014 21:48:53 +0200 Message-ID: <52FE731C.5040006@csd.auth.gr> Date: Fri, 14 Feb 2014 21:48:44 +0200 From: =?UTF-8?B?zpfOu86vzrHPgiDOms6xz4DOv8+Fz4HOrM69zrfPgg==?= User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: user@hama.apache.org Subject: Re: Groomservers IPC server port range References: <52F22A5B.8090605@csd.auth.gr> <52F238E3.8010700@csd.auth.gr> <52F346A7.800@csd.auth.gr> <52F36481.7090300@csd.auth.gr> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Scanned: clamav-milter 0.97.8 at hermes8 X-Virus-Status: Clean X-Virus-Checked: Checked by ClamAV on apache.org Hello there, I've seen that you resolved the HAMA-789. So, is it going to be available in a new version? How do we define the specific port we want? Στις 6/2/2014 1:07 μμ, ο/η Edward J. Yoon έγραψε: > I'll fix soon. > > I'm reopen https://issues.apache.org/jira/browse/HAMA-789 > > On Thu, Feb 6, 2014 at 7:31 PM, Ηλίας Καπουράνης wrote: >> Another error came up! Now it starts zookeepers but loses the connection. >> Specifically: >> >> attempt_201402061212_0001_000004_0: 14/02/06 12:12:58 INFO ipc.Server: >> Starting Socket Reader #1 for port 49564 >> attempt_201402061212_0001_000004_0: 14/02/06 12:12:58 INFO ipc.Server: IPC >> Server Responder: starting >> attempt_201402061212_0001_000004_0: 14/02/06 12:12:58 INFO ipc.Server: IPC >> Server listener on 49564: starting >> attempt_201402061212_0001_000004_0: 14/02/06 12:12:58 INFO >> message.HadoopMessageManagerImpl: BSPPeer address:host port:49564 >> attempt_201402061212_0001_000004_0: 14/02/06 12:12:58 INFO ipc.Server: IPC >> Server handler 0 on 49564: starting >> attempt_201402061212_0001_000004_0: 14/02/06 12:12:59 INFO >> sync.ZKSyncClient: Initializing ZK Sync Client >> attempt_201402061212_0001_000004_0: 14/02/06 12:12:59 INFO >> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At >> host/address:49564 >> attempt_201402061212_0001_000004_0: 14/02/06 12:12:59 ERROR >> sync.ZooKeeperSyncClientImpl: >> org.apache.zookeeper.KeeperException$ConnectionLossException: >> KeeperErrorCode = ConnectionLoss for /bsp/job_201402061212_0001/peers >> attempt_201402061212_0001_000004_0: 14/02/06 12:13:02 ERROR >> sync.ZKSyncClient: Error checking zk path >> /bsp/job_201402061212_0001/peers/host:49564 >> attempt_201402061212_0001_000004_0: >> org.apache.zookeeper.KeeperException$ConnectionLossException: >> KeeperErrorCode = ConnectionLoss for >> /bsp/job_201402061212_0001/peers/host:49564 >> >> >> So I think that the problem again is the port restriction. The 49564 port is >> randomly selected again. I tried seting it at the zookeeper quorum servers >> like host:port1:port2 but still it chooses randomly. Is there any way to >> "standarize" it like with groomserver? >> >> >> >> >> Στις 6/2/2014 10:24 πμ, ο/η Ηλίας Καπουράνης έγραψε: >> >>> Thanks a lot Edward! I will try it and let you know if something comes up! >>> >>> Have a nice day! >>> >>> >>> Στις 6/2/2014 7:15 πμ, ο/η Edward J. Yoon έγραψε: >>>> Again, >>>> >>>> You can specify the report server port by setting >>>> "bsp.groom.report.address" property like below: >>>> >>>> >>>> bsp.groom.report.address >>>> 127.0.0.1:5001 >>>> >>>> >>>> Default value was 127.0.0.1:0. (random port). >>>> >>>> >>>> >>>> On Thu, Feb 6, 2014 at 2:09 PM, Edward J. Yoon >>>> wrote: >>>>> Oh, .. thanks. >>>>> >>>>> The GroomServer.taskReportServer seems uses random port. I'll make it >>>>> configurable. >>>>> >>>>> On Wed, Feb 5, 2014 at 10:13 PM, Ηλίας Καπουράνης >>>>> wrote: >>>>>> Hey, >>>>>> >>>>>> So what the IPC Server stands for and it gets random ports? I ve seen >>>>>> 39865 >>>>>> and 52765. What is the deal with it? >>>>>> >>>>>> >>>>>> Στις 5/2/2014 2:53 μμ, ο/η Edward J. Yoon έγραψε: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> a Hama cluster uses ports "bsp.master.port" 40000 and >>>>>>> "bsp.groom.rpc.port" 50000 to communicate among the cluster nodes. >>>>>>> >>>>>>> And the default peer port is 61000. If you set 3 for the maximum >>>>>>> number of tasks per node, each task uses ports from 61000 to 61003 to >>>>>>> transfer messages between bsp tasks. >>>>>>> >>>>>>> On Wed, Feb 5, 2014 at 9:11 PM, Ηλίας Καπουράνης >>>>>>> >>>>>>> wrote: >>>>>>>> Hello there, >>>>>>>> >>>>>>>> I am trying to setup Hama in a cluster with really strict port >>>>>>>> permissions >>>>>>>> and I get across the error at every GroomServer: >>>>>>>> "There is a problem in establishing link with BSPMaster." >>>>>>>> >>>>>>>> - Do you know what is the range of ports that every GroomServers IPC >>>>>>>> Server >>>>>>>> gets when started? >>>>>>>> - Is there a way to limit this range to new user values? >>>>>>>> >>>>>>>> Thank you. >>>>>>> >>>>>>> >>>>> >>>>> -- >>>>> Best Regards, Edward J. Yoon >>>>> @eddieyoon >>>> >>>> > >