whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrei Savu <savu.and...@gmail.com>
Subject Re: Dying because - net.schmizz.sshj.transport.TransportException: Broken transport; encountered EOF
Date Wed, 26 Oct 2011 09:16:33 GMT
I don't have a good answer. Sometimes ssh connections are dropped but if the
cluster is able to start everything should be fine. 0.7.0 will ship with
jclouds 1.2.1 which has improvements around handling ssh connections.

Another thing we've noticed while testing is that sometimes AMIs change
without notice and things start to break.

I suggest that you create some sort of smoke test for the cluster and if it
fails you can rebuild it from scratch.

Also take a look at the configuration guide page because there are some
parameters you can tweak:
http://whirr.apache.org/docs/0.6.0/configuration-guide.html

(e.g. probably a cluster with only 80% of the region servers is still good
enough)

Cheers,

-- Andrei Savu / andreisavu.ro

On Wed, Oct 26, 2011 at 11:56 AM, Paolo Castagna <
castagna.lists@googlemail.com> wrote:

> Hi,
> I am using Whirr version 0.6.0-incubating.
>
> This is my recipe for a small Hadoop cluster:
>
> -----
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker, 6
> hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=${env:AWS_ACCESS_KEY_ID}
> whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
> whirr.hardware-id=m1.large
> whirr.image-id=eu-west-1/ami-ee0e3c9a
> whirr.location-id=eu-west-1
> whirr.private-key-file=${sys:user.home}/.ssh/whirr
> whirr.public-key-file=${whirr.private-key-file}.pub
> whirr.hadoop.version=0.20.204.0
> whirr.hadoop.tarball.url=
> http://archive.apache.org/dist/hadoop/core/hadoop-${whirr.hadoop.version}/hadoop-${whirr.hadoop.version}.tar.gz
> -----
>
> When I start up the cluster everything seems fine, but I see this
> message very early on:
>
> ----
> Dying because - net.schmizz.sshj.transport.TransportException: Broken
> transport; encountered EOF
> Dying because - net.schmizz.sshj.transport.TransportException: Broken
> transport; encountered EOF
> <<kex done>> woke to: net.schmizz.sshj.transport.TransportException:
> Broken transport; encountered EOF
> << (ubuntu@46.137.70.148:22) error acquiring
> SSHClient(ubuntu@46.137.70.148:22): Broken transport; encountered EOF
> net.schmizz.sshj.transport.TransportException: Broken transport;
> encountered EOF
>        at net.schmizz.sshj.transport.Reader.run(Reader.java:70)
> ----
>
> I do not understand the problem (or if indeed, this is a problem).
>
> ... and I am missing compression. Could that be the reason? :-(
>
> Which AMI would you recommend for Amazon EC2 eu-west-1?
>
> Thank you very much in advance for your help (and thanks for Whirr! ;-))
> Paolo
>

Mime
View raw message