spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Corey Nolet <cjno...@gmail.com>
Subject Re: Replacing Jetty with TomCat
Date Wed, 18 Feb 2015 04:29:43 GMT
Niranda,

I'm not sure if I'd say Spark's use of Jetty to expose its UI monitoring
layer constitutes a use of "two web servers in a single product". Hadoop
uses Jetty as well as do many other applications today that need embedded
http layers for serving up their monitoring UI to users. This is completely
aside from any web container an application developer would use to interact
with Spark and Hadoop and service domain-specific content to users. The two
are disjoint.

Many applications use Thrift as a means of establishing socket connections
between clients and across servers. One alternative to Thrift is Protobuf.
You wouldn't say "I want to swap out thrift for protobuf in Cassandra
because I want to use protobuf in my application and there shouldn't be two
different socket layer abstractions on my cluster."

I could understand wanting to do this if you were being forced to deploy a
war file to a web container in order to do the monitoring but Spark's UI is
embedded within the code. If you are worried about having the Jetty
libraries on your classpath, you can exclude the Jetty dependencies from
your servlet code if you want to interact with a SparkContext in Tomcat.



On Tue, Feb 17, 2015 at 10:22 PM, Patrick Wendell <pwendell@gmail.com>
wrote:

> Hey Niranda,
>
> It seems to me a lot of effort to support multiple libraries inside of
> Spark like this, so I'm not sure that's a great solution.
>
> If you are building an application that embeds Spark, is it not
> possible for you to continue to use Jetty for Spark's internal servers
> and use tomcat for your own server's? I would guess that many complex
> applications end up embedding multiple server libraries in various
> places (Spark itself has different transport mechanisms, etc.)
>
> - Patrick
>
> On Tue, Feb 17, 2015 at 7:14 PM, Niranda Perera
> <niranda.perera@gmail.com> wrote:
> > Hi Sean,
> > The main issue we have is, running two web servers in a single product.
> we
> > think it would not be an elegant solution.
> >
> > Could you please point me to the main areas where jetty server is tightly
> > coupled or extension points where I could plug tomcat instead of jetty?
> > If successful I could contribute it to the spark project. :-)
> >
> > cheers
> >
> >
> >
> > On Mon, Feb 16, 2015 at 4:51 PM, Sean Owen <sowen@cloudera.com> wrote:
> >
> >> There's no particular reason you have to remove the embedded Jetty
> >> server, right? it doesn't prevent you from using it inside another app
> >> that happens to run in Tomcat. You won't be able to switch it out
> >> without rewriting a fair bit of code, no, but you don't need to.
> >>
> >> On Mon, Feb 16, 2015 at 5:08 AM, Niranda Perera
> >> <niranda.perera@gmail.com> wrote:
> >> > Hi,
> >> >
> >> > We are thinking of integrating Spark server inside a product. Our
> current
> >> > product uses Tomcat as its webserver.
> >> >
> >> > Is it possible to switch the Jetty webserver in Spark to Tomcat
> >> > off-the-shelf?
> >> >
> >> > Cheers
> >> >
> >> > --
> >> > Niranda
> >>
> >
> >
> >
> > --
> > Niranda
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message