spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobias Pfeiffer <...@preferred.jp>
Subject Re: Spark sql results can't be printed out to system console from spark streaming application
Date Wed, 04 Mar 2015 01:13:27 GMT
Hi,

can you explain how you copied that into your *streaming* application?
Like, how do you issue the SQL, what data do you operate on, how do you
view the logs etc.?

Tobias

On Wed, Mar 4, 2015 at 8:55 AM, Cui Lin <cui.lin@hds.com> wrote:

>
>
> >Dear all,
> >
> >I found the below sample code can be printed out only in spark shell, but
> >when I moved them into my spark streaming application, nothing can be
> >printed out into system console. Can you explain why it happened? anything
> >related to new spark context? Thanks a lot!
> >
> >
> >val anotherPeopleRDD = sc_context.parallelize(
> >  """{"name":"Yin","address":{"city":"Columbus","state":"Ohio"}}""" ::
> >Nil)
> >
> >anotherPeopleRDD.toArray().foreach(line => System.out.println(line))
> >
> >val jsonMessage = sqlContext.jsonRDD(anotherPeopleRDD)
> >
> >jsonMessage.toArray().foreach(line => System.out.println(line))
> >
> >jsonMessage.registerTempTable("people")
> >
> >val test: SchemaRDD = sqlContext.sql("select count(*) from people")
> >
> >test.toArray().foreach(line => System.out.println(line))
> >
> >
> >
> >
> >Best regards,
> >
> >Cui Lin
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message