spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From buntu <>
Subject Re: spark-shell -- running into ArrayIndexOutOfBoundsException
Date Wed, 23 Jul 2014 17:29:33 GMT
Just wanted to add more info.. I was using SparkSQL reading in the
tab-delimited raw data files converting the timestamp to Date format:

  sc.textFile("rawdata/*").map(_.split("\t")).map(p => Point(df.format(new
Date( p(0).trim.toLong*1000L )), p(1), p(2).trim.toInt ,p(3).trim.toInt,
p(4).trim.toInt ,p(5)))

Then I go about registering it as table and when I run simple query like
select count(*) from 
, I get the ArrayIndexOutOfBoundsException.

I bumped up the SPARK_DRIVER_MEMORY to 8g but still didn't help: 

Let me know if I'm missing any steps.. thanks!

View this message in context:
Sent from the Apache Spark User List mailing list archive at

View raw message