spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mich Talebzadeh <>
Subject getting error: value toDF is not a member of Seq[columns]
Date Wed, 05 Sep 2018 08:11:32 GMT

I have spark streaming that send data and I need to put that data into
MongoDB for test purposes. The easiest way is to create a DF from the
individual list of columns as below

I loop over individual rows in RDD and perform the following

    case class columns(KEY: String, TICKER: String, TIMEISSUED: String,
PRICE: Float)

         for(line <- pricesRDD.collect.toArray)
            var key = line._2.split(',').view(0).toString
           var ticker =  line._2.split(',').view(1).toString
           var timeissued = line._2.split(',').view(2).toString
           var price = line._2.split(',').view(3).toFloat
           val priceToString = line._2.split(',').view(3)
           if (price > 90.0)
             println ("price > 90.0, saving to MongoDB collection!")
            // Save prices to mongoDB collection
           * var df = Seq(columns(key, ticker, timeissued, price)).toDF*

but it fails with message

 value toDF is not a member of Seq[columns].

What would be the easiest way of resolving this please?


Dr Mich Talebzadeh

LinkedIn *

*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.

View raw message