spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From doki_pen <>
Subject Saving compressed textFiles from a DStream in Scala
Date Fri, 05 Jun 2015 15:40:27 GMT
It looks like saveAsTextFiles doesn't support the compression parameter of
RDD.saveAsTextFile. Is there a way to add the functionality in my client
code without patching Spark? I tried making my own saveFunc function and
calling DStream.foreachRDD but ran into trouble with invoking rddToFileName
and making the RDD type parameter work properly. It's probably just do to my
lack of Scala knowledge. Can anyone give me a hand?

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message