spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From YaoPau <>
Subject Error: Unrecognized option '--conf' (trying to set auto.offset.reset)
Date Fri, 21 Nov 2014 15:58:45 GMT
I'm trying to configure my Spark Streaming + Kafka job so it always pulls
real-time data.  I think setting auto.offset.reset="largest" will do it, but
when I try to set that configuration at runtime I get an error:

*spark-submit --class com.autotrader.scalaspark.sbStreaming --master
yarn-client --driver-memory 10g --executor-memory 10g --conf

Error: Unrecognized option '--conf'.*

How can I configure my job to pull the latest data?

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message