spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <>
Subject [KafkaSourceProvider] Why topic option and column without reverting to path as the least priority?
Date Mon, 01 May 2017 13:52:23 GMT

I've just found out that KafkaSourceProvider supports topic option
that sets the Kafka topic to save a DataFrame to.

You can also use topic column to assign rows to topics.

Given the features, I've been wondering why "path" option is not
supported (even of least precedence) so when no topic column or option
are defined, save(path: String) would be the least priority.


It looks pretty trivial to support --> see KafkaSourceProvider at
lines [1] and [2] if I'm not mistaken.


Jacek Laskowski
Mastering Apache Spark 2
Follow me at

To unsubscribe e-mail:

View raw message