Would it make sense to add an optional validate parameter to applySchema() which defaults to False, both to give users the option to check the schema immediately and to make the default behavior clearer?

‚Äč

On Sat Feb 14 2015 at 9:18:59 AM Michael Armbrust <michael@databricks.com> wrote:
Doing runtime type checking is very expensive, so we only do it when necessary (i.e. you perform an operation like adding two columns together)

On Sat, Feb 14, 2015 at 2:19 AM, nitin <nitin2goyal@gmail.com> wrote:
AFAIK, this is the expected behavior. You have to make sure that the schema
matches the row. It won't give any error when you apply the schema as it
doesn't validate the nature of data.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/SQLContext-applySchema-strictness-tp21650p21653.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org