spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ruslan Dautkhanov <>
Subject df.dtypes -> pyspark.sql.types
Date Wed, 16 Mar 2016 16:18:38 GMT

Looking at

and can't wrap my head around how to convert string data types names to
pyspark.sql.types data types?

Does pyspark.sql.types has an interface to return StringType() for "string",
IntegerType() for "integer" etc? If it doesn't exist it would be great to
have such a
mapping function.

Thank you.

ps. I have a data frame, and use its dtypes to loop through all columns to
fix a few
columns' data types as a workaround for SPARK-13866.

Ruslan Dautkhanov

View raw message