spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Tran <kevin...@gmail.com>
Subject Spark StringType could hold how many characters ?
Date Sun, 28 Aug 2016 14:24:16 GMT 
Hi,
I wrote to parquet file as following:

+--------------------+
|                word|
+--------------------+
|THIS IS MY CHARACTERS ...|
|// ANOTHER LINE OF CHAC...|
+--------------------+

These lines are not full text and it is being trimmed down.
Does anyone know how many chacters StringType could handle ?

In the Spark code:
org.apache.spark.sql.types.StringType
/**
   * The default size of a value of the StringType is 4096 bytes.
   */
  override def defaultSize: Int = 4096


Thanks,
Kevin.
Mime
View raw message