No, it is just being truncated for display as the ... implies. Pass
truncate=false to the show command.
On Sun, Aug 28, 2016, 15:24 Kevin Tran <kevintvh@gmail.com> wrote:
> Hi,
> I wrote to parquet file as following:
>
> +--------------------+
> | word|
> +--------------------+
> |THIS IS MY CHARACTERS ...|
> |// ANOTHER LINE OF CHAC...|
> +--------------------+
>
> These lines are not full text and it is being trimmed down.
> Does anyone know how many chacters StringType could handle ?
>
> In the Spark code:
> org.apache.spark.sql.types.StringType
> /**
> * The default size of a value of the StringType is 4096 bytes.
> */
> override def defaultSize: Int = 4096
>
>
> Thanks,
> Kevin.
>
|