spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrés Ivaldi <iaiva...@gmail.com>
Subject Re: DataSet toJson
Date Tue, 08 Nov 2016 13:40:44 GMT
Ok, digging the code, I find out in the class JacksonGenerator the next
method

private def writeFields(
  row: InternalRow, schema: StructType, fieldWriters: Seq[ValueWriter]):
Unit = {
  var i = 0
  while (i < row.numFields) {
    val field = schema(i)
    if (!row.isNullAt(i)) {
      gen.writeFieldName(field.name)
      fieldWriters(i).apply(row, i)
    }
    i += 1
  }
}

So null values are directly ignored, I've to rewrite the method toJson to
use my own JacksonGenerator.

Regards.



On Tue, Nov 8, 2016 at 10:06 AM, Andrés Ivaldi <iaivaldi@gmail.com> wrote:

> Hello, I'm using spark 2.0 and I'm using toJson method. I've seen that
> Null values are omitted in the Json Record, witch is valid, but I need the
> field with null as value, it's possible to configure that?
>
> thanks.
>
>


-- 
Ing. Ivaldi Andres

Mime
View raw message