spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thijs Haarhuis <>
Subject Re: SparkR + binary type + how to get value
Date Thu, 14 Feb 2019 12:01:24 GMT
Hi Felix,

I have the following code:


      firstRow <- first(results)
      value <- firstRow$value

      cat(paste0("Value Type: '",typeof(value),"'\n\n\n"))
      cat(paste0("Value: '",value,"'\n\n\n"))

results is a Spark Data Frame here.

When I run this code the following is printed to console:


You can there is only a single column in this sdf of type binary
when I collect this value and print the type it prints it is a list.

Any idea how to get the actual value, or how to process the individual bytes?


From: Felix Cheung <>
Sent: Thursday, February 14, 2019 5:31 AM
To: Thijs Haarhuis;
Subject: Re: SparkR + binary type + how to get value

Please share your code

From: Thijs Haarhuis <>
Sent: Wednesday, February 13, 2019 6:09 AM
Subject: SparkR + binary type + how to get value

Hi all,

Does anybody have any experience in accessing the data from a column which has a binary type
in a Spark Data Frame in R?

I have a Spark Data Frame which has a column which is of a binary type. I want to access this
data and process it.

In my case I collect the spark data frame to a R data frame and access the first row.

When I print this row to the console it does print all the hex values correctly.

However when I access the column it prints it is a list of 1 …when I print the type of the
child again prints it is a list.

I expected this value to be of a raw type.

Anybody has some experience with this?



View raw message