spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From A Shaikh <shaikh.af...@gmail.com>
Subject Re: Saving from Dataset to Bigquery Table
Date Fri, 20 Jan 2017 12:24:29 GMT
Thanks for responding Jorn. Currently I upload the jar to Google Cloud and
run my job not ideal for development. Do you know if we can run this from
within our local machine? given that all the required jars are downloaded
by SBT anyways.

On 20 January 2017 at 11:22, Jörn Franke <jornfranke@gmail.com> wrote:

> It is only on pairdd
>
> On 20 Jan 2017, at 11:54, A Shaikh <shaikh.afzal@gmail.com> wrote:
>
> Has anyone experience saving Dataset to Bigquery Table?
>
> I am loading into BigQuery using the following example
> <https://cloud.google.com/hadoop/examples/bigquery-connector-spark-example>sucessfully.
> This uses RDD.saveAsNewAPIHadoopDataset method to save data.
> I am using Dataset(or DataFrame) and looking for saveAsNewAPIHadoopDataset
> method but unable to find it.
>
>

Mime
View raw message