spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shyla deshpande <deshpandesh...@gmail.com>
Subject Re: saveToCassandra issue. Please help
Date Fri, 03 Feb 2017 18:39:58 GMT
Thanks Fernando.  But I need to have only 1 row for a given user, date with
very low latency. So none of your options work for me.



On Fri, Feb 3, 2017 at 10:34 AM, Fernando Avalos <gavalosf@gmail.com> wrote:

> Hi Shyla,
>
> Maybe I am wrong, but I can see two options here.
>
> 1.- Do some grouping before insert to Cassandra.
> 2.- Insert to cassandra all the entries and add some logic to your
> request to get the most recent.
>
> Regards,
>
> 2017-02-03 10:26 GMT-08:00 shyla deshpande <deshpandeshyla@gmail.com>:
> > Hi All,
> >
> > I wanted to add more info ..
> > The first column is the user and the third is the period. and my key is
> > (userid, date) For a given user and date combination I want to see only 1
> > row. My problem is that PT0H10M0S is overwritten by PT0H9M30S, even
> though
> > the order of the rows in the RDD is PT0H9M30S and then PT0H10M0S.
> >
> > Appreciate your input. Thanks
> >
> > On Fri, Feb 3, 2017 at 12:45 AM, shyla deshpande <
> deshpandeshyla@gmail.com>
> > wrote:
> >>
> >> Hello All,
> >>
> >> This is the content of my RDD which I am saving to Cassandra table.
> >>
> >> But looks like the 2nd row is written first and then the first row
> >> overwrites it. So I end up with bad output.
> >>
> >> (494bce4f393b474980290b8d1b6ebef9, 2017-02-01, PT0H9M30S, WEDNESDAY)
> >> (494bce4f393b474980290b8d1b6ebef9, 2017-02-01, PT0H10M0S, WEDNESDAY)
> >>
> >> Is there a way to force the order of the rows written to Cassandra.
> >>
> >> Please help.
> >>
> >> Thanks
> >
> >
>

Mime
View raw message