spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russell Spitzer <russell.spit...@gmail.com>
Subject Re: Spark reading from cassandra
Date Wed, 04 Nov 2020 18:52:58 GMT
A where clause with a PK restriction should be identified by the Connector
and transformed into a single request. This should still be much slower
than doing the request directly but still much much faster than a full scan.

On Wed, Nov 4, 2020 at 12:51 PM Russell Spitzer <russell.spitzer@gmail.com>
wrote:

> Yes, the "Allow filtering" part isn't actually important other than for
> letting the query run in the first place. A where clause that utilizes a
> clustering column restriction will perform much better than a full scan,
> column pruning as well can be extremely beneficial.
>
> On Wed, Nov 4, 2020 at 11:12 AM Amit Sharma <resolve123@gmail.com> wrote:
>
>> Hi, i have a question while we are reading from cassandra should we use
>> partition key only in where clause from performance perspective or it does
>> not matter from spark perspective because it always allows filtering.
>>
>>
>> Thanks
>> Amit
>>
>

Mime
View raw message