spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russell Spitzer <>
Subject Re: Spark reading from cassandra
Date Wed, 04 Nov 2020 18:52:58 GMT
A where clause with a PK restriction should be identified by the Connector
and transformed into a single request. This should still be much slower
than doing the request directly but still much much faster than a full scan.

On Wed, Nov 4, 2020 at 12:51 PM Russell Spitzer <>

> Yes, the "Allow filtering" part isn't actually important other than for
> letting the query run in the first place. A where clause that utilizes a
> clustering column restriction will perform much better than a full scan,
> column pruning as well can be extremely beneficial.
> On Wed, Nov 4, 2020 at 11:12 AM Amit Sharma <> wrote:
>> Hi, i have a question while we are reading from cassandra should we use
>> partition key only in where clause from performance perspective or it does
>> not matter from spark perspective because it always allows filtering.
>> Thanks
>> Amit

View raw message