spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yang Jie (Jira)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-34309) Use Caffeine instead of Guava Cache
Date Fri, 05 Feb 2021 10:21:01 GMT

    [ https://issues.apache.org/jira/browse/SPARK-34309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279548#comment-17279548
] 

Yang Jie edited comment on SPARK-34309 at 2/5/21, 10:20 AM:
------------------------------------------------------------

[~dongjoon] I use the code in [GetPutBenchmark|https://github.com/ben-manes/caffeine/blob/master/caffeine/src/jmh/java/com/github/benmanes/caffeine/cache/GetPutBenchmark.java]
to compare the performance of caffeine, guava cache, LinkedHashMap and ConcurrentLinkedHashMap,
the result as follows:

** read write thrpt test (6 threads read and 2 threads write):*

||cacheType||Score(Units)||
|LinkedHashMap_Lru|11248568.197 ± 12737170.642  ops/s|
|Caffeine|44638046.442 ± 23455184.501  ops/s|
|ConcurrentLinkedHashMap|30133621.108 ± 21890181.254  ops/s|
|Guava|14867777.653 ±  5716763.921  ops/s|

*  *read only thrpt test(8 threads read):*

||cacheType||Score(Units)||
|LinkedHashMap_Lru|10976457.064 ± 26703667.288  ops/s|
|Caffeine|62507109.307 ± 72055321.581  ops/s|
|ConcurrentLinkedHashMap|29978890.384 ± 22266401.711  ops/s|
|Guava|15102099.448 ±  6613818.000  ops/s|

*  *write only thrpt test(8 threads write):*

||cacheType||Score(Units)||
|LinkedHashMap_Lru|9776881.873 ± 25659900.079  ops/s|
|Caffeine|27899813.470 ± 11399461.937  ops/s|
|ConcurrentLinkedHashMap|15839475.472 ± 18312421.561  ops/s|
|Guava|7675161.183 ±  6730863.169  ops/s|

*{color:red}It seems that the performance of caffeine is better than that of guava cache{color}*




was (Author: luciferyang):
[~dongjoon] I use the code in [GetPutBenchmark|https://github.com/ben-manes/caffeine/blob/master/caffeine/src/jmh/java/com/github/benmanes/caffeine/cache/GetPutBenchmark.java]
to compare the performance of caffeine, guava cache, LinkedHashMap and ConcurrentLinkedHashMap,
the result as follows:

** read write thrpt test (6 threads read and 2 threads write):*

||cacheType||Score(Units)||
|LinkedHashMap_Lru|11248568.197 ± 12737170.642  ops/s|
|Caffeine|44638046.442 ± 23455184.501  ops/s|
|ConcurrentLinkedHashMap|30133621.108 ± 21890181.254  ops/s|
|Guava|14867777.653 ±  5716763.921  ops/s|

*  *read only thrpt test(8 threads read):*

||cacheType||Score(Units)||
|LinkedHashMap_Lru|10976457.064 ± 26703667.288  ops/s|
|Caffeine|62507109.307 ± 72055321.581  ops/s|
|ConcurrentLinkedHashMap|29978890.384 ± 22266401.711  ops/s|
|Guava|15102099.448 ±  6613818.000  ops/s|

*  *write only thrpt test(8 threads write):*

||cacheType||Score(Units)||
|LinkedHashMap_Lru|9776881.873 ± 25659900.079  ops/s|
|Caffeine|27899813.470 ± 11399461.937  ops/s|
|ConcurrentLinkedHashMap|15839475.472 ± 18312421.561  ops/s|
|Guava|7675161.183 ±  6730863.169  ops/s|

It seems that the performance of caffeine is better than that of guava cache



> Use Caffeine instead of Guava Cache
> -----------------------------------
>
>                 Key: SPARK-34309
>                 URL: https://issues.apache.org/jira/browse/SPARK-34309
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core, SQL
>    Affects Versions: 3.2.0
>            Reporter: Yang Jie
>            Priority: Minor
>         Attachments: image-2021-02-05-18-08-48-852.png, screenshot-1.png
>
>
> Caffeine is a high performance, near optimal caching library based on Java 8, it is
used in a similar way to guava cache, but with better performance. The comparison results
are as follow are on the [caffeine benchmarks |https://github.com/ben-manes/caffeine/wiki/Benchmarks]
> At the same time, caffeine has been used in some open source projects like Cassandra,
Hbase, Neo4j, Druid, Spring and so on.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message