spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd <>
Subject Does spark support Apache Arrow
Date Fri, 20 May 2016 02:16:17 GMT
From the official site, Apache Arrow is used for Columnar In-Memory
storage. I have two quick questions:
1. Does spark support Apache Arrow?
2. When dataframe is cached in memory, the data are saved in columnar in-memory style. What
is the relationship between this feature and Apache Arrow,that is,
when the data is in Apache Arrow format´╝îdoes spark still need the effort to cache the dataframe
in columnar in-memory?
View raw message