| Message view | « Date » · « Thread » |
|---|---|
| Top | « Date » · « Thread » |
| From | Pralabh Kumar <pralabhku...@gmail.com> |
| Subject | org.apache.spark.shuffle.FetchFailedException: Too large frame: |
| Date | Tue, 01 May 2018 11:21:01 GMT |
Hi I am getting the above error in Spark SQL . I have increase (using 5000 ) number of partitions but still getting the same error . My data most probably is skew. org.apache.spark.shuffle.FetchFailedException: Too large frame: 4247124829 at org.apache.spark.storage.ShuffleBlockFetcherIterator.throwFetchFailedException(ShuffleBlockFetcherIterator.scala:419) at org.apache.spark.storage.ShuffleBlockFetcherIterator.next(ShuffleBlockFetcherIterator.scala:349) | |
| Mime |
|
| View raw message | |