spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From melin li <libinsong1...@gmail.com>
Subject Spark 3.0 yarn does not support cdh5
Date Mon, 21 Oct 2019 07:02:32 GMT
很多集群还是使用cdh5,希望继续支持cdh5,cdh5是基于hadoop 2.6

dev/make-distribution.sh --tgz -Pkubernetes -Pyarn -Phive-thriftserver
-Phive -Dhadoop.version=2.6.0-cdh5.15.0 -DskipTest

```
[INFO] Compiling 25 Scala sources to
/Users/libinsong/Documents/codes/tongdun/spark-3.0/resource-managers/yarn/target/scala-2.12/classes
...
[ERROR] [Error]
/Users/libinsong/Documents/codes/tongdun/spark-3.0/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala:298:
value setRolledLogsIncludePattern is not a member of
org.apache.hadoop.yarn.api.records.LogAggregationContext
[ERROR] [Error]
/Users/libinsong/Documents/codes/tongdun/spark-3.0/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala:300:
value setRolledLogsExcludePattern is not a member of
org.apache.hadoop.yarn.api.records.LogAggregationContext
[ERROR] [Error]
/Users/libinsong/Documents/codes/tongdun/spark-3.0/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala:551:
not found: value isLocalUri
[ERROR] [Error]
/Users/libinsong/Documents/codes/tongdun/spark-3.0/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala:1367:
not found: value isLocalUri
[ERROR] four errors found
```

Mime
View raw message