spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From melin li <libinsong1...@gmail.com>
Subject Re: Spark 3.0 yarn does not support cdh5
Date Mon, 21 Oct 2019 07:03:31 GMT
Many clusters still use cdh5, and want to continue to support cdh5,cdh5
based on hadoop 2.6

melin li <libinsong1204@gmail.com> 于2019年10月21日周一 下午3:02写道:

> 很多集群还是使用cdh5,希望继续支持cdh5,cdh5是基于hadoop 2.6
>
> dev/make-distribution.sh --tgz -Pkubernetes -Pyarn -Phive-thriftserver
> -Phive -Dhadoop.version=2.6.0-cdh5.15.0 -DskipTest
>
> ```
> [INFO] Compiling 25 Scala sources to
> /Users/libinsong/Documents/codes/tongdun/spark-3.0/resource-managers/yarn/target/scala-2.12/classes
> ...
> [ERROR] [Error]
> /Users/libinsong/Documents/codes/tongdun/spark-3.0/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala:298:
> value setRolledLogsIncludePattern is not a member of
> org.apache.hadoop.yarn.api.records.LogAggregationContext
> [ERROR] [Error]
> /Users/libinsong/Documents/codes/tongdun/spark-3.0/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala:300:
> value setRolledLogsExcludePattern is not a member of
> org.apache.hadoop.yarn.api.records.LogAggregationContext
> [ERROR] [Error]
> /Users/libinsong/Documents/codes/tongdun/spark-3.0/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala:551:
> not found: value isLocalUri
> [ERROR] [Error]
> /Users/libinsong/Documents/codes/tongdun/spark-3.0/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala:1367:
> not found: value isLocalUri
> [ERROR] four errors found
> ```
>

Mime
View raw message