spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitry <frostb...@gmail.com>
Subject Distribution of spark 3.0.1 with Hive1.2
Date Tue, 10 Nov 2020 13:41:22 GMT
Hi all, I am trying to make distribution 3.0.1 with spark 3 using
./dev/make-distribution.sh --name spark3-hive12 --pip  --tgz  -Phive-1.2
-Phadoop-2.7 -Pyarn
The problem is maven can't found right profile for hive and build ends
without hive jars
++ /Users/reireirei/spark/spark/build/mvn help:evaluate
-Dexpression=project.activeProfiles -pl sql/hive -Phive-1.2 -Phadoop-2.7
-Pyarn
++ grep -v INFO
++ grep -v WARNING
++ fgrep --count '<id>hive</id>'
++ echo -n
+ SPARK_HIVE=0
+ '[' spark3-hive12 == none ']'
+ echo 'Spark version is 3.0.1'
Spark version is 3.0.1

What is the correct way to build in hive 1.2  jars in spark distribution?

Mime
View raw message