spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Martishius <>
Subject [PySpark] how to use a JAR model to score a dataset?
Date Fri, 13 Aug 2021 12:11:34 GMT

This question has been addressed on Stack Overflow using the spark shell,
but not PySpark.

I found within the Spark SQL documentation where in PySpark SQL I can load
a JAR into my SparkSession config such as:

*spark = SparkSession\*
*    .builder\*
*    .appName("appname")\*
*    .config("spark.jars","location_of_model/MLMODEL.jar")\*
*    .getOrCreate()*

The above code works fine but I can't figure out how to actually use
MLMODEL.jar to score a dataset.  Can anyone help with this?  Thank you!

View raw message