Hello Dev Users,

I am struggling to parallelize JDBC Read in Spark, It is using 1 - 2 task only to read data and taking so much of time to read.

Ex.

val invoiceLineItemDF = ((spark.read.jdbc(url = t360jdbcURL,
table = invoiceLineItemQuery,
columnName = "INVOICE_LINE_ITEM_ID",
lowerBound = 1L,
upperBound = 1000000L,
numPartitions = 200,
connectionProperties = connectionProperties
)))

Thanks