spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nasrulla Khan Haris <Nasrulla.K...@microsoft.com.INVALID>
Subject RE: adding shutdownmanagerhook to spark.
Date Tue, 14 May 2019 01:04:41 GMT
Thanks Sean

Nasrulla

-----Original Message-----
From: Sean Owen <srowen@gmail.com> 
Sent: Monday, May 13, 2019 4:16 PM
To: Nasrulla Khan Haris <Nasrulla.Khan@microsoft.com>
Cc: dev@spark.apache.org
Subject: Re: adding shutdownmanagerhook to spark.

Spark just adds a hook to the mechanism that Hadoop exposes. You can do the same. You shouldn't
use Spark's.

On Mon, May 13, 2019 at 6:11 PM Nasrulla Khan Haris <Nasrulla.Khan@microsoft.com.invalid>
wrote:
>
> HI All,
>
>
>
> I am trying to add shutdown hook, but looks like shutdown manager object requires the
package to be spark only, is there any other API that can help me to do this ?
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> ub.com%2Fapache%2Fspark%2Fblob%2Fv2.4.0%2Fcore%2Fsrc%2Fmain%2Fscala%2F
> org%2Fapache%2Fspark%2Futil%2FShutdownHookManager.scala&amp;data=01%7C
> 01%7CNasrulla.Khan%40microsoft.com%7C8aadef9c8cf844139eca08d6d7f90058%
> 7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=Si4C7iF8PN5%2FJDDQy8F
> BzzuFh2XsYD%2BfjMtT5wLxbr4%3D&amp;reserved=0
>
>
>
> I can see that hiveserver2 in https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fspark%2Fcommit%2F515708d5f33d5acdb4206c626192d1838f8e691f&amp;data=01%7C01%7CNasrulla.Khan%40microsoft.com%7C8aadef9c8cf844139eca08d6d7f90058%7C72f988bf86f141af91ab2d7cd011db47%7C1&amp;sdata=AGNpXhauHp4Wkbe%2BVesAbO5H%2BJFhwApMCRcAi2oROl0%3D&amp;reserved=0
uses ShutdownHookManager in different package. But It expects me to have the package name
“org.apache.spark”
>
>
>
> Thanks,
>
> Nasrulla
>
>

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org

Mime
View raw message