spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sandeep katta (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-23545) [Spark-Core] port opened by the SparkDriver is vulnerable for flooding attacks
Date Tue, 06 Mar 2018 05:20:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-23545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387307#comment-16387307
] 

sandeep katta commented on SPARK-23545:
---------------------------------------

I will be working on this bug,

Solution is as follows

1.Send the HeartBeat(1 way message) from APP master to Driver ,so Driver treats this channel
as active

2.Driver can close all the inactive channels

 

If any questions regarding this solution,please be free to comment on this

 

> [Spark-Core] port opened by the SparkDriver is vulnerable for flooding attacks
> ------------------------------------------------------------------------------
>
>                 Key: SPARK-23545
>                 URL: https://issues.apache.org/jira/browse/SPARK-23545
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 2.2.1
>            Reporter: sandeep katta
>            Priority: Major
>
> port opened by the SparkDriver is vulnerable for flooding attacks
> *Steps*:
> set spark.network.timeout=60s //can be any value
> Start the thriftserver in client mode and you can see in below logs that the spark Driver
opens the port for AM and executors to communicate.
> Logs:
> 018-03-01 16:11:16,497 | INFO  | [main] | Successfully started service *'sparkDriver'*
on port *22643*. | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
> 2018-03-01 16:11:17,265 | INFO  | [main] | Successfully started service 'SparkUI' on
port 22950. | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
> 2018-03-01 16:11:44,640 | INFO  | [main] | Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService'
on port 22663. | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54)
> 2018-03-01 16:11:52,822 | INFO  | [Thread-56] | Starting ThriftBinaryCLIService on port
22550 with 5...501 worker threads | org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.run(ThriftBinaryCLIService.java:111)
> Do telnet to this port using *telnet IP 22643* command and keep it idle, after 60 seconds
check the status, connection is still established, it should be terminated
> *lsof command output along with the date*
>  
> host1:/var/ # date
>  Thu Mar 1 *16:12:55* CST 2018
>  host1:/var/ # lsof | grep 22643
>  java 66730 user1 292u IPv6 1482635919 0t0 TCP host1:22643->*10.18.152.191:59297*
(ESTABLISHED)
>  java 66730 user1 297u IPv6 1482374122 0t0 TCP host1:22643->BLR1000018529:43894 (ESTABLISHED)
>  java 66730 user1 346u IPv6 1482314249 0t0 TCP host1:22643 (LISTEN)
>  host1:/var/ # date
>  Thu Mar 1 16:13:43 CST 2018
>  host1:/var/ # date
>  Thu Mar 1 *16:16:55* CST 2018
>  host1:/var/ # lsof | grep 22643
>  java 66730 user1 292u IPv6 1482635919 0t0 TCP host1:22643->*10.18.152.191:59297*
(ESTABLISHED)
>  java 66730 user1 297u IPv6 1482374122 0t0 TCP host1:22643->BLR1000018529:43894 (ESTABLISHED)
>  java 66730 user1 346u IPv6 1482314249 0t0 TCP host1:22643 (LISTEN)
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message