Removing SSL from Spark's internal communications

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Removing SSL from Spark's internal communications

Rostyslav Sotnychenko
Hi all,

I am wondering what Community will say about the need of removing SSL in Spark's internal communication (File Server and Broadcast Server).

The problems I see are the following: 
1. Each user must have his own keystore/truststore to use for his jobs - sharing keystores is obviously unsecure at all. 
2. Both keystore and truststore must be present on local FS on each node in cluster

Lets say we have a large organization with hundreds of users and cluster with thousands of nodes.
The organization will be required to create and manage its own PKI, give the keys for each user and on every update distribute changes across all the nodes. Isn't this way too complicated? 


Would it be useful to replace SSL with something else, e.g. SASL that is already used in different parts of Spark or just plain AES? Or is there something that makes those changes impossible/non-appropriate?



Thanks,
Rostyslav
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Removing SSL from Spark's internal communications

Marcelo Vanzin
There's no file server anymore. And both the RPC endpoint (used to
transfer files) and the block manager (broadcasts + other blocks)
support encryption without SSL.

On Wed, Apr 19, 2017 at 8:55 AM, Rostyslav Sotnychenko
<[hidden email]> wrote:

> Hi all,
>
> I am wondering what Community will say about the need of removing SSL in
> Spark's internal communication (File Server and Broadcast Server).
>
> The problems I see are the following:
> 1. Each user must have his own keystore/truststore to use for his jobs -
> sharing keystores is obviously unsecure at all.
> 2. Both keystore and truststore must be present on local FS on each node in
> cluster
>
> Lets say we have a large organization with hundreds of users and cluster
> with thousands of nodes.
> The organization will be required to create and manage its own PKI, give the
> keys for each user and on every update distribute changes across all the
> nodes. Isn't this way too complicated?
>
>
> Would it be useful to replace SSL with something else, e.g. SASL that is
> already used in different parts of Spark or just plain AES? Or is there
> something that makes those changes impossible/non-appropriate?
>
>
>
> Thanks,
> Rostyslav



--
Marcelo

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Loading...