Spark History UI + Keycloak Integration

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Spark History UI + Keycloak Integration

G, Ajay (Nokia - IN/Bangalore)

Hello,

 

We were trying to enable spark-history UI authentication through keycloak. From the spark documentation we found out that we can use javax filters to enable the UI authentication. Keycloak already provides a java keycloak-servlet-filter-adapter which can be used.

 

I have added the following configuration in spark-defaults.conf

 

spark.ui.filters org.keycloak.adapters.servlet.KeycloakOIDCFilter

spark.org.keycloak.adapters.servlet.KeycloakOIDCFilter.param.keycloak.config.file /home/ag/spark-2.4.0-bin-2.7.3/conf/keycloak.json

 

I was facing the below issue while running

 

java.lang.IllegalStateException: No SessionManager

               at org.spark_project.jetty.server.Request.getSession(Request.java:1544)

               at org.keycloak.adapters.servlet.FilterSessionStore.saveRequest(FilterSessionStore.java:374)

 

 

This was because none of the ServletContext in spark-history has sessionManangment. I have made the below changes

  1. Added Session id manager in JettyUtils.scala

               server.setSessionIdManager(new HashSessionIdManager())

 

  1. Added session handler for all context -   contextHandler.setSessionHandler(new SessionHandler())

in

    1. JettyUtils.scala – at createServletHandler, createStaticHandler and createProxyHandler
    2. HistoryServer.scala – at initialize for /history context
    3. ApiRootResource.scala – at getServletHandler for /api context.

 

  1. Placed required Keycloak runtime jars in spark class-path.

 

 

Keycloak authentication seems to work, Is this the right approach ? If it is fine I can submit a PR.

@Vanzin I saw that you have done some refactoring in spark UI code, in https://github.com/apache/spark/pull/23302 can you please suggest some inputs.

 

Thanks and Regards,

Ajay G

Reply | Threaded
Open this post in threaded view
|

Re: Spark History UI + Keycloak Integration

Marcelo Vanzin-2
On Fri, Jan 4, 2019 at 3:25 AM G, Ajay (Nokia - IN/Bangalore)
<[hidden email]> wrote:
...
> Added session handler for all context -   contextHandler.setSessionHandler(new SessionHandler())
...
> Keycloak authentication seems to work, Is this the right approach ? If it is fine I can submit a PR.

I don't remember many details about servlet session management, and
whether it can be enabled some other way, but that seems ok. I'd just
make it a new config, since otherwise Spark doesn't need the extra
overhead.

--
Marcelo

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]