[VOTE][SPARK-29018][SPIP]:Build spark thrift server based on protocol v11

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[VOTE][SPARK-29018][SPIP]:Build spark thrift server based on protocol v11


With the development of Spark and Hive,in current sql/hive-thriftserver module, 

we need to do a lot of work to solve code conflicts for different built-in hive versions.

It's an annoying and unending work in current ways. And these issues have limited 

our ability and convenience to develop new features for Spark’s thrift server. 

    We suppose to implement a new thrift server and JDBC driver based on Hive’s latest v11 

TCLService.thrift thrift protocol. Finally, the new thrift server have below feature:

  1. Build new module spark-service as spark’s thrift server 

  2. Don't need as much reflection and inherited code as `hive-thriftser` modules

  3. Support all functions current `sql/hive-thriftserver` support

  4. Use all code maintained by spark itself, won’t depend on Hive

  5. Support origin functions use spark’s own way, won't limited by Hive's code

  6. Support running without hive metastore or with hive metastore

  7. Support user impersonation by Multi-tenant splited hive authentication and DFS authentication

  8. Support session hook for with spark’s own code

  9. Add a new jdbc driver spark-jdbc, with spark’s own connection url  “jdbc:spark:<host>:<port>/<db>”

  10. Support both hive-jdbc and spark-jdbc client, then we can support most clients and BI platform


[ ] +1: Accept the proposal as an official SPIP
[ ] +0
[ ] -1: I don't think this is a good idea because ...

I'll start with my +1

--------------------------------------------------------------------- To unsubscribe e-mail: [hidden email]