JDBC connector for DataSourceV2

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

JDBC connector for DataSourceV2

shivsood
Can someone please help understand the current Status of DataSource V2 based JDBC connector? I see connectors for various file formats in Master, but can't find a JDBC implementation or related JIRA.

DatasourceV2 APIs to me look in good shape to attempt a JDBC connector for READ/WRITE path.

Thanks & Regards,
Shiv
Reply | Threaded
Open this post in threaded view
|

Re: JDBC connector for DataSourceV2

Ryan Blue
I'm not aware of a JDBC connector effort. It would be great to have someone build one!

On Fri, Jul 12, 2019 at 3:33 PM Shiv Prashant Sood <[hidden email]> wrote:
Can someone please help understand the current Status of DataSource V2 based JDBC connector? I see connectors for various file formats in Master, but can't find a JDBC implementation or related JIRA.

DatasourceV2 APIs to me look in good shape to attempt a JDBC connector for READ/WRITE path.

Thanks & Regards,
Shiv


--
Ryan Blue
Software Engineer
Netflix
Reply | Threaded
Open this post in threaded view
|

Re: JDBC connector for DataSourceV2

Xianyin Xin

If there’s nobody working on that, I’d like to contribute.

 

Loop in @Gengliang Wang.

 

Xianyin

 

From: Ryan Blue <[hidden email]>
Reply-To: <[hidden email]>
Date: Saturday, July 13, 2019 at 6:54 AM
To: Shiv Prashant Sood <[hidden email]>
Cc: Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

I'm not aware of a JDBC connector effort. It would be great to have someone build one!

 

On Fri, Jul 12, 2019 at 3:33 PM Shiv Prashant Sood <[hidden email]> wrote:

Can someone please help understand the current Status of DataSource V2 based JDBC connector? I see connectors for various file formats in Master, but can't find a JDBC implementation or related JIRA.

 

DatasourceV2 APIs to me look in good shape to attempt a JDBC connector for READ/WRITE path.

Thanks & Regards,

Shiv


 

--

Ryan Blue

Software Engineer

Netflix

Reply | Threaded
Open this post in threaded view
|

Re: JDBC connector for DataSourceV2

Ryan Blue
Sounds great! Ping me on the review, I think this will be really valuable.

On Fri, Jul 12, 2019 at 6:51 PM Xianyin Xin <[hidden email]> wrote:

If there’s nobody working on that, I’d like to contribute.

 

Loop in @Gengliang Wang.

 

Xianyin

 

From: Ryan Blue <[hidden email]>
Reply-To: <[hidden email]>
Date: Saturday, July 13, 2019 at 6:54 AM
To: Shiv Prashant Sood <[hidden email]>
Cc: Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

I'm not aware of a JDBC connector effort. It would be great to have someone build one!

 

On Fri, Jul 12, 2019 at 3:33 PM Shiv Prashant Sood <[hidden email]> wrote:

Can someone please help understand the current Status of DataSource V2 based JDBC connector? I see connectors for various file formats in Master, but can't find a JDBC implementation or related JIRA.

 

DatasourceV2 APIs to me look in good shape to attempt a JDBC connector for READ/WRITE path.

Thanks & Regards,

Shiv


 

--

Ryan Blue

Software Engineer

Netflix



--
Ryan Blue
Software Engineer
Netflix
Reply | Threaded
Open this post in threaded view
|

Re: JDBC connector for DataSourceV2

shivsood
In reply to this post by Xianyin Xin
Thanks all. I can also contribute toward this effort.

Regards,
Shiv

Sent from my iPhone

On Jul 12, 2019, at 6:51 PM, Xianyin Xin <[hidden email]> wrote:

If there’s nobody working on that, I’d like to contribute.

 

Loop in @Gengliang Wang.

 

Xianyin

 

From: Ryan Blue <[hidden email]>
Reply-To: <[hidden email]>
Date: Saturday, July 13, 2019 at 6:54 AM
To: Shiv Prashant Sood <[hidden email]>
Cc: Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

I'm not aware of a JDBC connector effort. It would be great to have someone build one!

 

On Fri, Jul 12, 2019 at 3:33 PM Shiv Prashant Sood <[hidden email]> wrote:

Can someone please help understand the current Status of DataSource V2 based JDBC connector? I see connectors for various file formats in Master, but can't find a JDBC implementation or related JIRA.

 

DatasourceV2 APIs to me look in good shape to attempt a JDBC connector for READ/WRITE path.

Thanks & Regards,

Shiv


 

--

Ryan Blue

Software Engineer

Netflix

Reply | Threaded
Open this post in threaded view
|

Re: JDBC connector for DataSourceV2

Gabor Somogyi
Hi Guys,

Don't know what's the intention exactly here but there is such a PR: https://github.com/apache/spark/pull/22560
If that's what we need maybe we can resurrect it. BTW, I'm also interested in...

BR,
G


On Sat, Jul 13, 2019 at 4:09 AM Shiv Prashant Sood <[hidden email]> wrote:
Thanks all. I can also contribute toward this effort.

Regards,
Shiv

Sent from my iPhone

On Jul 12, 2019, at 6:51 PM, Xianyin Xin <[hidden email]> wrote:

If there’s nobody working on that, I’d like to contribute.

 

Loop in @Gengliang Wang.

 

Xianyin

 

From: Ryan Blue <[hidden email]>
Reply-To: <[hidden email]>
Date: Saturday, July 13, 2019 at 6:54 AM
To: Shiv Prashant Sood <[hidden email]>
Cc: Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

I'm not aware of a JDBC connector effort. It would be great to have someone build one!

 

On Fri, Jul 12, 2019 at 3:33 PM Shiv Prashant Sood <[hidden email]> wrote:

Can someone please help understand the current Status of DataSource V2 based JDBC connector? I see connectors for various file formats in Master, but can't find a JDBC implementation or related JIRA.

 

DatasourceV2 APIs to me look in good shape to attempt a JDBC connector for READ/WRITE path.

Thanks & Regards,

Shiv


 

--

Ryan Blue

Software Engineer

Netflix

Reply | Threaded
Open this post in threaded view
|

Re: JDBC connector for DataSourceV2

shivsood
To me this looks like refactoring of DS1 JDBC to enable user provided connection factories. In itself a good change, but IMO not DSV2 related.

I created a JIRA and added some goals. Please comments/add as relevant.


JIRA for DataSourceV2 API based JDBC connector.

Goals :

  • Generic connector based on JDBC that supports all databases (min bar is support for all V1 data bases).
  • Reference implementation and Interface for any specialized JDBC connectors.

Regards,
Shiv

On Sat, Jul 13, 2019 at 2:17 AM Gabor Somogyi <[hidden email]> wrote:
Hi Guys,

Don't know what's the intention exactly here but there is such a PR: https://github.com/apache/spark/pull/22560
If that's what we need maybe we can resurrect it. BTW, I'm also interested in...

BR,
G


On Sat, Jul 13, 2019 at 4:09 AM Shiv Prashant Sood <[hidden email]> wrote:
Thanks all. I can also contribute toward this effort.

Regards,
Shiv

Sent from my iPhone

On Jul 12, 2019, at 6:51 PM, Xianyin Xin <[hidden email]> wrote:

If there’s nobody working on that, I’d like to contribute.

 

Loop in @Gengliang Wang.

 

Xianyin

 

From: Ryan Blue <[hidden email]>
Reply-To: <[hidden email]>
Date: Saturday, July 13, 2019 at 6:54 AM
To: Shiv Prashant Sood <[hidden email]>
Cc: Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

I'm not aware of a JDBC connector effort. It would be great to have someone build one!

 

On Fri, Jul 12, 2019 at 3:33 PM Shiv Prashant Sood <[hidden email]> wrote:

Can someone please help understand the current Status of DataSource V2 based JDBC connector? I see connectors for various file formats in Master, but can't find a JDBC implementation or related JIRA.

 

DatasourceV2 APIs to me look in good shape to attempt a JDBC connector for READ/WRITE path.

Thanks & Regards,

Shiv


 

--

Ryan Blue

Software Engineer

Netflix

Reply | Threaded
Open this post in threaded view
|

Re: JDBC connector for DataSourceV2

Xianyin Xin

There’s another pr https://github.com/apache/spark/pull/21861 but which is based the old V2 APIs.

 

We’d better link the JIRAs, SPARK-24907, SPARK-25547, and SPARK-28380 and finalize a plan.

 

Xianyin

 

From: Shiv Prashant Sood <[hidden email]>
Date: Sunday, July 14, 2019 at 2:59 AM
To: Gabor Somogyi <[hidden email]>
Cc: Xianyin Xin <[hidden email]>, Ryan Blue <[hidden email]>, <[hidden email]>, Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

To me this looks like refactoring of DS1 JDBC to enable user provided connection factories. In itself a good change, but IMO not DSV2 related.

 

I created a JIRA and added some goals. Please comments/add as relevant.

 

 

JIRA for DataSourceV2 API based JDBC connector.

Goals :

  • Generic connector based on JDBC that supports all databases (min bar is support for all V1 data bases).
  • Reference implementation and Interface for any specialized JDBC connectors.

 

Regards,

Shiv

 

On Sat, Jul 13, 2019 at 2:17 AM Gabor Somogyi <[hidden email]> wrote:

Hi Guys,

 

Don't know what's the intention exactly here but there is such a PR: https://github.com/apache/spark/pull/22560

If that's what we need maybe we can resurrect it. BTW, I'm also interested in...

 

BR,

G

 

 

On Sat, Jul 13, 2019 at 4:09 AM Shiv Prashant Sood <[hidden email]> wrote:

Thanks all. I can also contribute toward this effort.

 

Regards,

Shiv

Sent from my iPhone


On Jul 12, 2019, at 6:51 PM, Xianyin Xin <[hidden email]> wrote:

If there’s nobody working on that, I’d like to contribute.

 

Loop in @Gengliang Wang.

 

Xianyin

 

From: Ryan Blue <[hidden email]>
Reply-To: <[hidden email]>
Date: Saturday, July 13, 2019 at 6:54 AM
To: Shiv Prashant Sood <[hidden email]>
Cc: Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

I'm not aware of a JDBC connector effort. It would be great to have someone build one!

 

On Fri, Jul 12, 2019 at 3:33 PM Shiv Prashant Sood <[hidden email]> wrote:

Can someone please help understand the current Status of DataSource V2 based JDBC connector? I see connectors for various file formats in Master, but can't find a JDBC implementation or related JIRA.

 

DatasourceV2 APIs to me look in good shape to attempt a JDBC connector for READ/WRITE path.

Thanks & Regards,

Shiv


 

--

Ryan Blue

Software Engineer

Netflix

Reply | Threaded
Open this post in threaded view
|

Re: JDBC connector for DataSourceV2

Gabor Somogyi
I've had a look at the jiras and seems like the intention is the same (correct me if I'm wrong).
I think one is enough and the rest can be closed with duplicate.
We should keep multiple jiras only when the intention is different.

BR,
G


On Mon, Jul 15, 2019 at 6:01 AM Xianyin Xin <[hidden email]> wrote:

There’s another pr https://github.com/apache/spark/pull/21861 but which is based the old V2 APIs.

 

We’d better link the JIRAs, SPARK-24907, SPARK-25547, and SPARK-28380 and finalize a plan.

 

Xianyin

 

From: Shiv Prashant Sood <[hidden email]>
Date: Sunday, July 14, 2019 at 2:59 AM
To: Gabor Somogyi <[hidden email]>
Cc: Xianyin Xin <[hidden email]>, Ryan Blue <[hidden email]>, <[hidden email]>, Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

To me this looks like refactoring of DS1 JDBC to enable user provided connection factories. In itself a good change, but IMO not DSV2 related.

 

I created a JIRA and added some goals. Please comments/add as relevant.

 

 

JIRA for DataSourceV2 API based JDBC connector.

Goals :

  • Generic connector based on JDBC that supports all databases (min bar is support for all V1 data bases).
  • Reference implementation and Interface for any specialized JDBC connectors.

 

Regards,

Shiv

 

On Sat, Jul 13, 2019 at 2:17 AM Gabor Somogyi <[hidden email]> wrote:

Hi Guys,

 

Don't know what's the intention exactly here but there is such a PR: https://github.com/apache/spark/pull/22560

If that's what we need maybe we can resurrect it. BTW, I'm also interested in...

 

BR,

G

 

 

On Sat, Jul 13, 2019 at 4:09 AM Shiv Prashant Sood <[hidden email]> wrote:

Thanks all. I can also contribute toward this effort.

 

Regards,

Shiv

Sent from my iPhone


On Jul 12, 2019, at 6:51 PM, Xianyin Xin <[hidden email]> wrote:

If there’s nobody working on that, I’d like to contribute.

 

Loop in @Gengliang Wang.

 

Xianyin

 

From: Ryan Blue <[hidden email]>
Reply-To: <[hidden email]>
Date: Saturday, July 13, 2019 at 6:54 AM
To: Shiv Prashant Sood <[hidden email]>
Cc: Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

I'm not aware of a JDBC connector effort. It would be great to have someone build one!

 

On Fri, Jul 12, 2019 at 3:33 PM Shiv Prashant Sood <[hidden email]> wrote:

Can someone please help understand the current Status of DataSource V2 based JDBC connector? I see connectors for various file formats in Master, but can't find a JDBC implementation or related JIRA.

 

DatasourceV2 APIs to me look in good shape to attempt a JDBC connector for READ/WRITE path.

Thanks & Regards,

Shiv


 

--

Ryan Blue

Software Engineer

Netflix

Reply | Threaded
Open this post in threaded view
|

Re: JDBC connector for DataSourceV2

shivsood
Agree. Let's use SPARK-24907 as the JIRA for this work. Thanks for resolving SPARK-28380 as dupe of this.

Regards,
Shiv

On Mon, Jul 15, 2019 at 1:50 AM Gabor Somogyi <[hidden email]> wrote:
I've had a look at the jiras and seems like the intention is the same (correct me if I'm wrong).
I think one is enough and the rest can be closed with duplicate.
We should keep multiple jiras only when the intention is different.

BR,
G


On Mon, Jul 15, 2019 at 6:01 AM Xianyin Xin <[hidden email]> wrote:

There’s another pr https://github.com/apache/spark/pull/21861 but which is based the old V2 APIs.

 

We’d better link the JIRAs, SPARK-24907, SPARK-25547, and SPARK-28380 and finalize a plan.

 

Xianyin

 

From: Shiv Prashant Sood <[hidden email]>
Date: Sunday, July 14, 2019 at 2:59 AM
To: Gabor Somogyi <[hidden email]>
Cc: Xianyin Xin <[hidden email]>, Ryan Blue <[hidden email]>, <[hidden email]>, Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

To me this looks like refactoring of DS1 JDBC to enable user provided connection factories. In itself a good change, but IMO not DSV2 related.

 

I created a JIRA and added some goals. Please comments/add as relevant.

 

 

JIRA for DataSourceV2 API based JDBC connector.

Goals :

  • Generic connector based on JDBC that supports all databases (min bar is support for all V1 data bases).
  • Reference implementation and Interface for any specialized JDBC connectors.

 

Regards,

Shiv

 

On Sat, Jul 13, 2019 at 2:17 AM Gabor Somogyi <[hidden email]> wrote:

Hi Guys,

 

Don't know what's the intention exactly here but there is such a PR: https://github.com/apache/spark/pull/22560

If that's what we need maybe we can resurrect it. BTW, I'm also interested in...

 

BR,

G

 

 

On Sat, Jul 13, 2019 at 4:09 AM Shiv Prashant Sood <[hidden email]> wrote:

Thanks all. I can also contribute toward this effort.

 

Regards,

Shiv

Sent from my iPhone


On Jul 12, 2019, at 6:51 PM, Xianyin Xin <[hidden email]> wrote:

If there’s nobody working on that, I’d like to contribute.

 

Loop in @Gengliang Wang.

 

Xianyin

 

From: Ryan Blue <[hidden email]>
Reply-To: <[hidden email]>
Date: Saturday, July 13, 2019 at 6:54 AM
To: Shiv Prashant Sood <[hidden email]>
Cc: Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

I'm not aware of a JDBC connector effort. It would be great to have someone build one!

 

On Fri, Jul 12, 2019 at 3:33 PM Shiv Prashant Sood <[hidden email]> wrote:

Can someone please help understand the current Status of DataSource V2 based JDBC connector? I see connectors for various file formats in Master, but can't find a JDBC implementation or related JIRA.

 

DatasourceV2 APIs to me look in good shape to attempt a JDBC connector for READ/WRITE path.

Thanks & Regards,

Shiv


 

--

Ryan Blue

Software Engineer

Netflix

Reply | Threaded
Open this post in threaded view
|

RE: JDBC connector for DataSourceV2

Priyanka Gomatam

I would have thought one of the most important goals would be pushing down limits since V2 supports it.

 

I am also interested in collaborating. Thanks!

 

Priyanka Gomatam

 

From: Shiv Prashant Sood <[hidden email]>
Sent: Monday, July 15, 2019 10:22 AM
To: Gabor Somogyi <[hidden email]>
Cc: Xianyin Xin <[hidden email]>; Ryan Blue <[hidden email]>; [hidden email]; Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

Agree. Let's use SPARK-24907 as the JIRA for this work. Thanks for resolving SPARK-28380 as dupe of this.

 

Regards,

Shiv

 

On Mon, Jul 15, 2019 at 1:50 AM Gabor Somogyi <[hidden email]> wrote:

I've had a look at the jiras and seems like the intention is the same (correct me if I'm wrong).

I think one is enough and the rest can be closed with duplicate.

We should keep multiple jiras only when the intention is different.

 

BR,

G

 

 

On Mon, Jul 15, 2019 at 6:01 AM Xianyin Xin <[hidden email]> wrote:

There’s another pr https://github.com/apache/spark/pull/21861 but which is based the old V2 APIs.

 

We’d better link the JIRAs, SPARK-24907, SPARK-25547, and SPARK-28380 and finalize a plan.

 

Xianyin

 

From: Shiv Prashant Sood <[hidden email]>
Date: Sunday, July 14, 2019 at 2:59 AM
To: Gabor Somogyi <[hidden email]>
Cc: Xianyin Xin <[hidden email]>, Ryan Blue <[hidden email]>, <[hidden email]>, Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

To me this looks like refactoring of DS1 JDBC to enable user provided connection factories. In itself a good change, but IMO not DSV2 related.

 

I created a JIRA and added some goals. Please comments/add as relevant.

 

 

JIRA for DataSourceV2 API based JDBC connector.

Goals :

  • Generic connector based on JDBC that supports all databases (min bar is support for all V1 data bases).
  • Reference implementation and Interface for any specialized JDBC connectors.

 

Regards,

Shiv

 

On Sat, Jul 13, 2019 at 2:17 AM Gabor Somogyi <[hidden email]> wrote:

Hi Guys,

 

Don't know what's the intention exactly here but there is such a PR: https://github.com/apache/spark/pull/22560

If that's what we need maybe we can resurrect it. BTW, I'm also interested in...

 

BR,

G

 

 

On Sat, Jul 13, 2019 at 4:09 AM Shiv Prashant Sood <[hidden email]> wrote:

Thanks all. I can also contribute toward this effort.

 

Regards,

Shiv

Sent from my iPhone


On Jul 12, 2019, at 6:51 PM, Xianyin Xin <[hidden email]> wrote:

If there’s nobody working on that, I’d like to contribute.

 

Loop in @Gengliang Wang.

 

Xianyin

 

From: Ryan Blue <[hidden email]>
Reply-To: <[hidden email]>
Date: Saturday, July 13, 2019 at 6:54 AM
To: Shiv Prashant Sood <[hidden email]>
Cc: Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

I'm not aware of a JDBC connector effort. It would be great to have someone build one!

 

On Fri, Jul 12, 2019 at 3:33 PM Shiv Prashant Sood <[hidden email]> wrote:

Can someone please help understand the current Status of DataSource V2 based JDBC connector? I see connectors for various file formats in Master, but can't find a JDBC implementation or related JIRA.

 

DatasourceV2 APIs to me look in good shape to attempt a JDBC connector for READ/WRITE path.

Thanks & Regards,

Shiv


 

--

Ryan Blue

Software Engineer

Netflix

Reply | Threaded
Open this post in threaded view
|

Re: JDBC connector for DataSourceV2

Shiv Prashant Sood
All ,

Have a first draft of DataSourceV2 based JDBC connector available now ( PR#25211) . The goal was a MVP implementation with support for batch read/write.
I am looking forward for your review comments to help guide direction. Note that i am still understanding/addressing some issues. The plan, status issues is capture in the Readme.md

Ssummary of changes
- V2 connector changes are under under sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc. The implementation heavily reuses infra provided by JDBCUtils.
- JDBCUtils(sql/core/../datasources/jdbc/JdbcUtils.scala) file is refactored ( for few functions) to suite V2 needs.
- E2E test cases are in external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/MsSqlServerIntegrationSuite.scala

image.png

Current Status ( Refer Readme.md for details)
image.png


Regards,
Shiv

On Mon, Jul 15, 2019 at 5:16 PM Priyanka Gomatam <[hidden email]> wrote:

I would have thought one of the most important goals would be pushing down limits since V2 supports it.

 

I am also interested in collaborating. Thanks!

 

Priyanka Gomatam

 

From: Shiv Prashant Sood <[hidden email]>
Sent: Monday, July 15, 2019 10:22 AM
To: Gabor Somogyi <[hidden email]>
Cc: Xianyin Xin <[hidden email]>; Ryan Blue <[hidden email]>; [hidden email]; Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

Agree. Let's use SPARK-24907 as the JIRA for this work. Thanks for resolving SPARK-28380 as dupe of this.

 

Regards,

Shiv

 

On Mon, Jul 15, 2019 at 1:50 AM Gabor Somogyi <[hidden email]> wrote:

I've had a look at the jiras and seems like the intention is the same (correct me if I'm wrong).

I think one is enough and the rest can be closed with duplicate.

We should keep multiple jiras only when the intention is different.

 

BR,

G

 

 

On Mon, Jul 15, 2019 at 6:01 AM Xianyin Xin <[hidden email]> wrote:

There’s another pr https://github.com/apache/spark/pull/21861 but which is based the old V2 APIs.

 

We’d better link the JIRAs, SPARK-24907, SPARK-25547, and SPARK-28380 and finalize a plan.

 

Xianyin

 

From: Shiv Prashant Sood <[hidden email]>
Date: Sunday, July 14, 2019 at 2:59 AM
To: Gabor Somogyi <[hidden email]>
Cc: Xianyin Xin <[hidden email]>, Ryan Blue <[hidden email]>, <[hidden email]>, Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

To me this looks like refactoring of DS1 JDBC to enable user provided connection factories. In itself a good change, but IMO not DSV2 related.

 

I created a JIRA and added some goals. Please comments/add as relevant.

 

 

JIRA for DataSourceV2 API based JDBC connector.

Goals :

  • Generic connector based on JDBC that supports all databases (min bar is support for all V1 data bases).
  • Reference implementation and Interface for any specialized JDBC connectors.

 

Regards,

Shiv

 

On Sat, Jul 13, 2019 at 2:17 AM Gabor Somogyi <[hidden email]> wrote:

Hi Guys,

 

Don't know what's the intention exactly here but there is such a PR: https://github.com/apache/spark/pull/22560

If that's what we need maybe we can resurrect it. BTW, I'm also interested in...

 

BR,

G

 

 

On Sat, Jul 13, 2019 at 4:09 AM Shiv Prashant Sood <[hidden email]> wrote:

Thanks all. I can also contribute toward this effort.

 

Regards,

Shiv

Sent from my iPhone


On Jul 12, 2019, at 6:51 PM, Xianyin Xin <[hidden email]> wrote:

If there’s nobody working on that, I’d like to contribute.

 

Loop in @Gengliang Wang.

 

Xianyin

 

From: Ryan Blue <[hidden email]>
Reply-To: <[hidden email]>
Date: Saturday, July 13, 2019 at 6:54 AM
To: Shiv Prashant Sood <[hidden email]>
Cc: Spark Dev List <[hidden email]>
Subject: Re: JDBC connector for DataSourceV2

 

I'm not aware of a JDBC connector effort. It would be great to have someone build one!

 

On Fri, Jul 12, 2019 at 3:33 PM Shiv Prashant Sood <[hidden email]> wrote:

Can someone please help understand the current Status of DataSource V2 based JDBC connector? I see connectors for various file formats in Master, but can't find a JDBC implementation or related JIRA.

 

DatasourceV2 APIs to me look in good shape to attempt a JDBC connector for READ/WRITE path.

Thanks & Regards,

Shiv


 

--

Ryan Blue

Software Engineer

Netflix