DataSourceV2 implementation for JDBC sources

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

DataSourceV2 implementation for JDBC sources

Thomas D'Silva

I am trying to use the DataSourceV2 API to implement a spark connector for Apache Phoenix. I am not using JDBCRelation because I want to optimize how partitions are created during reads and provide support for more complicated filter pushdown. 

For reading I am using JdbcUtils.resultSetToSparkInternalRows (whose visibility is private[spark]) to convert a ResultSet to an InternalRow. 
For writing I would like to use JdbcUtils.makeSetter to set the parameters of a PreparedStatement, but this function is private so it cannot be used. I was wondering if there is support for changing the visibility of this method to private[spark] so users don't have to re-implement this.

Thank you,