DataSourceV2 community sync tonight

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

DataSourceV2 community sync tonight

Ryan Blue

Hi everyone,

This is a quick reminder that there is a DSv2 community sync tonight at 5 PM PST. These community syncs are open to anyone that wants to participate. If you’d like to be added to the invite, please send me a direct message.

The main topic for this sync is the catalog API. To make discussion easier, I think we should separate the current discussion into a few orthogonal areas and discuss each:

  • Catalog API plugin system
  • Plan for migration to a new catalog API
  • Catalog API design approach using separate interfaces (TableCatalog, UDFCatalog, FunctionCatalog, etc.)
  • TableCatalog API proposal and implementation, PR #21306 (not the proposed user-facing API)

If we have time, we can also talk about the user-facing API proposed in the SPIP.

Thanks,

rb

--
Ryan Blue
Software Engineer
Netflix
Reply | Threaded
Open this post in threaded view
|

Re: DataSourceV2 community sync tonight

cloud0fan
There are 2 remaining problems in the write side API refactor:
1. how to put the `queryId` parameter in the write API
2. how to put the streaming OutputMode parameter in the write API

I'd like to discuss it and hopefully we can get a consensus to move forward.

Thanks,
Wenchen

On Thu, Jan 10, 2019 at 2:40 AM Ryan Blue <[hidden email]> wrote:

Hi everyone,

This is a quick reminder that there is a DSv2 community sync tonight at 5 PM PST. These community syncs are open to anyone that wants to participate. If you’d like to be added to the invite, please send me a direct message.

The main topic for this sync is the catalog API. To make discussion easier, I think we should separate the current discussion into a few orthogonal areas and discuss each:

  • Catalog API plugin system
  • Plan for migration to a new catalog API
  • Catalog API design approach using separate interfaces (TableCatalog, UDFCatalog, FunctionCatalog, etc.)
  • TableCatalog API proposal and implementation, PR #21306 (not the proposed user-facing API)

If we have time, we can also talk about the user-facing API proposed in the SPIP.

Thanks,

rb

--
Ryan Blue
Software Engineer
Netflix