Apache ORC 1.6.6 Release

[Dongjoon Hyun-2]

Hi, All.

I'd like to propose to cut Apache ORC 1.6.6 release as the first backward compatible version for Apache Spark. To achieve the backward compatibility, we resolved most backward compatibility issues after Apache ORC 1.6.5 releases and started to publish SNAPSHOTs.

ORC-689: Add GitHubAction job to publish snapshot
ORC-685: Add `ReaderImpl.extractFileTail` back
ORC-677: Add a deprecated legacy constructor SargApplier back
ORC-676. Add getRawDataSizeFromColIndices back to ReaderImpl
ORC-671. Add OrcTail.getStripeStatistics back for backward compatibility
ORC-669. Reduce breaking changes in ReaderImpl.java

As of today, the snapshot release passed Apache Spark and Apache Iceberg UTs.

https://github.com/dongjoon-hyun/spark/pull/41
https://github.com/dongjoon-hyun/iceberg/pull/1

I start to roll 1.6.6-rc0. After 1.6.6 release, 1.6.7 will focus on Apache Hive.

Thanks,

Dongjoon.

[Dongjoon Hyun-2]

Oh, my bad. The previous email was written for `[hidden email]`.

Apache ORC 1.6.6 is not for Apache Spark 3.1.

It's prepared for Apache Spark 3.2 (2020 Summer) to provide mainly columnar encryption and better ZStd support in Spark.

Bests,

Dongjoon.

On Thu, Dec 3, 2020 at 10:18 AM Dongjoon Hyun <[hidden email]> wrote:

Hi, All.

I'd like to propose to cut Apache ORC 1.6.6 release as the first backward compatible version for Apache Spark. To achieve the backward compatibility, we resolved most backward compatibility issues after Apache ORC 1.6.5 releases and started to publish SNAPSHOTs.

ORC-689: Add GitHubAction job to publish snapshot
ORC-685: Add `ReaderImpl.extractFileTail` back
ORC-677: Add a deprecated legacy constructor SargApplier back
ORC-676. Add getRawDataSizeFromColIndices back to ReaderImpl
ORC-671. Add OrcTail.getStripeStatistics back for backward compatibility
ORC-669. Reduce breaking changes in ReaderImpl.java

As of today, the snapshot release passed Apache Spark and Apache Iceberg UTs.

https://github.com/dongjoon-hyun/spark/pull/41
https://github.com/dongjoon-hyun/iceberg/pull/1

I start to roll 1.6.6-rc0. After 1.6.6 release, 1.6.7 will focus on Apache Hive.

Thanks,

Dongjoon.

[Hyukjin Kwon]

It's still good to know since Spark uses ORC :-)

2020년 12월 4일 (금) 오전 3:34, Dongjoon Hyun <[hidden email]>님이 작성:

Oh, my bad. The previous email was written for `[hidden email]`.

Apache ORC 1.6.6 is not for Apache Spark 3.1.

It's prepared for Apache Spark 3.2 (2020 Summer) to provide mainly columnar encryption and better ZStd support in Spark.

Bests,

Dongjoon.

On Thu, Dec 3, 2020 at 10:18 AM Dongjoon Hyun <[hidden email]> wrote:

Hi, All.

I'd like to propose to cut Apache ORC 1.6.6 release as the first backward compatible version for Apache Spark. To achieve the backward compatibility, we resolved most backward compatibility issues after Apache ORC 1.6.5 releases and started to publish SNAPSHOTs.

ORC-689: Add GitHubAction job to publish snapshot
ORC-685: Add `ReaderImpl.extractFileTail` back
ORC-677: Add a deprecated legacy constructor SargApplier back
ORC-676. Add getRawDataSizeFromColIndices back to ReaderImpl
ORC-671. Add OrcTail.getStripeStatistics back for backward compatibility
ORC-669. Reduce breaking changes in ReaderImpl.java

As of today, the snapshot release passed Apache Spark and Apache Iceberg UTs.

https://github.com/dongjoon-hyun/spark/pull/41
https://github.com/dongjoon-hyun/iceberg/pull/1

I start to roll 1.6.6-rc0. After 1.6.6 release, 1.6.7 will focus on Apache Hive.

Thanks,

Dongjoon.