Unable to infer schema pf Parquet in Spark 2.0.0

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Unable to infer schema pf Parquet in Spark 2.0.0

Priya Ch
Hi All, 

  I am trying to write a dataframe into S3 in PARQUET format in Overwrite mode. However, post the job succeeds, I could see the results when queried from Hive as -



When trying to read the same table in Spark, I am seeing 0 records and reading from S3 path, I see the error "Unable to infer schema for parquet." Here are the screenshots -




Please someone try to throw light on this.

Thanks, 
Padma CH