Error in custom encoder

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view

Error in custom encoder

Hello everyone,

I'm trying to implement new encoder, which will be able to encode Kotlin
data classes.

They can be generics, so I'm calculating their types in compiletime.

My trouble is I have following code generated:

/* 197 */       final InternalRow deserializetoobject_value_9 = false ?
/* 198 */       null : new kotlin.Pair(deserializetoobject_argValue_1,

It's obvious that InternalRow can't be assigned with kotlin.Pair

And therefore errors like this:
20/03/19 16:31:18 ERROR CodeGenerator: failed to compile:
org.codehaus.commons.compiler.CompileException: File '',
Line 195, Column 74: Assignment conversion not possible from type
"kotlin.Pair" to type "org.apache.spark.sql.catalyst.InternalRow"

Here [1] I put gist with more code: mine and generated and also stacktace

I'm building my encoder against current spark snapshot

Thank you,


0x87075DBDFED351B5.asc (7K) Download Attachment
signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view

Re: Error in custom encoder

Nevermind, solved it. It was incorrect Literal creation with StructType
instead of ObjectType

Sent from:

To unsubscribe e-mail: [hidden email]