Some Spark MLLIB tests failing due to some classes not being registered with Kryo

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Some Spark MLLIB tests failing due to some classes not being registered with Kryo

Jorge Sánchez
Hi Dev,

I'm running the MLLIB tests in the current Master branch and the following Suites are failing due to some classes not being registered with Kryo:

org.apache.spark.mllib.MatricesSuite
org.apache.spark.mllib.VectorsSuite
org.apache.spark.ml.InstanceSuite

I can solve it by registering the failing classes with Kryo, but I'm wondering if I'm missing something as these tests shouldn't be failing from Master.

Any suggestions on what I may be doing wrong?

Thank you.
Reply | Threaded
Open this post in threaded view
|

Re: Some Spark MLLIB tests failing due to some classes not being registered with Kryo

Marco Gaido
Hi Jorge,

then try running the tests not from the mllib folder, but on Spark base directory.
If you want to run only the tests in mllib, you can specify the project using the -pl argument of mvn.

Thanks,
Marco



2017-11-11 13:37 GMT+01:00 Jorge Sánchez <[hidden email]>:
Hi Marco,

Just mvn test from the mllib folder.

Thank you.

2017-11-11 12:36 GMT+00:00 Marco Gaido <[hidden email]>:
Hi Jorge,

how are you running those tests?

Thanks,
Marco

2017-11-11 13:21 GMT+01:00 Jorge Sánchez <[hidden email]>:
Hi Dev,

I'm running the MLLIB tests in the current Master branch and the following Suites are failing due to some classes not being registered with Kryo:

org.apache.spark.mllib.MatricesSuite
org.apache.spark.mllib.VectorsSuite
org.apache.spark.ml.InstanceSuite

I can solve it by registering the failing classes with Kryo, but I'm wondering if I'm missing something as these tests shouldn't be failing from Master.

Any suggestions on what I may be doing wrong?

Thank you.



Reply | Threaded
Open this post in threaded view
|

Re: Some Spark MLLIB tests failing due to some classes not being registered with Kryo

Jorge Sánchez
No luck running the full test suites with mvn test from the main folder or just mvn -pl mllib.

Any other suggestion would be much appreciated.

Thank you.

2017-11-11 12:46 GMT+00:00 Marco Gaido <[hidden email]>:
Hi Jorge,

then try running the tests not from the mllib folder, but on Spark base directory.
If you want to run only the tests in mllib, you can specify the project using the -pl argument of mvn.

Thanks,
Marco



2017-11-11 13:37 GMT+01:00 Jorge Sánchez <[hidden email]>:
Hi Marco,

Just mvn test from the mllib folder.

Thank you.

2017-11-11 12:36 GMT+00:00 Marco Gaido <[hidden email]>:
Hi Jorge,

how are you running those tests?

Thanks,
Marco

2017-11-11 13:21 GMT+01:00 Jorge Sánchez <[hidden email]>:
Hi Dev,

I'm running the MLLIB tests in the current Master branch and the following Suites are failing due to some classes not being registered with Kryo:

org.apache.spark.mllib.MatricesSuite
org.apache.spark.mllib.VectorsSuite
org.apache.spark.ml.InstanceSuite

I can solve it by registering the failing classes with Kryo, but I'm wondering if I'm missing something as these tests shouldn't be failing from Master.

Any suggestions on what I may be doing wrong?

Thank you.