[build system] maven master branch builds timing out en masse...

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[build system] maven master branch builds timing out en masse...

shane knapp
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7/
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7-ubuntu-testing/
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-3.2/
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-3.2-jdk-11/

spark-master-test-maven-hadoop-3.2 looks like it's been timing out for
~2 weeks, but the others only for a few days.

all of these builds have a 335 minute (5 hours, 35 mins) timeout configured.

honestly, i feel that 5.5 hours is already an insane amount of time
for these builds to run...   do we need to increase it?  can
someone(s) here figure out what's taking so long and refactor some of
the tests?

shane
--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [build system] maven master branch builds timing out en masse...

shane knapp
just chatted w/sean privately and i'm going to up the test timeouts to
480mins (8 hours).

i still don't like this but at least it should hopefully get things green again.

On Mon, Oct 7, 2019 at 11:31 AM Shane Knapp <[hidden email]> wrote:

>
> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7/
> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7-ubuntu-testing/
> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-3.2/
> https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-3.2-jdk-11/
>
> spark-master-test-maven-hadoop-3.2 looks like it's been timing out for
> ~2 weeks, but the others only for a few days.
>
> all of these builds have a 335 minute (5 hours, 35 mins) timeout configured.
>
> honestly, i feel that 5.5 hours is already an insane amount of time
> for these builds to run...   do we need to increase it?  can
> someone(s) here figure out what's taking so long and refactor some of
> the tests?
>
> shane
> --
> Shane Knapp
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu



--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [build system] maven master branch builds timing out en masse...

Sean Owen-2
Moving the conversation here -- yes, why on earth are they taking this
long all of the sudden? we'll have to look again when they come back
online.  The last successful build took 6 hours, of which 4:45 were
the unit tests themselves.

It's mostly SQL tests; SQLQuerySuite is approaching an hour.
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-3.2/lastStableBuild/testReport/org.apache.spark.sql/SQLQueryTestSuite/

Well, at least it seems like we legitimately need these full
integration tests to run that long, and that's just life right now, so
8 hours is better than failing. But yeah, seems like we just test a
whole lot more recently and it's taking a long time. Kudos to recent
efforts to parallelize some of that.


On Mon, Oct 7, 2019 at 1:52 PM Shane Knapp <[hidden email]> wrote:

>
> just chatted w/sean privately and i'm going to up the test timeouts to
> 480mins (8 hours).
>
> i still don't like this but at least it should hopefully get things green again.
>
> On Mon, Oct 7, 2019 at 11:31 AM Shane Knapp <[hidden email]> wrote:
> >
> > https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7/
> > https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7-ubuntu-testing/
> > https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-3.2/
> > https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-3.2-jdk-11/
> >
> > spark-master-test-maven-hadoop-3.2 looks like it's been timing out for
> > ~2 weeks, but the others only for a few days.
> >
> > all of these builds have a 335 minute (5 hours, 35 mins) timeout configured.
> >
> > honestly, i feel that 5.5 hours is already an insane amount of time
> > for these builds to run...   do we need to increase it?  can
> > someone(s) here figure out what's taking so long and refactor some of
> > the tests?
> >
> > shane
> > --
> > Shane Knapp
> > UC Berkeley EECS Research / RISELab Staff Technical Lead
> > https://rise.cs.berkeley.edu
>
>
>
> --
> Shane Knapp
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]