Committing while Jenkins down?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Committing while Jenkins down?

Thomas graves-2
This is directed towards committers/PMC members.

It looks like Jenkins will be down for a while, what is everyone's
thoughts on committing PRs while its down?  Do we want to wait for
Jenkins to come back up, manually run things ourselves and commit?

Tom

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Committing while Jenkins down?

Xiao Li
Please check the note from Shane. 

[build system] IMPORTANT! northern california fire danger, potential power outage(s)

Thomas graves <[hidden email]> 于2019年10月10日周四 上午8:35写道:
This is directed towards committers/PMC members.

It looks like Jenkins will be down for a while, what is everyone's
thoughts on committing PRs while its down?  Do we want to wait for
Jenkins to come back up, manually run things ourselves and commit?

Tom

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Committing while Jenkins down?

Xiao Li
I think we are unable to merge any major PR if we do not know whether the tests can pass. 

Xiao

Xiao Li <[hidden email]> 于2019年10月10日周四 上午8:36写道:
Please check the note from Shane. 

[build system] IMPORTANT! northern california fire danger, potential power outage(s)

Thomas graves <[hidden email]> 于2019年10月10日周四 上午8:35写道:
This is directed towards committers/PMC members.

It looks like Jenkins will be down for a while, what is everyone's
thoughts on committing PRs while its down?  Do we want to wait for
Jenkins to come back up, manually run things ourselves and commit?

Tom

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Committing while Jenkins down?

Holden Karau
I think a reasonable, albeit slow, option is to run the tests locally. Since the outage could be as long as five days I’d rather not just have PRs pile up for that entire period.

On Thu, Oct 10, 2019 at 8:38 AM Xiao Li <[hidden email]> wrote:
I think we are unable to merge any major PR if we do not know whether the tests can pass. 

Xiao

Xiao Li <[hidden email]> 于2019年10月10日周四 上午8:36写道:
Please check the note from Shane. 

[build system] IMPORTANT! northern california fire danger, potential power outage(s)

Thomas graves <[hidden email]> 于2019年10月10日周四 上午8:35写道:
This is directed towards committers/PMC members.

It looks like Jenkins will be down for a while, what is everyone's
thoughts on committing PRs while its down?  Do we want to wait for
Jenkins to come back up, manually run things ourselves and commit?

Tom

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

--
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 
Reply | Threaded
Open this post in threaded view
|

Re: Committing while Jenkins down?

shane knapp
if we do get power back before the weekend, i can have my sysadmin
head down to the colo friday afternoon and power up jenkins.  he knows
the drill.


On Thu, Oct 10, 2019 at 8:50 AM Holden Karau <[hidden email]> wrote:

>
> I think a reasonable, albeit slow, option is to run the tests locally. Since the outage could be as long as five days I’d rather not just have PRs pile up for that entire period.
>
> On Thu, Oct 10, 2019 at 8:38 AM Xiao Li <[hidden email]> wrote:
>>
>> I think we are unable to merge any major PR if we do not know whether the tests can pass.
>>
>> Xiao
>>
>> Xiao Li <[hidden email]> 于2019年10月10日周四 上午8:36写道:
>>>
>>> Please check the note from Shane.
>>>
>>> [build system] IMPORTANT! northern california fire danger, potential power outage(s)
>>>
>>> Thomas graves <[hidden email]> 于2019年10月10日周四 上午8:35写道:
>>>>
>>>> This is directed towards committers/PMC members.
>>>>
>>>> It looks like Jenkins will be down for a while, what is everyone's
>>>> thoughts on committing PRs while its down?  Do we want to wait for
>>>> Jenkins to come back up, manually run things ourselves and commit?
>>>>
>>>> Tom
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: [hidden email]
>>>>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau



--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Committing while Jenkins down?

Holden Karau
Awesome, thanks Shane :)

In the meantime I think committers can just run tests locally and it’ll be a slower process but I don’t think we need to halt all merging.

On Thu, Oct 10, 2019 at 9:07 AM Shane Knapp <[hidden email]> wrote:
if we do get power back before the weekend, i can have my sysadmin
head down to the colo friday afternoon and power up jenkins.  he knows
the drill.


On Thu, Oct 10, 2019 at 8:50 AM Holden Karau <[hidden email]> wrote:
>
> I think a reasonable, albeit slow, option is to run the tests locally. Since the outage could be as long as five days I’d rather not just have PRs pile up for that entire period.
>
> On Thu, Oct 10, 2019 at 8:38 AM Xiao Li <[hidden email]> wrote:
>>
>> I think we are unable to merge any major PR if we do not know whether the tests can pass.
>>
>> Xiao
>>
>> Xiao Li <[hidden email]> 于2019年10月10日周四 上午8:36写道:
>>>
>>> Please check the note from Shane.
>>>
>>> [build system] IMPORTANT! northern california fire danger, potential power outage(s)
>>>
>>> Thomas graves <[hidden email]> 于2019年10月10日周四 上午8:35写道:
>>>>
>>>> This is directed towards committers/PMC members.
>>>>
>>>> It looks like Jenkins will be down for a while, what is everyone's
>>>> thoughts on committing PRs while its down?  Do we want to wait for
>>>> Jenkins to come back up, manually run things ourselves and commit?
>>>>
>>>> Tom
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: [hidden email]
>>>>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau



--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
--
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 
Reply | Threaded
Open this post in threaded view
|

Re: Committing while Jenkins down?

Xiao Li
Thanks! Shane! 

AFAIK, it normally takes more than 5/6 hours to run all the tests. Any major changes in Core/SQL require running all the tests. If any committer did it before merging the code, I think it is fine to merge it.

Xiao 

Holden Karau <[hidden email]> 于2019年10月10日周四 上午9:11写道:
Awesome, thanks Shane :)

In the meantime I think committers can just run tests locally and it’ll be a slower process but I don’t think we need to halt all merging.

On Thu, Oct 10, 2019 at 9:07 AM Shane Knapp <[hidden email]> wrote:
if we do get power back before the weekend, i can have my sysadmin
head down to the colo friday afternoon and power up jenkins.  he knows
the drill.


On Thu, Oct 10, 2019 at 8:50 AM Holden Karau <[hidden email]> wrote:
>
> I think a reasonable, albeit slow, option is to run the tests locally. Since the outage could be as long as five days I’d rather not just have PRs pile up for that entire period.
>
> On Thu, Oct 10, 2019 at 8:38 AM Xiao Li <[hidden email]> wrote:
>>
>> I think we are unable to merge any major PR if we do not know whether the tests can pass.
>>
>> Xiao
>>
>> Xiao Li <[hidden email]> 于2019年10月10日周四 上午8:36写道:
>>>
>>> Please check the note from Shane.
>>>
>>> [build system] IMPORTANT! northern california fire danger, potential power outage(s)
>>>
>>> Thomas graves <[hidden email]> 于2019年10月10日周四 上午8:35写道:
>>>>
>>>> This is directed towards committers/PMC members.
>>>>
>>>> It looks like Jenkins will be down for a while, what is everyone's
>>>> thoughts on committing PRs while its down?  Do we want to wait for
>>>> Jenkins to come back up, manually run things ourselves and commit?
>>>>
>>>> Tom
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: [hidden email]
>>>>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau



--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
--
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 
Reply | Threaded
Open this post in threaded view
|

Re: Committing while Jenkins down?

Holden Karau


On Thu, Oct 10, 2019 at 9:13 AM Xiao Li <[hidden email]> wrote:
Thanks! Shane! 

AFAIK, it normally takes more than 5/6 hours to run all the tests. Any major changes in Core/SQL require running all the tests. If any committer did it before merging the code, I think it is fine to merge it.
Glad were on the same page. Personally I have a desktop I can kick off the tests on while I keep working on other things. Just need a separate check out.

Note: Some of the k8s integration tests don’t work outside of Jenkins without extra configuration, so I think we need to be careful with k8s related changes.

Xiao 

Holden Karau <[hidden email]> 于2019年10月10日周四 上午9:11写道:
Awesome, thanks Shane :)

In the meantime I think committers can just run tests locally and it’ll be a slower process but I don’t think we need to halt all merging.

On Thu, Oct 10, 2019 at 9:07 AM Shane Knapp <[hidden email]> wrote:
if we do get power back before the weekend, i can have my sysadmin
head down to the colo friday afternoon and power up jenkins.  he knows
the drill.


On Thu, Oct 10, 2019 at 8:50 AM Holden Karau <[hidden email]> wrote:
>
> I think a reasonable, albeit slow, option is to run the tests locally. Since the outage could be as long as five days I’d rather not just have PRs pile up for that entire period.
>
> On Thu, Oct 10, 2019 at 8:38 AM Xiao Li <[hidden email]> wrote:
>>
>> I think we are unable to merge any major PR if we do not know whether the tests can pass.
>>
>> Xiao
>>
>> Xiao Li <[hidden email]> 于2019年10月10日周四 上午8:36写道:
>>>
>>> Please check the note from Shane.
>>>
>>> [build system] IMPORTANT! northern california fire danger, potential power outage(s)
>>>
>>> Thomas graves <[hidden email]> 于2019年10月10日周四 上午8:35写道:
>>>>
>>>> This is directed towards committers/PMC members.
>>>>
>>>> It looks like Jenkins will be down for a while, what is everyone's
>>>> thoughts on committing PRs while its down?  Do we want to wait for
>>>> Jenkins to come back up, manually run things ourselves and commit?
>>>>
>>>> Tom
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: [hidden email]
>>>>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau



--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
--
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 
--
Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 
Reply | Threaded
Open this post in threaded view
|

Re: Committing while Jenkins down?

shane knapp
In reply to this post by Xiao Li
yeah, as long as tests are run locally i'm ok w/merging.  once
california is elevated from 'developing country' status to 'holy crap,
the magic of electricity has returned!' and we can automatically build
again any errors that slipped through will be caught in jenkins.

this means we'll need to keep a close eye on the 2.4 and master builds
for most of next week to ensure correctness.

shane

On Thu, Oct 10, 2019 at 9:13 AM Xiao Li <[hidden email]> wrote:

>
> Thanks! Shane!
>
> AFAIK, it normally takes more than 5/6 hours to run all the tests. Any major changes in Core/SQL require running all the tests. If any committer did it before merging the code, I think it is fine to merge it.
>
> Xiao
>
> Holden Karau <[hidden email]> 于2019年10月10日周四 上午9:11写道:
>>
>> Awesome, thanks Shane :)
>>
>> In the meantime I think committers can just run tests locally and it’ll be a slower process but I don’t think we need to halt all merging.
>>
>> On Thu, Oct 10, 2019 at 9:07 AM Shane Knapp <[hidden email]> wrote:
>>>
>>> if we do get power back before the weekend, i can have my sysadmin
>>> head down to the colo friday afternoon and power up jenkins.  he knows
>>> the drill.
>>>
>>>
>>> On Thu, Oct 10, 2019 at 8:50 AM Holden Karau <[hidden email]> wrote:
>>> >
>>> > I think a reasonable, albeit slow, option is to run the tests locally. Since the outage could be as long as five days I’d rather not just have PRs pile up for that entire period.
>>> >
>>> > On Thu, Oct 10, 2019 at 8:38 AM Xiao Li <[hidden email]> wrote:
>>> >>
>>> >> I think we are unable to merge any major PR if we do not know whether the tests can pass.
>>> >>
>>> >> Xiao
>>> >>
>>> >> Xiao Li <[hidden email]> 于2019年10月10日周四 上午8:36写道:
>>> >>>
>>> >>> Please check the note from Shane.
>>> >>>
>>> >>> [build system] IMPORTANT! northern california fire danger, potential power outage(s)
>>> >>>
>>> >>> Thomas graves <[hidden email]> 于2019年10月10日周四 上午8:35写道:
>>> >>>>
>>> >>>> This is directed towards committers/PMC members.
>>> >>>>
>>> >>>> It looks like Jenkins will be down for a while, what is everyone's
>>> >>>> thoughts on committing PRs while its down?  Do we want to wait for
>>> >>>> Jenkins to come back up, manually run things ourselves and commit?
>>> >>>>
>>> >>>> Tom
>>> >>>>
>>> >>>> ---------------------------------------------------------------------
>>> >>>> To unsubscribe e-mail: [hidden email]
>>> >>>>
>>> > --
>>> > Twitter: https://twitter.com/holdenkarau
>>> > Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9
>>> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>
>>>
>>>
>>> --
>>> Shane Knapp
>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>> https://rise.cs.berkeley.edu
>>
>> --
>> Twitter: https://twitter.com/holdenkarau
>> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9
>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau



--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Committing while Jenkins down?

shane knapp
In reply to this post by Holden Karau
for running k8s tests locally, i have a section dedicated to that here:
https://spark.apache.org/developer-tools.html

minikube and friends is pretty straightforward to set up, but we're
running an older version of the former.  i am planning on addressing
that (and moving us to a recent release) by the end of this month...

...if we get power back.  ;)

shane

On Thu, Oct 10, 2019 at 9:16 AM Holden Karau <[hidden email]> wrote:

>
>
>
> On Thu, Oct 10, 2019 at 9:13 AM Xiao Li <[hidden email]> wrote:
>>
>> Thanks! Shane!
>>
>> AFAIK, it normally takes more than 5/6 hours to run all the tests. Any major changes in Core/SQL require running all the tests. If any committer did it before merging the code, I think it is fine to merge it.
>
> Glad were on the same page. Personally I have a desktop I can kick off the tests on while I keep working on other things. Just need a separate check out.
>
> Note: Some of the k8s integration tests don’t work outside of Jenkins without extra configuration, so I think we need to be careful with k8s related changes.
>>
>>
>> Xiao
>>
>> Holden Karau <[hidden email]> 于2019年10月10日周四 上午9:11写道:
>>>
>>> Awesome, thanks Shane :)
>>>
>>> In the meantime I think committers can just run tests locally and it’ll be a slower process but I don’t think we need to halt all merging.
>>>
>>> On Thu, Oct 10, 2019 at 9:07 AM Shane Knapp <[hidden email]> wrote:
>>>>
>>>> if we do get power back before the weekend, i can have my sysadmin
>>>> head down to the colo friday afternoon and power up jenkins.  he knows
>>>> the drill.
>>>>
>>>>
>>>> On Thu, Oct 10, 2019 at 8:50 AM Holden Karau <[hidden email]> wrote:
>>>> >
>>>> > I think a reasonable, albeit slow, option is to run the tests locally. Since the outage could be as long as five days I’d rather not just have PRs pile up for that entire period.
>>>> >
>>>> > On Thu, Oct 10, 2019 at 8:38 AM Xiao Li <[hidden email]> wrote:
>>>> >>
>>>> >> I think we are unable to merge any major PR if we do not know whether the tests can pass.
>>>> >>
>>>> >> Xiao
>>>> >>
>>>> >> Xiao Li <[hidden email]> 于2019年10月10日周四 上午8:36写道:
>>>> >>>
>>>> >>> Please check the note from Shane.
>>>> >>>
>>>> >>> [build system] IMPORTANT! northern california fire danger, potential power outage(s)
>>>> >>>
>>>> >>> Thomas graves <[hidden email]> 于2019年10月10日周四 上午8:35写道:
>>>> >>>>
>>>> >>>> This is directed towards committers/PMC members.
>>>> >>>>
>>>> >>>> It looks like Jenkins will be down for a while, what is everyone's
>>>> >>>> thoughts on committing PRs while its down?  Do we want to wait for
>>>> >>>> Jenkins to come back up, manually run things ourselves and commit?
>>>> >>>>
>>>> >>>> Tom
>>>> >>>>
>>>> >>>> ---------------------------------------------------------------------
>>>> >>>> To unsubscribe e-mail: [hidden email]
>>>> >>>>
>>>> > --
>>>> > Twitter: https://twitter.com/holdenkarau
>>>> > Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9
>>>> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>>
>>>>
>>>>
>>>> --
>>>> Shane Knapp
>>>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>>>> https://rise.cs.berkeley.edu
>>>
>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9
>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau



--
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]