Fwd: Beam's recent community development work

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Fwd: Beam's recent community development work

Sean Owen-3
Worth, I think, a read and consideration from Spark folks. I'd be interested in comments; I have a few reactions too.

---------- Forwarded message ---------
From: Kenneth Knowles <[hidden email]>
Date: Sat, Jun 30, 2018 at 1:15 AM
Subject: Beam's recent community development work
To: <[hidden email]>, <[hidden email]>, Griselda Cuevas <[hidden email]>, dev <[hidden email]>


Hi all,

The ASF board suggested that we (Beam) share some of what we've been doing for community development with [hidden email] and [hidden email]. So here is a long description. I have included [hidden email] because it is the subject, really, and this is & should be all public knowledge.

We would love feedback! We based a lot of this on reading the community project site, and probably could have learned even more with more study.

# Background

We face two problems in our contributor/committer-base:

1. Not enough committers to review all the code being contributed, in part due to recent departure of a few committers
2. We want our contributor-base (hence committer-base) to be more spread across companies and backgrounds, for the usual Apache reasons. Our user base is not active and varied enough to make this automatic. One solution is to make the right software to get a varied user base, but that is a different thread :-) so instead we have to work hard to build our community around the software we have.

# What we did

## Committer guidelines

We published committer guidelines [1] for transparency and as an invitation. We start by emphasizing that there are many kinds of contributions, not just code (we have committers from community development, tech writing, training, etc). Then we have three aspects:

1. ASF code of conduct
2. ASF committer responsibilities
3. Beam-specific committer responsibilities

The best way to understand is to follow the link at the bottom of this email. The important part is that you shouldn't be proposing a committer for other reasons, and you shouldn't be blocking a committer for other reasons.

## Instead of just "[DISCUSS] Potential committer XYZ" we discuss every layer

Gris (CC'd) outlined this: people go through these phases of relationship with our project:

1. aware of it
2. interested in it / checking it out
3. using it for real
4. first-time contributor
5. repeat contributor
6. committer
7. PMC

As soon as we notice someone, like a user asking really deep questions, we invite discussion on private@ on how we can move them to the next level of engagement.

## Monthly cadence

Every ~month, we call for new discussions and revisit ~all prior discussions. This way we do not forget to keep up this effort.

## Individual discussions

For each person we have a separate thread on private@. This ensures we have quality focused discussions that lead to feedback. In collective discussions that we used to do, we often didn't really come up with actionable feedback and ended up not even contacting potential committers to encourage them. And consensus was much less clear.

## Feedback!

If someone is brought up for a discussion, that means they got enough attention that we hope to engage them more. But unsolicited feedback is never a good idea. For a potential committer, we did this:

1. Send an email saying something like "you were discussed as a potential committer - do you want to become one? do you want feedback?"
2. If they say yes (so far everyone) we send a few bullet points from the discussion and *most important* tie each bullet to the committer guidelines. If we have no feedback about which guidelines were a concern, that is a red flag that we are being driven by bias.

We saw a *very* significant increase in engagement from those we sent feedback to, and the trend is that they almost all will become committers over time.

## Results

 - Q1 we had no process and we added no new committers or PMC members.
 - Q2 when we tried these new things we added 7 committers and 1 PMC member, with ~3~4 obvious committer candidates for next time around, plus positive feedback from other contributors, spread across five companies.

We've had a pretty major uptick in building Beam as a result.

## Review-then-commit

We are dedicated to RTC as the best way to build software. Reviews not only make the code better, but (with apologies to ASF/GNU differences) as RMS says "The fundamental act of friendship among programmers is the sharing of programs" and reviews are where we do that.

As a minor point, we also changed our "review-then-commit" policy to require that *either* the reviewer or the author be a committer. Previously the reviewer had to be a committer. Rationale: if we trust someone as a committer, we should trust their choice of reviewer. This also helps the community, as it engages non-committers as reviewers.

----

If you made it through this long email, thanks for reading!

Kenn

Reply | Threaded
Open this post in threaded view
|

Re: Beam's recent community development work

Holden Karau
As someone who floats a bit between both projects (as a contributor) I'd love to see us adopt some of these techniques to be pro-active about growing our committer-ship (I think perhaps we could do this by also moving some of the newer committers into the PMC faster so there are more eyes out looking for people to bring forward)?

On Mon, Jul 2, 2018 at 4:54 PM, Sean Owen <[hidden email]> wrote:
Worth, I think, a read and consideration from Spark folks. I'd be interested in comments; I have a few reactions too.


---------- Forwarded message ---------
From: Kenneth Knowles <[hidden email]>
Date: Sat, Jun 30, 2018 at 1:15 AM
Subject: Beam's recent community development work
To: <[hidden email]>, <[hidden email]>, Griselda Cuevas <[hidden email]>, dev <[hidden email]>


Hi all,

The ASF board suggested that we (Beam) share some of what we've been doing for community development with [hidden email] and [hidden email]. So here is a long description. I have included [hidden email] because it is the subject, really, and this is & should be all public knowledge.

We would love feedback! We based a lot of this on reading the community project site, and probably could have learned even more with more study.

# Background

We face two problems in our contributor/committer-base:

1. Not enough committers to review all the code being contributed, in part due to recent departure of a few committers
2. We want our contributor-base (hence committer-base) to be more spread across companies and backgrounds, for the usual Apache reasons. Our user base is not active and varied enough to make this automatic. One solution is to make the right software to get a varied user base, but that is a different thread :-) so instead we have to work hard to build our community around the software we have.

# What we did

## Committer guidelines

We published committer guidelines [1] for transparency and as an invitation. We start by emphasizing that there are many kinds of contributions, not just code (we have committers from community development, tech writing, training, etc). Then we have three aspects:

1. ASF code of conduct
2. ASF committer responsibilities
3. Beam-specific committer responsibilities

The best way to understand is to follow the link at the bottom of this email. The important part is that you shouldn't be proposing a committer for other reasons, and you shouldn't be blocking a committer for other reasons.

## Instead of just "[DISCUSS] Potential committer XYZ" we discuss every layer

Gris (CC'd) outlined this: people go through these phases of relationship with our project:

1. aware of it
2. interested in it / checking it out
3. using it for real
4. first-time contributor
5. repeat contributor
6. committer
7. PMC

As soon as we notice someone, like a user asking really deep questions, we invite discussion on private@ on how we can move them to the next level of engagement.

## Monthly cadence

Every ~month, we call for new discussions and revisit ~all prior discussions. This way we do not forget to keep up this effort.

## Individual discussions

For each person we have a separate thread on private@. This ensures we have quality focused discussions that lead to feedback. In collective discussions that we used to do, we often didn't really come up with actionable feedback and ended up not even contacting potential committers to encourage them. And consensus was much less clear.

## Feedback!

If someone is brought up for a discussion, that means they got enough attention that we hope to engage them more. But unsolicited feedback is never a good idea. For a potential committer, we did this:

1. Send an email saying something like "you were discussed as a potential committer - do you want to become one? do you want feedback?"
2. If they say yes (so far everyone) we send a few bullet points from the discussion and *most important* tie each bullet to the committer guidelines. If we have no feedback about which guidelines were a concern, that is a red flag that we are being driven by bias.

We saw a *very* significant increase in engagement from those we sent feedback to, and the trend is that they almost all will become committers over time.

## Results

 - Q1 we had no process and we added no new committers or PMC members.
 - Q2 when we tried these new things we added 7 committers and 1 PMC member, with ~3~4 obvious committer candidates for next time around, plus positive feedback from other contributors, spread across five companies.

We've had a pretty major uptick in building Beam as a result.

## Review-then-commit

We are dedicated to RTC as the best way to build software. Reviews not only make the code better, but (with apologies to ASF/GNU differences) as RMS says "The fundamental act of friendship among programmers is the sharing of programs" and reviews are where we do that.

As a minor point, we also changed our "review-then-commit" policy to require that *either* the reviewer or the author be a committer. Previously the reviewer had to be a committer. Rationale: if we trust someone as a committer, we should trust their choice of reviewer. This also helps the community, as it engages non-committers as reviewers.

----

If you made it through this long email, thanks for reading!

Kenn




--
Reply | Threaded
Open this post in threaded view
|

Re: Beam's recent community development work

rxin
That's fair, and it's great to find high quality contributors. But I also feel the two projects have very different background and maturity phase. There are 1300+ contributors to Spark, and only 300 to Beam, with the vast majority of contributions coming from a single company for Beam (based on my cursory look at the two pages of commits on github). With the recent security and correctness storms, I actually worry about more quality (which requires more infrastructure) than just people adding more code to the project.



On Mon, Jul 2, 2018 at 5:25 PM Holden Karau <[hidden email]> wrote:
As someone who floats a bit between both projects (as a contributor) I'd love to see us adopt some of these techniques to be pro-active about growing our committer-ship (I think perhaps we could do this by also moving some of the newer committers into the PMC faster so there are more eyes out looking for people to bring forward)?

On Mon, Jul 2, 2018 at 4:54 PM, Sean Owen <[hidden email]> wrote:
Worth, I think, a read and consideration from Spark folks. I'd be interested in comments; I have a few reactions too.


---------- Forwarded message ---------
From: Kenneth Knowles <[hidden email]>
Date: Sat, Jun 30, 2018 at 1:15 AM
Subject: Beam's recent community development work
To: <[hidden email]>, <[hidden email]>, Griselda Cuevas <[hidden email]>, dev <[hidden email]>


Hi all,

The ASF board suggested that we (Beam) share some of what we've been doing for community development with [hidden email] and [hidden email]. So here is a long description. I have included [hidden email] because it is the subject, really, and this is & should be all public knowledge.

We would love feedback! We based a lot of this on reading the community project site, and probably could have learned even more with more study.

# Background

We face two problems in our contributor/committer-base:

1. Not enough committers to review all the code being contributed, in part due to recent departure of a few committers
2. We want our contributor-base (hence committer-base) to be more spread across companies and backgrounds, for the usual Apache reasons. Our user base is not active and varied enough to make this automatic. One solution is to make the right software to get a varied user base, but that is a different thread :-) so instead we have to work hard to build our community around the software we have.

# What we did

## Committer guidelines

We published committer guidelines [1] for transparency and as an invitation. We start by emphasizing that there are many kinds of contributions, not just code (we have committers from community development, tech writing, training, etc). Then we have three aspects:

1. ASF code of conduct
2. ASF committer responsibilities
3. Beam-specific committer responsibilities

The best way to understand is to follow the link at the bottom of this email. The important part is that you shouldn't be proposing a committer for other reasons, and you shouldn't be blocking a committer for other reasons.

## Instead of just "[DISCUSS] Potential committer XYZ" we discuss every layer

Gris (CC'd) outlined this: people go through these phases of relationship with our project:

1. aware of it
2. interested in it / checking it out
3. using it for real
4. first-time contributor
5. repeat contributor
6. committer
7. PMC

As soon as we notice someone, like a user asking really deep questions, we invite discussion on private@ on how we can move them to the next level of engagement.

## Monthly cadence

Every ~month, we call for new discussions and revisit ~all prior discussions. This way we do not forget to keep up this effort.

## Individual discussions

For each person we have a separate thread on private@. This ensures we have quality focused discussions that lead to feedback. In collective discussions that we used to do, we often didn't really come up with actionable feedback and ended up not even contacting potential committers to encourage them. And consensus was much less clear.

## Feedback!

If someone is brought up for a discussion, that means they got enough attention that we hope to engage them more. But unsolicited feedback is never a good idea. For a potential committer, we did this:

1. Send an email saying something like "you were discussed as a potential committer - do you want to become one? do you want feedback?"
2. If they say yes (so far everyone) we send a few bullet points from the discussion and *most important* tie each bullet to the committer guidelines. If we have no feedback about which guidelines were a concern, that is a red flag that we are being driven by bias.

We saw a *very* significant increase in engagement from those we sent feedback to, and the trend is that they almost all will become committers over time.

## Results

 - Q1 we had no process and we added no new committers or PMC members.
 - Q2 when we tried these new things we added 7 committers and 1 PMC member, with ~3~4 obvious committer candidates for next time around, plus positive feedback from other contributors, spread across five companies.

We've had a pretty major uptick in building Beam as a result.

## Review-then-commit

We are dedicated to RTC as the best way to build software. Reviews not only make the code better, but (with apologies to ASF/GNU differences) as RMS says "The fundamental act of friendship among programmers is the sharing of programs" and reviews are where we do that.

As a minor point, we also changed our "review-then-commit" policy to require that *either* the reviewer or the author be a committer. Previously the reviewer had to be a committer. Rationale: if we trust someone as a committer, we should trust their choice of reviewer. This also helps the community, as it engages non-committers as reviewers.

----

If you made it through this long email, thanks for reading!

Kenn




--
Reply | Threaded
Open this post in threaded view
|

Re: Beam's recent community development work

Matei Zaharia
Administrator
I think telling people that they’re being considered as committers early on is a good idea, but AFAIK we’ve always had individual committers do that with contributors who were doing great work in various areas. We don’t have a centralized process for it though — it’s up to whoever wants to work with each contributor.

Matei

> On Jul 2, 2018, at 5:35 PM, Reynold Xin <[hidden email]> wrote:
>
> That's fair, and it's great to find high quality contributors. But I also feel the two projects have very different background and maturity phase. There are 1300+ contributors to Spark, and only 300 to Beam, with the vast majority of contributions coming from a single company for Beam (based on my cursory look at the two pages of commits on github). With the recent security and correctness storms, I actually worry about more quality (which requires more infrastructure) than just people adding more code to the project.
>
>
>
> On Mon, Jul 2, 2018 at 5:25 PM Holden Karau <[hidden email]> wrote:
> As someone who floats a bit between both projects (as a contributor) I'd love to see us adopt some of these techniques to be pro-active about growing our committer-ship (I think perhaps we could do this by also moving some of the newer committers into the PMC faster so there are more eyes out looking for people to bring forward)?
>
> On Mon, Jul 2, 2018 at 4:54 PM, Sean Owen <[hidden email]> wrote:
> Worth, I think, a read and consideration from Spark folks. I'd be interested in comments; I have a few reactions too.
>
>
> ---------- Forwarded message ---------
> From: Kenneth Knowles <[hidden email]>
> Date: Sat, Jun 30, 2018 at 1:15 AM
> Subject: Beam's recent community development work
> To: <[hidden email]>, <[hidden email]>, Griselda Cuevas <[hidden email]>, dev <[hidden email]>
>
>
> Hi all,
>
> The ASF board suggested that we (Beam) share some of what we've been doing for community development with [hidden email] and [hidden email]. So here is a long description. I have included [hidden email] because it is the subject, really, and this is & should be all public knowledge.
>
> We would love feedback! We based a lot of this on reading the community project site, and probably could have learned even more with more study.
>
> # Background
>
> We face two problems in our contributor/committer-base:
>
> 1. Not enough committers to review all the code being contributed, in part due to recent departure of a few committers
> 2. We want our contributor-base (hence committer-base) to be more spread across companies and backgrounds, for the usual Apache reasons. Our user base is not active and varied enough to make this automatic. One solution is to make the right software to get a varied user base, but that is a different thread :-) so instead we have to work hard to build our community around the software we have.
>
> # What we did
>
> ## Committer guidelines
>
> We published committer guidelines [1] for transparency and as an invitation. We start by emphasizing that there are many kinds of contributions, not just code (we have committers from community development, tech writing, training, etc). Then we have three aspects:
>
> 1. ASF code of conduct
> 2. ASF committer responsibilities
> 3. Beam-specific committer responsibilities
>
> The best way to understand is to follow the link at the bottom of this email. The important part is that you shouldn't be proposing a committer for other reasons, and you shouldn't be blocking a committer for other reasons.
>
> ## Instead of just "[DISCUSS] Potential committer XYZ" we discuss every layer
>
> Gris (CC'd) outlined this: people go through these phases of relationship with our project:
>
> 1. aware of it
> 2. interested in it / checking it out
> 3. using it for real
> 4. first-time contributor
> 5. repeat contributor
> 6. committer
> 7. PMC
>
> As soon as we notice someone, like a user asking really deep questions, we invite discussion on private@ on how we can move them to the next level of engagement.
>
> ## Monthly cadence
>
> Every ~month, we call for new discussions and revisit ~all prior discussions. This way we do not forget to keep up this effort.
>
> ## Individual discussions
>
> For each person we have a separate thread on private@. This ensures we have quality focused discussions that lead to feedback. In collective discussions that we used to do, we often didn't really come up with actionable feedback and ended up not even contacting potential committers to encourage them. And consensus was much less clear.
>
> ## Feedback!
>
> If someone is brought up for a discussion, that means they got enough attention that we hope to engage them more. But unsolicited feedback is never a good idea. For a potential committer, we did this:
>
> 1. Send an email saying something like "you were discussed as a potential committer - do you want to become one? do you want feedback?"
> 2. If they say yes (so far everyone) we send a few bullet points from the discussion and *most important* tie each bullet to the committer guidelines. If we have no feedback about which guidelines were a concern, that is a red flag that we are being driven by bias.
>
> We saw a *very* significant increase in engagement from those we sent feedback to, and the trend is that they almost all will become committers over time.
>
> ## Results
>
>  - Q1 we had no process and we added no new committers or PMC members.
>  - Q2 when we tried these new things we added 7 committers and 1 PMC member, with ~3~4 obvious committer candidates for next time around, plus positive feedback from other contributors, spread across five companies.
>
> We've had a pretty major uptick in building Beam as a result.
>
> ## Review-then-commit
>
> We are dedicated to RTC as the best way to build software. Reviews not only make the code better, but (with apologies to ASF/GNU differences) as RMS says "The fundamental act of friendship among programmers is the sharing of programs" and reviews are where we do that.
>
> As a minor point, we also changed our "review-then-commit" policy to require that *either* the reviewer or the author be a committer. Previously the reviewer had to be a committer. Rationale: if we trust someone as a committer, we should trust their choice of reviewer. This also helps the community, as it engages non-committers as reviewers.
>
> ----
>
> If you made it through this long email, thanks for reading!
>
> Kenn
>
> [1] https://beam.apache.org/contribute/become-a-committer/
>
>
>
> --
> Twitter: https://twitter.com/holdenkarau


---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]