XMPP: janlehnardt@jabber.ccc.de
GPG Fingerprint: D2B1 7F9D A23C 0A10 991A F2E3 D9EE 01E4 7852 AEE4
OTR Key: DD971A73 2C16B1AB FDEB5FB4 571DECA1 55F54AD1

Sustainable Open Source

Recently, two major open source projects in the JavaScript world have started pleas to contributions, because they were running out of people to move the projects forward.

Reading the issues and tweets around this, I accidentally a Twitter rant (*). Now I want to write this up with a little more detail.

All of this is based on the premise that the lifeblood of an open source project are the people who work on the project. If you disagree with that premise, you don’t have to keep reading (I’m not saying you are wrong, but you won’t get anything out of this).

Another premise here is that open source projects are largely contributed to by people in their spare time. Of course that’s not all open source projects, and those who have paid full- or part-time staff still benefit from this discussion, but I want to focus on the scenario where people contribute to projects, and create a benefit for the larger community, in their own time.

In the twitter rant, I postulated a list of priorities for open source projects, if they want to be run sustainably. Before I go into the priorities, let me explain what I mean by a sustainable open source project.

Sustainable Open Source Project

A sustainable open source project is one that, during its lifetime, does not run out of contributors. There are two factors here that make projects run out of people:

1. Existing people leave the project.
2. No new people join the project.

To keep people who are already contributing, there are few basics things you need to get right.

1. Care about them as people, rather than producers of contributions (whether it is code, or docs, or designs for the website).

For example: if somebody promised to do something at a certain time, but then life gets in the way: don’t chastise them, and don’t have a culture of where they are even afraid of telling you that they are missing the deadline. Support them in their hardship, be open and understanding of people’s limited time. You don’t have to become their counsellor, but caring about your collaborators as people instead of just the code they commit makes all the difference.

2. Make sure your contributors don’t burn out,

In open source projects there is always a lot to do, more than could ever be done in a reasonable amount of time, even if paid for full-time. That creates pressure for the people who care about moving the project forward, respond to all issues, review all pull requests, write all the docs, etc.

As the people who run an open source project, it is your responsibility that this pressure doesn’t eat up contributors. For example, observe whether people take a break from contributing. As in everything, you must take breaks to be successful long-term. Maybe have a policy where people have to take a week or two off from participating in the project every three months. I’m sure you can think of other ways that would work for you.

3, Make sure there are enough people who, together, can take on the workload of the project, even with people taking regular breaks. If you can’t get enough people together, reduce the scope of the project until it matches what your contributors can handle.

This specifically includes not only writing code, but writing docs, supporting end-users in issues, on Slack/IRC, StackOverflow etc, project marketing (updating the blog/twitter/tumblr), keeping drive-by contributors happy by reviewing pull requests and issues in a timely and friendly manner, and so on. There are ton of time-hogs that are not obvious in open source projects.

For larger projects this even includes press inquiries, legal topics (licensing and naming are just two of the very hairy topics that eat a lot of time, in my experience), coordination with other projects in the space, work on foundations (hello Node.js) etc.

tl;dr: Open Source projects are a lot more than just committing code to a repository somewhere. Make sure you have enough people to handle what you want to achieve.

4. Have a Code of Conduct and be prepared to enforce it. As with all human things, people are bound to disagree. A CoC will bind everyone to keep disagreements civil and productive. And yet, some people won’t play by the rules, or understand the rules in a different way. And with a potentially global audience, cultural difference will make this doubly complicated.

But: you have to have a Code of Conduct that explains what happens if somebody repeatedly violates it. And when that happens, you have to act on what you wrote down. In most extreme cases, this means expelling an individual from the community. This is one of the hardest things I’ve ever done while running an open source project, but it has to be done.

Getting new people to join the project

All of the above will help you retain people, once they are engaged with your project. Here are a few tips on how to make sure you have enough new people coming in to shoulder all the work, account for people taking breaks, and account for others losing interest, or having other commitments.

1. Always be recruiting.

As an open source project, your scarcest resource is people’s time and attention. In order to get anything done, you will need more people. And as everywhere else, recruiting is hard, so it has to be your top priority.

For example: there’s a tough bug in your code, you could sit down for an hour and fix it. Or you could sit down for an hour and write an issue that explains enough of the context of the project so someone else would be able to fix it. Do the latter and you have a chance of recruiting someone, while at the same time more than one person can learn about the project’s internals from the issue, maybe take that part of the issue and put it into your README under “developer documentation.”

There are many, many more opportunities on how to involve new people rather than “quickly doing it yourself”, try to find out which ones apply to your project.

2. Again: Care about the people first and their contributions second.

Naturally, newcomer’s contributions aren’t as complete or far-reaching than those of experienced contributors, so it is doubly important for you care about the people and their enthusiasm about your project more than that typo-fix they put on the website. We’ve turned someone who fixed a single typo on the website to a steady contributor and well respected community member that now helps out all over the project at Hoodie, for example.

At the Apache Software Foundation, where the CouchDB project is run, we have a mantra: Community over Code. — It is a hard learned lesson from running high-profile open source projects like the Apache Web Server, Hadoop or Lucene, for over 20 years now. The reasoning is simple: if the right people rally around an idea, the right code will follow. But without a community who cares, no ever so brilliant piece of code, is worth anything.

This takes very little of your time and makes all the difference.

3. Document your processes.

Have you ever been to a (tech-) event where you didn’t know anyone, but everyone else seemed to stand in groups and chat like old friends? — I’ve been to tons of them, and as an introvert, this is not a lot of fun, and even as an extrovert, this is a challenge.

Now imagine you want to contribute to an open source project where you see some people doing some work, but you don’t see why or how this is happening and where people talk about things.

Here are a few things that happen in your open source project that are handled implicitly. You personally have no problem with this, because you either invented the practices, or were around when they were established, but for a newcomer, these are impenetrable barriers to participating.

Releases. Someone somewhere puts a tag into git, a tarball on a website, or a package into your package manager. Who decides when that happens? What objective criteria need to be fulfilled before a new release is made? Who or what decides whether a particular contribution, feature, or bugfix makes it into the next (or any) release? Who decides on the tag? Who publishes the tarball or package?

If you don’t document all of this in detail, you won’t get anyone ever helping with this. Usually, much of this is very implicit, and by making it explicit by documenting it, you often find that you have some flaw in your procedure, that you can simplify or automate a certain aspect, but most of all, somebody else can join and help.

“Hey, we are stuck on step 3 of the release process, does somebody have time to do 4 and 5?”

Accepting and declining Issues, feature requests and pull requests. Who decides whether an issue is an issue? Who decides whether a particular feature is merged from a pull request, or against the direction the project wants to go in? What are your commit message, code-listing and test-coverage requirements (use GitHub’s CONTRIBUTING.md)?

Put as much of your project’s vision and direction in writing. Of course a lot of this will be in flux as you go along, but if you copiously document this, anyone can jump in at any time and make the right call.

At CouchDB, we even went so far as to document our decision making process as part of our bylaws, which explain how the project operates. If this seems overly tedious bureaucracy to you, you do have a point: this takes time, especially agreeing on everything in large community, but it is very worth it. Since establishing the bylaws, we’ve had numerous new contributors come in and work as part of the project as if they’ve been around forever, following the right procedures to get their initiatives to a successful conclusion and thus making CouchDB a lot better.

Make your communication channels explicit*. At the ASF we have a rule that every project decision must happen on the dev@ mailing list in order for everyone having a chance to read about it in their own time and join the conversation at a central and transparent point.

At the same time, we have a busy IRC channel where we discuss a lot of ideas in a higher-bandwidth environment. However, once we get out of the half-baked stage for anything, we write it up on dev@ and have everyone else who couldn’t e.g. be on IRC at that moment (because they are on the other side of the globe, or at work) catch up with everything and partake in the conversation. Yes, this slows things down a little, but it makes sure everyone has a chance to see everything and be heard.

Whatever your communication channels are, make sure they are documented in your README, on your website and documentation. Make it easy (and free) for people to join and explain the etiquette of the medium.

Special shout out to the FreeBSD project, which is outstanding at documenting their inner workings <3

Specifically reach out to people for “positions” you want to see filled. At Hoodie, we’ve recently decided that we’d like to have a dedicated editorial team that handles our website, twitter, tumblr, etc. because the rest of us didn’t really have the capacities to maintain them all, and previous attempts made us lose two community members getting overwhelmed with the responsibility, so now we are recruiting a team (c.f. taking on only so much that you have enough people to handle it, see above).

As a result, we published a dedicated “job ad” for the position. We are just getting started, but the response has been so overwhelmingly positive that I strongly believe we’ll be very successful going forward with this for other positions. Try it.


In an earlier life I was deeply involved in the PHP community. One of the prime places to get support for problems with PHP was the IRC channel #php on EFnet. IRC and EFnet in particular are notoriously hostile places (I know there are worse IRC networks, I don’t need to be reminded of them), and that culture translated to the #php channel.

I was young and just thought that’s how things are. So I helped people out, and made fun of people who didn’t seem to know there way around (“press F4 to make this chat go faster”). I’m not very proud of being part of a hostile culture in hindsight, but at the time, I thought that’s just how things are.

Over time, I learned how beneficial it is to not antagonise people who are trying to use your open source project. Maybe in the context of PHP it didn’t matter because it was so popular, but later on, this got me thinking.

BIG FAT DISCLAIMER INTERMISSION: This is not a generalisation about the PHP community. As any community, the PHP community goes through its ups and downs and has its good and bad actors, but overall, the people who work on and with PHP are a great bunch that I have learned from half a career’s worth of knowledge and experience. Senior developers have mentored me (a high school student at the time) and helped me understand a lot about the web, not just PHP. I’m eternally grateful for this. INTERMISSION END.

When I started out at CouchDB and we finally joined the ASF and it was standard procedure to have a user@ mailing list for end-user support, I remembered my days in the #php channel and decided that that’s not the culture I want to have there. So for the first three or so years, I tried to be involved with every thread on the mailing list, and showing exemplary behaviour, being nice to people, taking their issues seriously and trying to be helpful overall. After a while, people stuck around not to only ask questions, but to help with the answering as well, and to my complete delight, they mimicked my style.

My responsibilities since have shifted quite a bit and I’m no longer checking on the user@ mailing list day-to-day, but every time I join and read, I see the culture that I helped plant there seven years ago and it makes me very proud.

Culture isn’t just what happens to you, culture is what you make of it.

Whatever you want your culture to be for your open source project’s community, as the people who are most visible, it is up to you to lead by example.


If you follow my premise of people being the biggest asset in open source and recruiting and retaining them is your first priority, then it follows that it would be really easy to grow your contributors.

The ratio of women in IT is about 17-25% (depending on the study), but the ratio of women in open source are 0.1-5%. For me that means: we have spare talent that isn’t participating in our projects that could help with all of the above.

Sadly, we don’t have good numbers (as far as I know, if you have them, please send them along!) for other groups of people who are underrepresented in technology and open source specifically, including, but not limited to, people of colour, LGBTQI+ people, people with disabilities, people who do private care work (for children or elderly family members), or people of lower economic status. All of this section equally applies to all of these groups and I’m just taking women as an example because we have the numbers there. Rest assured, the numbers are worse than they are for women in open source, so we have some work ahead of us.

The summary here for why this disparity exists is that most people in open source are young white guys from a upper-middle-class (in the EU sense) and above, like myself. We all have the privileges that allow us to spend our spare time, or even paid time, on open source projects, but as is evident by the numbers, that privilege doesn’t extend to your average single mother of two, or person of colour who couldn’t land a job in IT, despite the qualification.

One way out of this would be a way for people to get paid to work on open source projects, and especially those people who are underrepresented in tech, because their time needs to be paid for. But this also includes the dedicated individuals who were privileged enough to have enough time to work on an open source project that is now so popular that it demands so much of their time that they can’t do it in their spare time sustainably.

Unfortunately, I have no guidelines for this. Financing open source is an open topic and people have far-reaching opinions from wanting to keep money out of open source at all cost (rimshot) or not, and people who are already raising money for tech projects by marginalised people.

However, this is not a mainstream discussion yet, although smaller initiatives exist, even after the Heartbleed disaster. Even though OpenSSL is comfortably funded now, this was a purely reactionary initiative (that worked well, no qualms about that), but we have no strategy about how to address this and it’s about time that we started having this discussion.

So much of our work and infrastructure depends on Open Source, and it’s time we took the people doing this work as seriously as as the highly financed startups and corporations that depend on of all this freely available work.

Let’s make open source sustainable!

Why Technology Hype is a Good Thing

Hype for technologies is usually seen as a bad thing™.

While I don’t want to deny that there are downsides to this, I also see an upside.

tl;dr: Don’t consider a hype an annoying fashion, but rather a massively distributed mechanism for more quickly figuring out value.

Hype gets a technology and the ideas behind it exposure with a lot of people. Not all ideas are going to be good, and the more people take a good look at one, the earlier we will find out what is good and what isn’t.

History is full of this: XML started out as a document structuring format. But soon after it became popular, people started using it for everything, including computer to computer communication (SOAP, WS-*).

And that was very very very popular. Until we collectively realised that we’ve taken things a little too far. XML is a fine format for the things it was designed to do, but it is not a good generic data protocol format.

Some of the good ideas of XML survive though: Validation is still useful, but it is more optional in the JSON world we live today. There are enough cases where validation is not needed, or would be a prohibitive barrier to entry. We know that now. So we made it opt-in.

Another thing I see hyped these days (for a while now, actually) is Docker. I don’t have too much experience with it, but it appears that people are starting to do everything with Docker. And I’m excited about this. Not because I believe Docker is the end to all means, but because we will collectively find out more quickly what the good and not so good ideas behind Docker are.

The more popular something gets, the more critics it attracts. And in the view of the discourse of community learning and understanding, that’s a good thing. Criticism that has merit will spread, and we will all learn what the good ideas are.

In the JavaScript world, React is all the rage. At JSConf EU, we got a lot of proposals for React-ish talks, which is just another way of saying, people are doing everything with React.

We don’t know yet, what the good ideas behind React are going to be, so we have to have a little patience before things will shake out. But they will.

To hype!

Engaging With Hateful People in Your Community Lends Legitimacy to Their Presence

A woman friend who stays anonymous privately tweeted this discussion earlier. I believe anyone involved with Open Source communities should read and understand this. This is published with her consent.

People, and men in particular, let’s talk about how we deal with people from hate movements showing up in our repos.

OK so for everyone’s context, screencaps of a Gamergater commenting on a Node repo. We need to talk about this.

And to be clear, I’m not shaming @mikeal or @izs (you’re awesome). IMO there’s something broken in our response when male supremacists …

… show up, and men who clearly care about inclusiveness are getting this wrong. So I simply hope my tweets will add useful perspective.

For you men the screencapped exchange might read as
(1) horrible male supremacist argues
(2) rebuttal by @mikeal & @izs
Nothing wrong with (2)

So you might be surprised that to me it reads very different:

(1) male supremacist person shows up - clearly a safety issue
(2) people in the community are actually engaging with this person :(

Note the difference: As a man you might have read the arguments. I literally stopped at the usernames.

(Aside: If the “gr.amergr.ater” username somehow doesn’t spell “misogynistic hate mob” to you, please go read up on this.

This isn’t random internet drama; other people’s safety depends on you understanding this stuff.)

So why do you men get to care about the bigoted arguments and even engage & rebut? Because you’re unlikely to be targeted.

They read as “abhorrent” to you, but not as “threat to your safety”. Good for you!

But for me, the presence of this person is a problem.

When I see a male supremacist show up in an online space, the likelihood that I will participate drops to zero.

Because as much as I care about participating in you all’s projects, no open source project is worth compromising my safety over.

You can have a male supremacist in your online space. You can have me. But you can’t have both.

It’s probably not unreasonable to extrapolate that other women are similarly put off by the presence of male supremacists.

So when you decide to engage & rebut hateful arguments, this comes at the expense of excluding women and minorities from the online space.

Okay, part 2: What’s the right way to deal with male supremacists and similar hate groups showing up?

I don’t have a clear answer. What I care most about is that community members are protected.

Here’s my suggestion #1: Don’t engage. It’s better to instantly block that person from the repo and delete their comments.

GitHub’s combination of “everyone can read”, “everyone can comment”, and “weak moderation features” can make it hard to ban effectively.

Hence suggestion #2: Take discussions (sensitive ones especially) off GitHub completely, to more private, better-moderated spaces.

GitHub’s weaknesses make it not very safe for women and minorities, so if you want those voices heard, avoid the GitHub issue tracker.

By the way: Similar things apply when male supremacists send you reasonable-looking pull requests.

I noticed that this gr.amergr.ate person had sent a small PR to a [my-project] plugin, and the plugin maintainer merged it.

This made me super uncomfortable, and I hope I don’t have to interact with that maintainer, because I really don’t trust their judgment.

When you get a PR from an author whose very name spells hate, then even if the diff looks reasonable, don’t merge it.

Reject it, or ignore it until becomes obsolete. Or even reimplement the fix yourself. Just don’t merge. Let’s not have hate in git log.

The Innovator in Hindsight

In where Clayton Christiensen predicts that Linux on the Desktop will never come and that it will fuel the mobile revolution. In 2004.

Including an introduction to the Innovator’s Dilemma and Solution, and how this quares against Open Source.


The State of CouchDB 2013

This is a rough transcript of the CouchDB Conf, Vancouver Keynote.


Good morning everyone. I thank you all for coming on this fine day in Vancouver. I’m very happy to be here. My name is Jan Lehnardt and I am the Vice President of Apache CouchDB at the Apache Software Foundation, but that’s just a fancy title that means I have to do a bunch of extra work behind the scenes. I’m also a core contributor to Apache CouchDB and I am the longest active committer to the project at this point.

I started helping out with CouchDB in 2006 and that feels like a lifetime ago. We’ve come a long way, we’ve shaped the database industry in a big way, we went though a phoenix from the ashes time and came out still inspiring future generations of developers to do great things.

So it is with great honour that I get to be here on stage before you to take a look at the state of CouchDB.


I’d like to start with some numbers:

Commit History

We have made a lot of changes in 2012 to make 2013 a great year for CouchDB and it sure looks like we succeeded and that 2014 is only going to trump that.

I’d like to thank everyone on the team for their hard work.


We’ve just shipped CouchDB 1.5.0 last week and it comes with a few exciting new things as previews, for you to try out and play with and report any issues with back to us. And that is on top of all the regular bug fixing and other improvements.


  1. A completely new developed admin UI, nicknamed Fauxton, that is poised to replace the much-loved, but increasingly dated Futon. I’d like to personally thank the Fauxton team: Sue “Deathbear” Lockwood, Russell “Chewbranca” Branca, Garren Smith and many more volunteers for their work as well as the company Cloudant for sponsoring a good chunk of that work. Great job everyone! Fauxton is going to be replacing Futon in one of the next few releases and will give us the foundation for the next stage of CouchDB’s life.

  2. Plugins. While it was always possible to write plugins for CouchDB, you kind of had to be an expert in CouchDB to get started. We believe that writing plugins is a great gateway drug to getting more people to hack on CouchDB proper, so we made it simpler to build plugins and to install plugins into a running instance of CouchDB. It is still very early days, we don’t even have a plugin registry yet, but we are surely excited about the prospects of installing GeoCouch with a single click of a button in Futon or Fauxton. We also included a template plugin that you can easily extend and make your own, along with a guide to get you started.

The plugins effort also supports a larger trend we are starting to follow with the CouchDB core codebase: decide on a well-defined core set of functionality and delegate more esoteric things to a rich plugin system That means we no longer have to decline the inclusion of useful code like we’ve done in the past, because it wasn’t applicable to the majority of CouchDB users. Now we can support fringe features and plugins that are only useful to a few of our users, but who really need them.

  1. A Node.JS query server. CouchDB relies on JavaScript for a number of core features and we want to continue to do so. In order to keep up with the rapid improvements made to the JavaScript ecosystem we have tentative plans to switch from a Spidermonkey-driven query server to a V8-driven one. In addition, the Node.js project has a really good installation story, something that we had trouble with in the past, and includes a few utilities that make it very easy for us to switch the query server over.

All this however is not to blindly follow the latest trends, but to encourage the community to take on the query server and introduce much needed improvements. The current view server is a tricky mix of JS, Erlang and C and we are not seeing many people daring to jump into that. In a second step we expect these improvements to trickle down to the other query server implementations like Python or PHP and make things better for everyone. For now this is also a developer preview and we are inviting all Node.js developers to join us and build a a better query server.


  1. Docs landed in 1.4.0, but 1.5.0 is seeing a major update to the now built-in documentation system. With major thanks to Alexander Shorin, Dirkjan Ochtmann and Dave Cottlehuber who were instrumental in that effort, CouchDB now has “really good docs” instead of a “really crappy wiki”, that are shipped with every release and are integrated with Futon and Fauxton.


The immediate next area of focus for the CouchDB project is the merging of two forks: BigCouch and rcouch.

BigCouch is a Dynamo implementation on top of CouchDB that manages a cluster of machines and makes them look as a single one, adding performance improvements and fault tolerance to a CouchDB installation. This is a major step in CouchDB’s evolution as it was designed for such a system from the start, but the core project never included a way to use and manage a cluster. Cloudant have donated their BigCouch codebase to the Apache project already and we are working on an integration.

rcouch is a what I would call a “future port” of CouchDB by longtime committer and contributor Benoit Chesneau. rcouch looks like CouchDB would, if we started fresh today with a modern architecture. Together with BigCouch’s improvements, this will thoroughly modernise CouchDB’s codebase to the latest state of the art of Erlang projects. rcouch also includes a good number of nifty features that make a great addition to CouchDB’s core feature set and some great plugins.

Finally, we’ve just started an effort to set up infrastructure and called for volunteers to translate the CouchDB documentation and admin interface into all major languages. Driven by Andy Wenk from Hamburg, we already have a handful of people signed up to help with translations for a number of different languages.

This is going to keep us busy for a bit and we are looking forward to ship some great releases with these features.


2013 was a phenomenal year for Apache CouchDB. 2014 is poised to be even greater, there are more people than ever pushing CouchDB forward and there is plenty of stuff to do and hopefully, we get to shape some more of the future of computing.

Thank you!

Archive →