How is team-member-1 doing?

The engineer that gave the unfortunate command to delete our primary database was not only on our minds but also of other people. He's known by the community as "team-member-1", as we referred to him by this expression in our public communications during the incident.

After we posted the postmortem of the incident with GitLab.com, we received notes from our community asking how was team-member-1 doing. We're here to tell you that.

We are still putting all our efforts into improving GitLab.com's infrastructure as a whole, to ensure this type of incident never happens again.

#HugOps

We were all really touched by the love our community sent us via #HugOps:

At the end of the day, we are all team-member-1:

Support from our community

The support our engineers received from our community was fantastic, people were appreciative of our transparency in working on the solution. Even teams from other companies, who had been in that situation themselves, showed their support.

❤️  Thank you all! We're very touched by your support!  ❤️

The Codefresh team brought cookies to our Experience Center in San Francisco with a card saying:

Hey GitLab, We thought you could use a chance to destress a little. Don't sweat it we're rooting for you. We've all been there before and we have a lot of faith in you to work it out. You got THIS! From Dan, and the Codefresh Team

A group from Google came to our Experience Center and got our engineers $300 to spend on something to make them feel better after all of this was over:

Yay for you! We've all been there! :) From friends of yours at Google

❤️  Thank you Codefresh and Google!  ❤️

The engineers involved have agreed that this extremely generous gift from Google will be spent on sponsoring Rails Girls events. But the cookies were obviously eaten by the one who grabbed the pack! 😛

Needless to say, some special thank you swag is on its way to these amazing people that took the time to come to our boardroom and try to help us feel a bit better.

GitLab Values

At GitLab we think that the people making the most mistakes frequently correlate with the people doing the most work. I certainly make a lot of mistakes every day. It is important not to double down on them but to acknowledge them and learn from it.

We make mistakes. What's different from person to person, organization to organization, is how to deal with them. What we value most at GitLab is:

• Transparency: "Don't be afraid to admit you made a mistake or were wrong. When something went wrong it is a great opportunity to say 'What’s the kaizen moment here?' and find a better way without hurt feelings."

• Collaboration: "Say sorry if you made a mistake apologize. Saying sorry is not a sign of weakness but one of strength. The people that do the most will likely make the most mistakes."
• Behaviour:
  • Be truthful and honest.
  • Be dependable, reliable, fair, and respectful.
  • Be committed, creative, inspiring, and passionate.

How could we value transparency if our team members were afraid of assuming their mistakes?

It is not our intent to have one of our team members implicated by the transparency. (...) We are very aware of the stress that such a mistake might cause and the rest of the team has been very supportive. (...) We recognize the risk to the company of being transparent, but your values are defined by what you do when it is hard. Sid Sijbrandij, CEO.

Team-member-1

While we were putting the fire out, we received this comment:

The GitLab engineer (team-member-1) is one of the smartest people he's ever known - in fact, he's actually brilliant. So you can rest assured that the data loss wasn't caused by a an inexperienced kid. (HN)

And this thoughtful tweet:

We heard you Rubén! :)

Yes, team-member-1 is doing very well!

Coincidentally, just before the DB incident, team-member-1 had qualified for a promotion to senior developer. The outage did not change that decision.

Promotions are to be based on meeting the criteria of the role the individual is to be promoted in to (i.e. promote based on performance) - GitLab PeopleOps Handbook

When we promote people at GitLab, or give a them a bonus, we share the reasons for that with the whole company. With the permission of team-member-1, this blog post is both the internal and external announcement of that promotion.

Reasons for promoting team-member-1

Pablo Carranza (Production Lead), provided GitLab with the following reasons to promote team-member-1:

We support each other

For those developers involved in the outage, we made a special T-shirt:

This T-shirt has two purposes. It reminds us what happened and motivates us not to let this happen again. It was also meant to thank the team that handled the incident, for that reason only they have gotten one.

Wrapping Up

At GitLab, we value most positive achievements and performance improvements of our team members, instead of focusing our attention on negative random situations.

Of course, we take situations like this very seriously, but we'd rather learn from them, and put all our efforts to avoid that they happen again, than punish honest and talented people for mistakes that can happen to anyone.

What doesn't kill you makes you stronger. (HN)