Blind Reviews at RailsConf 2013

One of the reasons I was so pleased to be accepted to speak at RailsConf this year was the conference used blind review. It had been several years since I’d spoken at a conference, and I’d never been selected for a conference as large as RailsConf. Knowing that my talk was accepted on its merits instead of who I am or how much I could burnish my biography made me very proud. (It also led to some hurt feelings from at least one of the usual suspects who wasn’t picked.)

Marty Haught explains their process:

For RailsConf 2013, we used a different approach for reviewing all the proposals we got — all reviews were blind. This is where the reviewer can only see the proposal’s title, abstract and detailed information. We reviewers had no idea who submitted the talk, their experience level or gender/ethnicity. Though I have been organizing conferences in some fashion over the last seven years, this is the first time I’ve done a blind review process.

Before I continue, I want to point out that I strive to not let a person’s background or identity unfairly affect how I rate their talks. I think I do a good job since I truly believe that gender, ethnicity and other ways people are discriminated against play no part in the quality of a presentation they might be offering. However, the reality is I can’t be completely objective. We all carry unconscious bias with us and seeing someone’s name or bio could well color how we value their talk. Thus the first benefit of the blind review is that these things are removed.

Second, I know a lot of people in the community, many of which might be submitting talks. No doubt I could easily be swayed to give extra value to a talk from a good friend or a high quality speaker. Worse yet, I might not give an honest rating since it may hurt their feelings. The blind review eliminates these things as well.

It’s quite freeing to not worry about these things. It allows me as the reviewer to totally focus on just the quality of the abstract and talk details. It also makes talk reviews faster since I don’t need to factor in the quality of the speaker or his/her background. For RailsConf, I only did initial screen and rating of the talks and allowed others to sift through nearly 400 proposals in narrowing down to around 50 talks.

If you organize a conference with a selection process, I encourage you to try blind review. I believe it will help you present a more diverse conference, both in terms of gender and racial diversity, but also in terms of those who are new to speaking and not a regular part of the conference circuit.