Part 6: A Conversation with Bill and Rocco (aka: “How the Sausage is Rated”)

To help commemorate its 25th anniversary, the Entertainment Software Rating Board (ESRB) reached out to Blake J. Harris, the best-selling author of Console Wars: Sega, Nintendo, and the Battle That Defined a Generation and The History of the Future: Oculus, Facebook and the Revolution That Swept Virtual Reality to document the behind-the-scenes origins of the rating system for video games and how it has evolved over the past quarter century. Content Rated By: An Oral History of the ESRB provides eyewitness accounts from the key people involved in the ESRB’s creation and its development into one of the country’s finest examples of industry self-regulation. 

PART 6: Behind-the-Scenes with Bill & Rocco (aka: “How the Sausage is Rated”)

Bill Garrity and Rocco Buffalino have worked closely together for over a decade at the ESRB—to the point where it can seem like they may spend more time together than they do apart. Garrity is currently the organization’s Senior Vice President of Ratings; Buffalino is its Senior Director of Rating Submissions and Compliance.

ROCCO BUFFALINO (ESRB): So I’m a lifelong gamer. I live, eat, drink, breathe video games. It’s my passion, it’s what I do, it’s what I care about.

BILL GARRITY (ESRB): He’s not kidding. The dude has a PO Box in Japan so he can get Japanese stuff and ship it here. He has Japanese credit cards.

ROCCO BUFFALINO (ESRB): I just got a box [delivered] today.

BILL GARRITY (ESRB): I mean, you just got a book in Japanese today about [Satoru] Iwata [the late president and CEO of Nintendo], and you don’t even read Japanese!

ROCCO BUFFALINO (ESRB): Just gotta point Google Translate on it and brute force my way through. So how I got here… this was back in 2005. I had a friend who was working here—I didn’t know him super well—but he knew that I was a big games guy, and they were hiring for, I believe, a Manager of Ratings Submissions at the time.

BILL GARRITY (ESRB): Sounds about right.

ROCCO BUFFALINO (ESRB): Yeah, so I applied for the job and got an interview with Bill. I came into the interview with a college paper I had written about the ESRB for an electronic media policy class. The paper was about 50 pages long and was about the organization and what they did and stuff. I think they thought that was really cool.

BILL GARRITY (ESRB): Yeah, we definitely did! He came in, basically with this book report. And he knew the whole cast of characters [working at the ESRB] and was sort of looking around the hallways for people’s faces he might recognize.

ROCCO BUFFALINO (ESRB): I came prepared.

BILL GARRITY (ESRB): He was the most prepared person we’ve ever hired. And it’s borne out over the past 12…13 years?

ROCCO BUFFALINO (ESRB): I’m in my 14th year.

BILL GARRITY (ESRB): I started in 2002. Before that, I was doing some web projects for the National Multiple Sclerosis Society. Yeah… so I’m on my 17th year… Whew….

Bill and Rocco’s tenure at ESRB has instilled in them a comprehensive knowledge of, and respect for, ESRB’s two rating processes.

ROCCO BUFFALINO (ESRB): I guess the first question is: how are you distributing that game? If you’re publishing a physical product, you’ll need to get in touch with us and fill out our “Long Form.” And the Long Form—well, you may describe it as “onerous.” But it’s not impossible! And there are a lot of people here who are happy to walk you through it.

BILL GARRITY (ESRB): That’s 100% true. We’re not some empty void of people who you can never get a hold of. Not at all. If you’ve got questions, we’ve got answers.

ROCCO BUFFALINO (ESRB): We do a lot of hand holding in the beginning. Because our process can be intimidating for somebody who’s never dealt with it, who has no idea what they need to do, doesn’t understand the requirements. You’re looking at a submissions guide that’s many, many pages, and, you know, it’s not the easiest thing if you’re like, a one-man development team. But we have people here, myself included, who will literally walk you through every step of the way.

BILL GARRITY (ESRB): And we urge you to take advantage of this “hand holding.” Because if you fill out the Long Form incorrectly—

ROCCO BUFFALINO (ESRB): —or it’s missing pertinent information…

BILL GARRITY (ESRB): It could end up resulting in a violation.

ROCCO BUFFALINO (ESRB): But we do have warning points built into the system so that we’re never—out of nowhere—dropping the hammer on you. Bottom line: it is called the “Long Form,” but we’ve been continually refining the process for over 15 years so it’s much smoother and easier than it’s ever been. And like I said earlier: many of us are available almost all the time to answer any sort of question—to help every publisher, big or small…

The Long Form process technically starts before ESRB even sees the game.

The Long Form Rating Process

Step 1: Request Access

ROCCO BUFFALINO (ESRB): You [need to] apply for access via our website, and—within the “Publishers” section of our website—there’s a submission guide that kind of tells you what you need to do to get your game rated.

Step 2: Where the “Long Form” Comes In

ROCCO BUFFALINO (ESRB): You fill [the form] out, you tell us about your game in the different, pertinent content categories: violence, sex, language, gambling, etc.  And they’re kind of like long essay questions. You tell us about the content of your game and, concurrently, you…

Step 3: “Prepare a Video”

BILL GARRITY (ESRB): For the video component, obviously, it’s not a video of your entire game. The video should consist of “pertinent content”—which is a defined term. It’s basically the most extreme content from a variety of categories: the most extreme violence, the most extreme sex, etc. But it also has to be in context.

ROCCO BUFFALINO (ESRB): That’s an important distinction to make. We don’t want to just see a decapitation montage. We want to see the full context so we know why those heads are being cut off. We want to see the buildup, we want to see the atmosphere.

BILL GARRITY (ESRB): Some people think that when we’re rating a game, what we do is play it through from start to finish and then, after we’re done, we give it a rating. But that’s not actually how it works. During the rating process our raters don’t actually play the game. Their evaluation is based purely on your video and any potential written supplements. Now, just to be clear, we do test games after they’re released. For retail games, we’ll usually do about four or five hours of play through, but the video you submit is critical to the rating process.

ROCCO BUFFALINO (ESRB): Like we said, there can be steep fines for being…less than truthful. Anyway, so you prepare that video. You send it to us in high definition through our secure FTP. The internal staff gets your submission materials. At this point, you’ll also probably need to pay the necessary fees and agree to our terms and conditions. And then we get all those materials, we go through what you sent us. We vet it, we make sure it’s okay, that what’s been disclosed in writing is also on the video and vice versa; and then we basically polish it up to a sheen and…

Step 4: “Put it in Front of Raters”

BILL GARRITY (ESRB): We have two full-time raters that are here all the time. And then we have a pool of about five part-timers currently.

The identities of all raters are a tightly held secret.

BILL GARRITY (ESRB): The publishers don’t even know their names. This way, there’s a total firewall. We have a minimum of three raters per product. They watch all the pertinent content that’s in your video. During this part of the process, the raters are not allowed to communicate with each other. They’ll read what they need to read, they’ll watch what they need to watch, and then—before they all come together—each of them will put down a “pre-rating.” And then they’ll all talk, share what they think, and after they’re done they’ll put down a “post- rating.” The reason we have them put down a pre-rating is because if we only did a post-rating there’s a temptation to agree with the other raters—to “go along to get along.” We want them to take a stance beforehand. This way they’ll have a little “skin in the game.” After they’ve all talked, we have a meeting in my office. One of the raters is designated “Foreman” of the session and they’ll come in and explain it to myself and James [Chang, Director of Rating Assignments].

Step 5: Have a Content Conversation and Finalize the Rating

This is when the real decisions get made, and previously rated games with similar content become an essential part of the equation.

BILL GARRITY (ESRB): James is basically the parity guy who’s got all the info about how we’ve rated similar products in the past. Parity is important. One way we have built trust with consumers is by being consistent. Anyway, he’ll say something like: okay, these are World War II shooters, first-person, minimal blood, this-kind-of- depiction; and we’ve said that’s “Teen, Blood, Violence” for the past 12 years. You’re saying it’s “Mature;” why do you feel that way? We do have the ability to over-rule the raters’ decision, but that happens very rarely. Most of the time these meetings end with everyone agreeing on the decision. Then, after a decision is made, we build the “cert” and it gets emailed out to the game maker.

We’re always weighing our decisions against past precedent. Everything is about defensibility—making sure it makes sense, making sure it’s a robust decision. That you can say there’s a reason for every decision. It can’t just be as simple as “Oh, I see some nudity” and nudity gets you this rating. It needs to be more like: “Okay, let’s look at the spectrum.” You look for precedents to previous evaluations and then you analyze the differences. Like: yes, similar content is in another game, but this one has more voyeuristic elements. And those are the sort of discussions that you have. I realize that as I say all this right now, it might sound a little informal. But when we’re having these types of conversations, it’s all very clinical and analytical. That’s how it needs to be. Because the decisions we end up making don’t just impact the game we’re rating that day; but we’re also creating context and setting precedents for future evaluations.

ROCCO BUFFALINO (ESRB): And that’s pretty much it in a nutshell.

Given the trajectory of the industry, especially the explosion of digital and mobile games coming into the market, the ESRB developed a revolutionary new process that it now uses for all games released without a physical component—and got other rating boards around the world to use it too.

…

Read Part 7: Think Globally, Act Digitally >>

The next two parts of Content Rated By: An Oral History of the ESRB give more insight into how the ESRB created a scalable global rating solution to meet the needs of mobile app and other digital game storefronts. Part 7 will be published next week in the ESRB About section.

Blake J. Harris is the best-selling author of Console Wars: Sega, Nintendo and the Battle that Defined a Generation, which is currently being adapted for television by Legendary Entertainment.