How One Line of Ancient Code Haunted a Major MMO For Six Months
'Aliens: Colonial Marines' isn't the only game to have shipped with problems the developers would only figure out much later.
Image courtesy of EA
One of the most common refrains you hear from game developers is a simple notion: It’s shocking any game is released. They are complicated, unwieldy beasts held together by the digital equivalent of duct tape. It’s no surprise when they fall apart.
Aliens: Colonial Marines was a troubled game for many reasons, but it quickly became a laughing stock when clips like this, showing the game’s xenomorphs aimlessly wandering, went viral, undermined its premise. Earlier this month, a modder discovered the shoddy xenomorph AI was due to a typo in the game’s code. Woops. But it’s not the first time a game has been humbled by an overlooked keyboard error this year, either! In early 2018, fans discovered why Civilization VI’s AIs were suspiciously into religion: the word “yield” had been spelled “yeild” in the code.
It's not shocking, then, to learn development is full of moments like this.
Leah Miller was a designer at former MMO developer Mythic Entertainment for years, working on writing, content design, and systems design for Dark Age of Camelot and Warhammer Online: Age of Reckoning. At launch, the studio noted a complaint from fans: the game felt slow and unresponsive. Players couldn’t point to anything specific.
It was just...a feeling?
“The team overhauled combat to try to make it feel faster and searched for server inefficiencies that could be eliminated,” said Miller. “Still, the feedback was vague enough that nobody could be certain of the exact cause. Was it some flaw in the game’s holistic design? Many changes were made in an attempt to improve the game’s feel, and while they were often genuine improvements, none of them fixed the core issue.”
The team wouldn’t stumble into a solution, buried in the game’s code, for six months.
A relatively new programmer was poking at Age of Reckoning’s code, and noticed something strange. Age of Reckoning’s code still included a line related to how the previous MMO, Dark Age of Camelot, handled dial-up players. Yes, dial-up. Dark Age of Camelot launched in 2001, an era before high-speed broadband became standard, a time when you had to make sure no one picked up the phone while you were online, lest they break your connection. (The day my family got a 56K modem was magical.)
“He was relatively new to the company and unfamiliar with the code,” said Miller, “which may be why he looked at this section more carefully than anyone had before.”
Once the line was removed, the problem disappeared. The game felt smooth again.
So, what happened? These days, computers are powerful and Internet is fast. That wasn’t the case in the early days of MMOs, which meant games had to compromise.
“In Dark Age of Camelot’s netcode, there was a single line that artificially paced out the rate at which certain types of data were sent by the client and server,” explained Miller. “I believe the main function of this was bandwidth optimization, though it may also have been part of a system designed to make sure dialup players could still be competitive in PvP. This delay was mostly invisible to Dark Age of Camelot players, since that game’s combat was custom-designed for the standard bandwidth and processing power of that era. Very few players had systems that could process data more quickly than it was being sent and received.”
As Dark Age of Camelot evolved, its network code evolved, too, responsible for managing spikes in lag, login servers, and other tasks. After a decade of operating Dark Age of Camelot, Mythic decided to bring the same code to Age of Reckoning.
As it turned out, it broke something they didn’t anticipate.
By the time Mythic had fixed the problem, most of the game’s players had migrated back to World of Warcraft. This bug was not solely responsible for Age of Reckoning’s troubles, but it didn’t help.
I heard from Miller after publicly asking on Twitter for other horror stories, even if the examples went beyond something as immediately humorous as a typo. It synced up with a discussion that’d been happening in other circles, after the Colonial Marines incident, and I wanted to share some of the tales that stood out.
I also want to highlight one typo that blew my mind.
As a kid, I was obsessed with Super Mario 64. It was the first game I went out of my way to play in Japanese, thanks to an import store around the corner from a movie theater we’d frequent. (They would charge people by the hour to play imported copies of Mario’s first 3D platformer! It was totally worth it.) Anyway, when I was finally able to take home my own copy of Super Mario 64, I was one of many people who, along the way to collecting 120 stars and tracking down Yoshi, wondered why the heck it was so hard to swim through the underwater rings in the game. Well, there’s a reason!
Incredible. Video games shouldn't exist.
Follow Patrick on Twitter. If you have a tip or a story idea, drop him an email: firstname.lastname@example.org.