This is a story about how I knew within a window of 48 hours when the invasion of Iraq (2003) was going to happen.

It was early March, 2003.  I didn’t know exactly who the guys in suits were, but I knew they weren’t Microsoft.  Only one person I knew wore a suit daily to work at Microsoft, that was Raymond Chen. And he wore a much better class of suit than the guys who suddenly appeared late one evening on floor 6 of Building 40 on the Microsoft campus.

I had joined the Microsoft Security Response Center in November of 2002.  The Slammer attack was my first introduction to *the entire Internet* going offline as a result of a Microsoft security issue.

We were only just recovering from that event.  While all the appropriate and smart people had been mobilized to deal with Slammer, we were not happy with how ad hoc the response was.  So during the month of February and March we developed the Microsoft Internet Security Emergency Response process, MISER. Bill Gates hated the name. It was soon changed to Software Security Incident Response Process, SSIRP.

All I knew was that I had just been given one of the largest offices in the building, where I had installed a bar and held press calls on the security updates for all of MSRC and the ones I had program managed through the Windows team. Back then security updates happened every Wed. morning at 10am Pacific time, instead of every second Tuesday of the month like today. 

As release manager at the time, I would fire up “Yo, Pumpkin Head” on my computer and crank the speakers up as the updates propagated across the cluster of Microsoft.com and Windows Update.  We’d gather in the hallway and chatter as we made sure the updates and security bulletins reached their checkpoints while listening to the music. The entire process took almost exactly long as the song, around four minutes. When that music flooded the hallway, you knew updates were being launched. After that four minutes, I took press calls from CNN, MSNBC, ZDNET, NYT, etc for the rest of the day.

Point being, I was finally settling into the role vs. being in emergency mode for weeks over Slammer.

Then the guys in the suits showed up.

Our process was pretty established.  Microsoft issued security bulletins with updates to fix the problem. We didn’t issue warnings or advisories, we were dead set on issuing the transparent communication of the issue only when there was an update to correct it. At the time we viewed warnings or advisories as the equivalent of leaving a box of guns on the street corner and issuing a notice to citizens that there was a murderer in the area, go get your guns.  As many bad guys would get them, if not more, than attentive good guys. We learned better later, but this was the state in 2003.

I had just settled into the job as I mentioned.  I even had theme music. Then the guys in the suits showed up.

I wasn’t even involved at first.  I walked past our reserved emergency conf. room and in it were George, Ian my boss, Dr. Lipner, and the dudes in suits. I just walked on.  The most prized skill in information security is knowing when you do not want to be burdened with knowing what you do not already know.

It wasn’t until later that Ian showed up in my office to talk about it.

“You know what’s going on?” Ian knew I usually had my ear to the ground.  On this I didn’t.

“Dudes in suits. Usually US government.” I replied.  Ian had served in foreign military, specifically artillery. If it was US gov. in the room I’m sure they were roiling over what they would have to make him sign.

“Yea but do you know what’s going on?” Ian said.

“Nope!” I said.  I’d been knee deep in the regular reported vulnerabilities and MSRC work.

“How much do you know about WebDAV?” he asked.

Turns out I knew a lot.  Back then, WebDAV was a godsend to moving files around over the Internet vs. FTP or trying to use straight up HTTP.  WebDAV essentially treated certain web stores like a mapped network drive.

And in Windows 2000 it had a huge gaping hole.  It was enabled by default.  On all versions.

Ian explained carefully the issue to me, and that the guys in suits, from a section of the US government I’m not going to specify, had discovered it because they were attacked.  And that section of the government had a very important operation about to begin within 14 days.

“How soon do you think we could do a patch?” Ian asked.

I knew the Windows Sustained Engineering team’s schedule and backlog and made a scratch guess.

“No test, smoke test, full test, 14, 21 and 30 days.”  No test meant make the update, someone next to you tests that it fixes it, and you just ship it. Never mind the hundreds of millions of configurations in the world. It was the worst kind of update to ever release.  One we had never done before. 

Smoke test meant some more testing meaning seven days of in house testing.  Full test meant we would release the update to a number of high profile volunteer customers without letting them know specifically what it was for, so that we could understand the full impact.

“No good,” Ian said.  “We need to have it before mid March.”

“Ok, But that’s going to be a realignment of just about everything in the pipe.”

“This issue is worth it.”

That was no easy thing, and Ian knew it.  Before long I found myself in the room with George and Dr. Lipner and Ian and Mike Nash our VP.  Oh and the guys in suits, who I was never introduced to.

Here was the crux of the problem.  All Windows 2000 machines were essentially open to a trivial wormable attack like Slammer through this WebDAV vector.  It had been discovered by a government agency who had been attacked. Suddenly we had to re-evaluate how we communicated about updates.  This was bad enough we would have to consider going with how to block the attack before we actually had an update.  At the time that was anathema to the MSRC.  But this situation caused us to rethink everything.  We drew a line a long time ago before I joined, that no government got preference over users. But this wasn’t about an update per se it was about the existence of the hole. We had to figure out what to do if it became known, not for the agency involved but for everyone.

We handled it like we did any other update.  The reporter in this case we decided didn’t matter.  The severity drove the update, not who reported it.

The Windows team worked night and day to produce a fully tested update within 10 days.

On March 15th I wrote the very first Microsoft “Security Advisory” without a patch which contained information describing the issue and how to manually disable the functionality.  It was never released. We sweated the next two days until Wed, March 17th 2003 and released the update.  The security bulletin for the update contained much of the content I wrote for the advisory.

That particular event ended up forming the nascent idea that we should consider advisories when issues might take time to fix.

As I played the music down the MSRC hallway in building 40 that day, I was approached by a member of the senior staff. (Nope, not saying who)

“You know who got hit right?”

I had a good idea.  But just nodded. “Kinda ironic the patch is 007.”

“Watch the news in the next 48 hours.”

War fever has been gripping the US for the past 2 months, it wasn’t difficult to figure out what was about to happen.

On March 19th, the United States of America invaded Iraq.