IA Logo

IA Information

Dave Mark's books

Post-Play'em - Observations on Game AI

Posts Tagged ‘FPS’

Splinter Cell: Conviction — Last Known Position

Thursday, July 21st, 2011

I’ve been playing quite a bit more of Splinter Cell: Conviction. Thankfully, I’ve been finding the game a lot easier than I did early on. Part of it has to do with getting used to the flow of things in a more stealth-based game.

I had noted in my “First Look” post that the enemies do a decent job focusing on the “last known position” mechanic that the game is proud of. To explain, when you are sighted and then go back into cover, the game draws a black and white line art model of your position when you were last seen. This is meant to be a marker of where the enemy thinks you are. The encouraged play is to separate yourself from that point so that you can either hide elsewhere or flank and surprise them. This is a helpful reminder and is only made possible by the fact that the AI has a mental model of your position other than the old fashioned “omniscient” method that games used to use.

Again, they do a good job of searching for you—whether at your actual location or your believed one. They will use cover, approach cautiously, and when the discover you aren’t really there, start to search in what looks like a meaningful way. This is where the problem occurs, however.

The enemies will begin to search the entire area even when they should know
that you are not in those places.

In some instances, when they find that you aren’t where they thought you were, there could be many directions or places to which you could have escaped. At other times, however, there are very few options—or even only one option—that you could have selected. However, the enemies will begin to search the entire area even when they should know that you are not in those places.

One example I saw that was glaring was when I had engaged enemies from a door to their room. I never actually entered the room but rather was firing from the doorway. Eventually, outgunned as I was, it was in my best interests to retreat into the room that I had come from. As I moved away, the line art silhouette marked me as still being crouched next to the door. I watched as they came up to the door to attack where they thought I was. Upon discovering me, however, they exclaimed something or other about losing me… and began searching their own room! They had been watching the doorway the whole time and should have known that I had never crossed to their side of it. The logical conclusion should have been that I was on my side of the door. The result was that I was able to slowly re-assault their position because they were no longer wary of me in the direction of my approach. Good for my survival; bad for my suspension of disbelief.

Good for my survival; bad for my
suspension of disbelief.

The solution to this problem is to leverage “occupancy maps”. The concept has been around for a while but very much came to the game dev forefront when Damián Isla gave an excellent presentation on it at the GDC AI Summit in 2009. The idea is based on the concept of influence maps where the grid space values represent the probability that a target is there. Obviously, if you can see a location, then the target is not there. You then simply spread the probabilities over the areas that he could have gone. By selecting the highest grid value at any one time, you are, by definition, going to the most likely place that the target could be. The result is a spectacular mimicry of how people (and animals for that matter) search for something that disappeared from site.

This method is slightly more memory and processor intensive than simply having a shared (x,y,z) location for the last known position of the player. I honestly don’t know if they had resources to burn in the game. However, I have to wonder if they even thought of trying the approach. I wish they had, though. An otherwise well-done stealth based game would have been that much better for their efforts.

Splinter Cell: Conviction – First Look

Sunday, July 10th, 2011

I’ve been putzing with Splinter Cell: Conviction on and off in recent weeks. As a side note, my use of it has mostly been “off” because I find it frustratingly difficult, even on “normal”. That being said, I’m impressed with the game as a whole.

Too often in games can we do stuff that is completely ridiculous and people don’t even react at all.

One of the first thing I noticed was how “by-standers” react to me. If I’m crawling around on a wall or doing something else “odd”, they will turn and look or making some snarky comment. I like that in a game. Too often in games can we do stuff that is completely ridiculous and people don’t even react at all. In order to facilitate this, I assume that actions and/or locations were simply flagged as “odd” and broadcast to the people in the area.

Similarly, I like how the “Thief-like” environmental triggers can arouse the attention of the bad guys. The voice acting in this case really sells it. They are often very subtle about it, saying things like, “hey… I’m gonna go check something over there.” It could have been done in a more heavy-handed way (e.g. “I heard something over there.”)… but the way they did it almost projects how the AI is actually questioning itself. The minor difference makes you (the player) think more “I need to be more careful” rather than jumping straight to “Oh shit!”

Anyway, that isn’t what I was wanting to write about…

No Rails Here!

Obviously, being the type of game that it is, there are more than a few opportunities to lie in way for people. Early on, there was a point where I was waiting for a guy who was unaware of me. I found myself falling back on old gaming mentalities and trying to detect his “patrol route”. That has long been a staple of games—to the point where it has been ingrained in me.

I kinda just had to mentally will him in my direction.

Needless to say, the baddie didn’t follow a pattern of any sort. I kinda just had to wait him out and mentally will him in my direction. Plenty of that time was spent watching him not move at all. I actually wondered for a while if he was a static set-piece rather than an ambulatory one. He wasn’t… he was just alternating between hanging out and wandering around. It really did a good job of selling the idea that this guard was that reasonable mixture of responsible and bored to tears.

On the other side of the coin, when the guys are actually searching for me, the game does a good job of making them look reasonable in their search for me, but also looking somewhat unpredictable. They don’t do the standard “run right toward where the player should be” gameplan once they lose track of me. That would be too silly. Once they have “lost me”, however, they don’t just go searching around willy-nilly; they keep to a reasonable area. That makes for a lot of tense moments when you are holed up in a corner!

While they are searching, the agents do a good job of looking around with that slightly spooked feeling. They will swing their flashlights around—sometimes in radical changes of direction like they are either surprised by a noise or trying to catch me by surprise. I actually found myself just watching the AI search for me to see what all they would do. (Well… the fact that I was on the outside of a windowsill with nowhere to go was a factor.)

Down in Front!

The combat AI seems solid. Their reactions are decent and they usually take cover fairly well. I don’t believe they use cover quite as well as what I have been seeing in Gears of War 2 (which I am finally almost done with). Sometimes they seem to be on the wrong side of it. Not on my side, mind you… just on perhaps the 90° side of things rather than opposite me.

A guy ran up and hid along the side of the couch that I was crouching behind.

On the other hand, the enemies don’t seem to do as much pointless running around as they did in Gears. They might want to consider more than simply cover points, though. There was one moment where a guy ran up and hid along the side of the couch that I was crouching behind. I was so startled that he would run up to get within point-blank range of me that I was very taken aback.

They don’t seem to use their environment in as dynamic a way as F.E.A.R. did, but it doesn’t set them back. In fact, if they were flipping couches and knocking over bookshelves, it might have looked a little contrived.

You Really Get Around!

The pathfinding is decent (thanks, I’m sure, to the excellent navmesh work of my buddy Martin Walsh). I believe there was one moment early on where I saw someone backtrack 2 steps before doing a turn the other way as if he was on a grid. That may have been something else, however. I just remember noticing it because I knew ahead of time that the game was on a navmesh.

While the local avoidance on the streets seems OK, I think that the “personal space” buffer could have been increased slightly. People coming the other direction weren’t steering to go around me soon enough. That’s minor, though.

When I make Sam stop and reverse direction quickly while running, the animation reminds me a lot of the same action in Assassin’s Creed. I have to wonder if that is a coincidence or not.

All in all, I’m quite pleased with it so far. I do need to pay a bit more attention to the AI in the combat situations. On the other hand, the fact that I’m so distracted with staying alive means that the enemies are certainly doing their job well!

Gears of War 2: Cover vs. Running Around

Sunday, June 12th, 2011

The recent E3 (2011) reminded me that it was time to catch up on some games. Additionally, it reminded me that I really want to comment on the AI of the games that I play—something that I haven’t been doing as much of lately… at least not on here. So, after having not touched it for a while (2 years!), I picked up my copy of Gears of War 2. (Thanks again to Epic AI Programmer, Matt Tonks, for providing me with a copy of the game.)


The AI in Gears 2 actually uses cover in a reasonably intelligent manner.

It took me a while to get back into the groove of the control system—most notably how to use their cover system. At the time, using cover that way was a fairly fresh idea. It has been utilized in plenty more games since then, of course. Even better was how the AI was using cover. Unlike other games that I have played where the AI either subscribes to the “Doom model” of standing out in the open, or half-heartedly takes cover, the AI in Gears 2 actually uses cover in a reasonably intelligent manner.

For example, compared to how the AI in Mass Effect used cover as two ends of a shooting gallery path, the AI in Gears 2 seems to take advantageous cover and keep it. This really adds some nice depth to the game in that, often, the only way to solve the resulting stalemate is to flank them so that their cover is less useful.

I can’t just sit there and play Whack-a-Mole.

Their “pop-outs” to fire on me are quite unpredictable which is frustrating (in a good way). Additionally, if the cover is long enough for them to move behind it, they seem to do so. Again, this is one of those things that, unless I was running a video recording, it would really be hard to discern in the middle of a firefight. What I do know, is that I can’t just sit there and play Whack-a-Mole with someone who is in cover—waiting with my cursor positioned perfectly so that when they pop up, I can pop them. That has always been a tactic that my slow thumbs relished because it was so simple to defeat. If the enemies are moving and popping up at unpredictable intervals, I either can’t anticipate where or when they will do so, or I simply get anxious and move on to another target that is visible.

As a follow -up to that last point, it is often hard to switch from one target to another in time because their exposure is so limited. They truly do only “pop out” rather than “come out and remain a really obvious, easy-to-hit target.” All of this serves to create a very frenetic atmosphere in the combat… even the “trench warfare” combat that could normally get pretty stale.


Unfortunately, there is some issue I take with the combat AI. Sometimes, both friend and foe seem to get into a mode where they are dashing around in odd ways. This seems related both to target selection (and by this I mean not just who to shoot but where to go) and idle situation positioning. In the former, it seems similar to what I experienced with Dragon Age: Origins. In that case, enemies would run past one of my allies—completely ignoring them—in order to get to me.

It simply doesn’t make tactical sense to leave your group like that.

In this situation, it isn’t necessarily limited to the enemy having an unhealthy preoccupation with me. My allies will run off into combat way out in front of what are obviously established lines. This surprises me quite a bit when I’m looking way downrange from our location (even sniping) and suddenly see one of my allies appear in my scope. It’s not as if I am holding back—often there is combat happening directly in front of us. In a game where one of the mechanics is being “revived” by one of your allies, it is disturbing when one of them runs so far afield. Even without that mechanic, the nature of this game is one of squad tactics. It simply doesn’t make tactical sense to leave your group and go deep into a group of enemies like that.

The issue (both friend and foe) could be solved by a better decision selection algorithm that takes into account proximity better. This could be achieved in two ways — first by scoring individual choices better (e.g. including better opportunity cost for cover points), or on the whole by including influence mapping. Certainly, influence mapping would then be rolled up into the decision-making process, but you could manage better mathematical decisions without necessarily inducing the overhead of an influence map.

There is no concept of “personal space“.

When the characters are supposed to be idle, I’ve had a number of situations occur where my allies were simply not paying attention to me or other allies. They will bump into me in an open area obviously trying to get to a spot on which I’m already standing or at least near to it. There is no concept of “personal space” even to the point of them trying to be in my space.

Sure, much of this could be solved by the inclusion of simple steering and separation algos, but I have to wonder if the decision of where to go isn’t being locked in too early and therefore overriding the sensibility of how to get there. What was a legitimate destination when you started there became a poor choice by the time you were half-way. Therefore, you either look like an idiot going there in the first place or, once you get there, you immediately change your mind and go someplace else. This would certainly lead to some of the “running around” behaviors as well. The solution partially lays in updating your target location often. Sometimes, changing your target location is really not that big of a deal anyway. If your goal is simply a random walk, your destination is unknown to the observer and largely irrelevant anyway.

Working Together

One thing that I haven’t seen (yet?) is a good sense that the enemy is working together. I mentioned this in my First Look post on the game over 2 years ago. Having played plenty of Halo 3 and Halo: Reach, I’m used to seeing the “organized fall-back” behavior that was built into their enemy AI. Not seeing it here in Gears 2 is actually kind of startling by its absence. Certainly building in that kind of behavior goes beyond the AI brain of the characters. It needs to be designed into the levels, etc. That may not have even been an option for the Gears team much less a consideration. I just wanted to point out that I “felt” it not being there.

Anyway, I’m not done with Gears of War 2, so if anything else comes up, you will see it here.

Left 4 Dead: Companion State Changes

Wednesday, February 25th, 2009

Another interesting observation in Left 4 Dead. In this case, it is about the state machine that the companion AIs are using. First, the observations:

The first clip in the video below shows me getting ready to leave the safe house at the beginning of the level. My companions did their usual “grab some stuff” behaviors and then lapsed into “random wander” idle behaviors (I couldn’t hit the screen shot key fast enough to show Louis standing with his nose in the corner like a punished boy.) When I went down the stairs to the door, I was mildly perturbed that they didn’t follow.

I then opened the door and shot the zombie standing outside. They still had not moved to join me. A zombie rushed me, I leveled him, and my sidekicks were still admiring the walls upstairs. Only when I stepped across the threshold did they move to join me (with Francis doing a completely unnecessary walk on the railing… but that’s a future post). Something not shown in the video is something I have experienced before. Usually, when I step across the threshold of the safe house, they are in quite a hurry to leave the room, even to the point of pushing through me to do so. It probably would have happened in this case if we had not been under attack at the time.

In the second clip, we were at one of the intermediate buildings on a map. We had run inside, closed the door, stocked up on ammo and health packs, healed ourselves and whatnot. I opened the door and left the building. The video picks up as I look back inside realizing that my pals didn’t seem to want to leave. This was different than the first clip in that the trigger was not leaving the room.

There was a Boomer behind the building that I could hear. Even when the Boomer came around the corner and started waddling toward me, they didn’t move. Only when I fired my weapon did they decide it was time to rush out.

Now, for the explanation. It seems that Valve is using an HFSM (Hierarchical Finite State Machine) or another such tiered approach (a behavior tree can cause this as well). There is likely a high-level state that we will call “In Safe House”. When in this state, other lower-level states are things like “random wander,” and “random comment.” The only thing that seems to override it is if they see a zombie outside a door (many of the safe house doors have those barred windows). They will actually engage and kill zombies outside the safe house from inside the door. Therefore, there is an “engage/kill” state that is contained under “In Safe House”.

On the other hand, another high-level state is “In the World”. It is in this state that the AI spends most of its time. Apparently, “stick with the player” and “defend the player” are only included in this high-level state. That is why they would not follow me down the stairs while I was still in the safe house or defend me when I got attacked. However, once I crossed the threshold, a message was sent to them to change states to “In the World” at which point, they were free to analyze their usual parameters such as distance (move to the player) and threats (defend the player).

Note that this would not have been a big deal if I had simply stepped out the door. Alternately, in other safe house situations, the random wander location is in sight of the door. Therefore, when I got attacked, they would have likely seen the zombies and fought back. This particular arrangement did not allow for that.

Now, I don’t know what happened with the building in the second half of the clip. Because that isn’t an “beginning/end of level” safe house, I would suspect the above rules don’t apply. Why did they not leave, then? I have seen other behaviors where they don’t seem to follow me like I would expect, but I have usually found other explanations for that (another post on that later). In this case, it would have made sense for them to leave along with me just like they tend to stick close in other circumstances. That leads me to believe that there was an artificial state in play that led them to believe that they were supposed to be there (or rather had yet to convince them it was time to leave).

Regardless, in this case, the obvious trigger was me firing my weapon. This was not the case in the safe house example.

Neither of these issues is dreadfully wrong in a gameplay sense. They are only noticeable in certain circumstances. And certainly the companion AI in L4D is better than some we have seen. However, when issues like these happen, they do make us pause and ask “what are you guys thinking?” Therefore, while the logistics of the game may not be affected too much, the perception of the game is. It breaks that coveted suspension of disbelief by making us ask (in typical gamer parlance) “WTF?!?”

A simple solution would have been to pay more attention to what was under the HFSM state of “In Safe House”. Alternately, have more than one trigger to transition from “In Safe House” to “In the World” would have been better. For example, opening the door is an obvious trigger that it is time to go. Getting attacked certainly is urgent enough to warrant attention as well.

Additionally, abandoning the rigidity of a HFSM could be the answer as well. Much of the problem would be solved by using a system of free-floating priorities such as what I describe in my book, “Behavioral Mathematics for Game AI”. In that arrangement, you can generally dispense with the state/transition model in favor of one that always has all possible actions in play through a system of calculated utilities and priorities. Something similar to this is probably already in effect in L4D for things like target selection, action selection (fight, heal, reload, etc.) and other actions. Therefore, extending it to cover the situations covered above would not be terribly difficult, in my opinion.

Anyway, all in all the companion AI seems to be fairly decent. As we AI programmers know, companion AI is a beast simply because of how involved the companion is with the player. There’s more scrutiny, more options of what to do, and far more potential for the “WTF?” moment and the ensuing frustration. I think that companion AI is the next frontier of game AI that we are already in the middle of. L4D is in the vanguard of this movement and doing an admirable job of it.

Remember to click the tags below for more Post-Play’em observations on Left 4 Dead and other related subjects!

Also, if you liked this article, please take a moment to submit this link to StumbleUpon, Digg, or Reddit. I would appreciate it, as would many other game AI enthusiasts!

Left 4 Dead: First Look

Monday, February 23rd, 2009

Ok… half price weekend on Steam finally got me into Left 4 Dead. After watching all my TF2 buddies drifting away to this mysterious “other game”, I needed to jump in and take a look. Also, since it will likely be the buzz of GDC next month, I figured I had better have an idea of what it looked and felt like.

I have to admit that it’s a pretty groovy game. All the stuff we have heard so far is correct. The online, team play is cool. It’s so much better than most team shooters in that it seriously punishes you for separating from your cohorts too much. Also, because there is such a significantly different array of tactics that are necessary for playing as the infected, the versus mode is almost like having a second game. But that’s not what this column is about…

The big AI pull on Left 4 Dead is the “AI Director” that we have all heard so much about. In an interview with Edge Online, Valve’s Gabe Newell talks about all the nifty stuff they tried to take into account. Wikipedia even touches on the AI Director. It’s no secret that this tech was a big deal.

When I finally got to playing it, I was very impressed with it myself. Having played through some of the levels more than once (like No Mercy, for example), I am well aware that I don’t know where anything is going to be next. Compare that to other games where the 2nd time through you already have an idea of what is coming next… by the 3rd or 4th time, you have it down. I don’t like that amount of rigid predictability. With L4D, I have been on my toes at all times.

It does seem like it does a decent job of balancing the flow of the game. There are plenty of times when I’m “on the edge”. Compare this to the rubber banding I suspected in Doom 3 where it just seemed that the lower I got on health, the less damage the enemy did. The result was an (almost) asymptotic approach to 0 health. Exciting? A little. Contrived? Definitely. With L4D, this feeling is much more natural in that it is adjust the pacing of the enemies themselves rather than adjusting the abilities they present. Rather than the confusion of “why is this horrible monster doing very little damage now that I’m almost dead?”, I find myself saying “thank God they aren’t coming after us just now… I need to heal up a little!” It’s a subtle difference, but it is there.

That’s not to say that the game is always winnable in true rubber band fashion. On my first day of play, three friends and I played that last rooftop level of “No Mercy” about 8 times and kept coming up just short. It was exciting each time. Finally, we beat it by only barely dragging our asses into the chopper. Quite the getaway! As we reflected on what had happened (the game was new for all of us), I noted that every single time was different. Different monster placement, different ammo placement, different items, different pacing. We couldn’t do the same thing each time because the computer wasn’t doing the same thing. One time we used gas cans to light fires. The next time, I went looking for the gas cans… only to find that they weren’t there (they were across the level). The next time, there were no gas cans at all… just more propane tanks — which meant a different strategy entirely.

What I have found interesting is, while playing the infected (which I am having trouble getting the hang of), I can watch the other zombies “pop in” as they spawn. It’s kinda fun to see them do that. Oddly, I feel a strange kinship with them at that point. (“Yeah, right here buddy! We’re gonna get ‘em from this closet, right dude? Right? Hey… pay attention to me!”)

The other thing I get to watch as the infected is how their “idle” behaviors work. There are quite a few of them and they blend well. Admittedly, by their nature, zombies don’t have to be terribly engaging, but it is a nice touch to come around a corner or see a group far off actually doing stuff. Sometimes, I feel that the sensory distance is a little short, however. If a little beepy grenade gets every zombie in the same zip code to freak out, why doesn’t my shotgun wake them all out of their staggering stupor as well? Again, people just don’t model the sense of hearing well in games. *sigh*

The AI of your counterparts is pretty decent, for the most part. I’ve seen a couple of odd glitches, however. Therefore, I think I’m going to make that a separate post so I can possibly post some FRAPS-caps of them in action.

(As always, make sure you check the tags below to see what else I may have written about this game.)

First Encounter with F.E.A.R.

Monday, January 21st, 2008

Ok… this isn’t necessarily going to be a full-fledged review of all of the AI in F.E.A.R. (Monolith studios) Actually, to do that justice may take an entire book anyway. However, I have been playing it for a few days. One of the reasons that I had to get a hold of it anyway (I bought the Platinum Collection) is that I also got my hands on the SDK. It is a special treat to be able to see the actual code that went into making this ground-breaking gem of a game.

Anyway, I had a taste of the game a few months back via the downloadable demo from Gamespot. Even with that brief glimpse I was impressed. Now that I have been able to “dork with it” (research term) a little more, I have found myself saying a few things that surprise me.

“Son of a bitch! Where did he come from?”

This usually occurs when I fall into old patterns of thinking that the enemies are going to generally either stay put or move toward me in something resembling a direct assault. The first time this happened was on one of the more open, but box-laden arenas (which I am realizing was a design decision to show off this exact effect). I was slowly closing on a group of enemies… or at least where I thought they were (more on that later) when I get pegged from behind.

Now this isn’t a “Doom 3″ sort of assault (see my post on the subject) where the game either spawns a monster directly behind me or pops open a completely illogical hidden panel in order to literally kick me in the ass. As I thought about where this dude could have come from, I realized that it was one of the enemies that had been in front of me… but off to one side. He actually had circled around some obstacles and come up behind me. Sure, he probably didn’t realize that I had moved until he reached the spot where he had last seen me – but then rather than stand there, he continued on the only logical way I could have gone until he did discover me – and proceeded to politely pop a proverbial cap in my ass. Score one for the bad guys.

“Get your butt back here, wimp!”

Again, unlike shooters that I have played in the past, I encountered something that was actually almost frustrating in the novelty of it. It was realistic… which actually took some getting used to. When I would engage an enemy, they were just as likely to fall back as they were to move forward. I may take a shot or two at them only to see them walk, run, or dive around corners. They weren’t just going to cover, they were pulling back. This left me in the uncomfortable spot of having to move into a hostile environment where I knew dudes were camping for me… a position that I have always tried to put the enemy AI in. Now I know how effective it is – since I don’t really relish having to be the one doing the hunting.

“I don’t have all freakin’ day!”

Rather the opposite of above, I have tried to fall back to patterns of “agro-ing” the enemy and then dropping back to wait. As often as not, they don’t fall for it. If they know I’m there, they may very well not come for me – especially if I have nowhere to go. I’m used to being quite comfortable simply waiting around a corner with a shotgun to my shoulder ready to multi-perforate the first moving object that shows itself. I wait… and I wait… until I hear “Flush him out!” followed by that delightful ping of a grenade rattling around at my feet. Crap! But do they just come running dumbly around a corner like my cat hearing the food in his bowl? Nope… I gotta come to them.

“Would you show yourself, damnit?!?”

Somewhat related to the above is their stubborn insistence on using cover. Yeah, using cover is cool. We’ve been talking about it at GDC roundtables and message boards for years. For a while, AI programmers were all happy to use preset “cover points”. In a general sense, they looked good… but they were easy to exploit by just being in a place where that specific cover point was not truly a cover point at all. I get a feeling that these assholes would be perfectly comfortable playing hide-and-seek in a round room with a round pillar in the center of it. They seem to process cover the same way that a human does… “can this specific spot be seen by that dude over there?”

It gets really frustrating when I get into peek-a-boo mode with a guy. The enemy may position itself in the shadow zone of a strip of wall, a column or something to take cover. If I peek around one side, he will move a little to keep the cover between us. If I move to look around the other side, he moves also. There isn’t any invisible pre-defined spot that he’s on, he’s simply trying to not be seen. It pisses me off! Game AI is not supposed to act this way!!

“Quit acting like you guys like each other!”

[Cascade this from above...] If Ol’ Chuck there is running to cover like the little bitch that he is (my language gets salty when I’m pwning), do me a favor and let me gun him down like an arcade ducky. Do NOT annoy me with suppressing fire and all that military squad nonsense. He’s got his back turned and I want to blast him before he gets to that box because, once he gets there, we have already determined that he’s not going to show me anything more than the barrel of his gun for the next 20 minutes. You really are not helping me out by scattering an endless cornucopia of metal alloy in my general direction. It really is distracting and makes it awfully hard for me to jump out here in the middle of the doorway and calmly aim down the sight at his weenie little ass. You act as if he’s on the same team as you or something! What the hell is wrong with you, anyway? Sheesh!

“OK… that’s not funny anymore!”

This was my latest little adventure. I finally met up with one of these “Watchers”. They are like freakin’ Spiderman The Rabbit Puncher. The first time I saw one, I thought it was one of the hallucinations again so I didn’t think to actually shoot it until it walked up and bitch-slapped me… and then disappeared into the damn ceiling before I could blink. Uh. Ok… that was odd. Until, from the corner of my eye, I saw him (or his buddy) swing down from the ceiling, off the wall, over the desk, smack me on the butt again, and then perform a similarly frenetic egress.

For the next 3 minutes, I was twitching around all over the place like I was going through the DTs while on LSD. These two dudes were zipping up, down, “over, under, around and through” so as to keep striking me from behind. If I turned and saw them in time, they were just as likely to about face and retreat and replan. I began to realize that “replan” is exactly what they were doing. This wasn’t a pre-set script to whack me when I got to a certain location… they were making this up on the fly (crawl, leap, cling, whatever)! I finally managed to pop one with the shotgun but the other got me. I was too shaken by the fact that they were actually being clever that I had to quit… and start writing.


I’m not sure what I want to do now. Crawl through a billion lines of AI code spread across 100 different AI classes so I know what they are doing… or keep playing so I can experience it more and maybe put myself into a position where I can actually understand some of the more esoteric stuff that I encounter in the code. Right now I’m too shaken up by my encounter with real AI to do either one.

Maybe I’ll go play Doom 3 instead. For some reason, creepy lighting and environments and stupid enemies is not as daunting as generically bland lighting and environments and monsters that actually act like they have a brain for a change.

Congrats to Jeff Orkin and company – I’ll see you next month at GDC. I will be honored and excited to meet you and “talk shop” such as it is. But don’t be surprised if I have a PTSD reaction and slug you before you get a chance flip over that damn table and hide behind it.

(More on F.E.A.R. AI at AIGameDev.com)

Doom 3: Pretty Wind-up Toys

Thursday, December 13th, 2007

I dug out my Doom 3 again specifically for this blog. I hadn’t ever finished it when I started playing it about a year ago. It sure was purty, but it just didn’t engage me the way that I was hoping it would. Anyway, my thought as I returned to my saved progress was “pay attention to the AI so I can review it.” As I played, I became a little unsettled – not by my progress in the game, but by my progress in my quest for AI. I finally had to come to a realization that seemed almost blasphemous or sacrilegious.

Doom 3 doesn’t have AI.

Sure, I know that statement is a little over the top. Let’s face it, the AI in Doom 3is better than that of games from when I was a kid in the 80s. But that is also the heart of the problem. Doom 3′s AI isn’t much better than what was in the original Doom in 1993. Considering the related concept of Moore’s Law, you would think thought that AI would have increased at a geometric rate similar to that of anything else in computing. It is with that expectation that I claim that Doom 3 “doesn’t have AI”. Looking backwards down a Moore’s Law curve of AI, one could make the statement that the AIs in the past approach zero… therefore being zero for all intents and purposes – especially in a relative sense to where they should be today. So… my statement is now qualified somewhat. (Why do I feel like I’m going to end up having an uncomfortable beer with someone from id at the GDC in February?)

The comparison to the original Doom (or Quake, et al) is not far from the truth, however. The state machine has to look something along the lines of:

  1. Idle
  2. Agro
  3. If has ranged weapons – fire
  4. If no ranged weapons – approach target
  5. If health <= 0 then Die

That’s it. In the newer versions, you can splice in something towards the end that involves a side-step. The bad guy marines especially will do this. To me, the player, this makes me do the very involved strategic and tactical process of… uh… re-aiming my gun. Wow.

Admittedly, there are some baddies that give me more fits than others because they are not approaching me in a straight line. Lost Souls tend to zip around like gnats once they get close to you which is mildly annoying and Cacodemons will float around as you shoot them – but that means they have as much AI as a helium-filled piñata. There are other differences as well. The half baby/half fly Cherubs will back off a bit, prepare a moment, and then leap. Of course, so do the little spider things.

This is a major let down. I remember seeing some in-game clips at a game conference during an interview with Carmack. (I believe it was at E3.) The scene where the pink baddie tried to beat down the door, gave up and instead came crashing through the window was terrifying! “Wow! It gave up and used the window!” I told myself. When I played the game… and that exact same sequence happened, I realized that the whole thing was scripted. Oh. Bummer. Again, it wasn’t AI. Unlike F.E.A.R, where the agents do use their environment and re-plan when faced with obstacles such as the door above, the Doom solution was a movie crafted for my benefit.

I have been flanked by the computer, however. This I admit. Often! But that has nothing to do with the AI and everything to do with the placement of triggers. In true horror movie fashion, the computer does come up behind you often – but only because a designer placed a trigger on the floor that says “when player gets here, spawn evil dude behind him”. That isn’t AI. It isn’t even fake AI. In fact, it’s getting rather tiresome to know that every step I take is likely to generate an attack from a place I just cleared out. It makes for great anxiety and twitchyness – which is great for a horror movie. I admit that I have a good deal of tension when I play the game not knowing from which dark corner or what hidden panel I will receive the next assault. But it’s not AI. Really, it’s like making the “only average” fastball pitcher harder to hit by moving him closer to the plate or allowing him to throw from foul territory. He isn’t a better pitcher now… he just has been given an artificial advantage.

Really, the movement and attack logic for the enemy is not much more advanced than the little table-top wind-up toys. They will chatter away in my general direction, but there isn’t much purpose or reactivity to their actions. That gets very disappointing.

There does seem to be a bit of cheating going on, as well. The first time I fought a Mancubus, I was doing my best to flank him around walls. Despite having moved at least 60° off to the side, when I poked my nose around the wall to take another shot, I found he was already facing right at me. The explanation would have to be that there is omniscience on my position. Here, I had decided to do a simple, but what I thought clever, evasive maneuver only to have the game say “tough… I don’t care”. It’s deflating to have that happen.

There also may be some cheating going on for my benefit as well. This doesn’t fall exactly into AI, but it is relevant nonetheless. I swear that I take less damage per attack when I am almost dead. I haven’t done the math yet, but it seems that I can be in the teens or single-digit health for far longer than I spend in any other 10-number range. A hit that takes me down from 100 to 80, for example, may also only take me from 15 to 10. The result is that I spend a ton of time between 0 and about 40. Of course, if this is happening, this is a great device for creating tension in the game – and is a variant on rubber-banding. It’s rather artificial and arbitrary, though.

All in all, Doom 3 is a pretty game. It’s gorgeous. It’s use of dynamic lighting was cutting edge and very impressive for 2004. However, when you put it up against the interactivity and dynamicity of other shooters, there is something empty about it. It’s like trying to engage in a dialog with paintings in the Louvre. They are very impressive as art – but they aren’t going to talk back.

I have always respected John Carmack for his vision – and the “visions” that his games have provided for me. I just can’t help wondering what would happen if they were to put a bit more focus on AI. I notice on the Wikipedia site that Johnathan Wright is listed as an “AI Programmer” but that in the credits for Doom 3, he is just listed as a “programmer”. I would love to sit with him at a GDC party and have that beer. “What’s up, dog? Is there anything else you could be doing?” I can’t blame him, necessarily. Given the depth of the graphics rendering, he may very well just be out of clock cycles. *shrug* I hope that he has a better chance in the future. I would very much like to see an id offering that does “speak to me.”
Until then… there’s more than enough to challenge me elsewhere.

Call of Duty 2: "I saw that grenade coming!"

Friday, November 30th, 2007

Another quick observation from my perusal of “Call of Duty 2″ for Xbox 360. I’ve noticed rather often that, when I toss a grenade, the intended “recipients” seem to know that it is coming as soon as it leaves my hand. They will yell some German variant of “Holy crap, there is a grenade! Run, dude!” This is all well and good if they are seeing the grenade coming and would like to amscray prior to its arrival. However, the quirky behavior comes in when I am doing something like bouncing it off a wall around a corner. I will hear them shout prior to the offending pineapple even making it to the corner that I am trying to circumnavigate. In other words, they either saw the grenade through the wall, or there is some sort of mirror-like sheen on the wall that allows them to see it coming.

The programmers seem to be using a messaging architecture wherein an event triggers a reaction in all agents within range. For example, if the grenade were to explode in front of them, the explosion itself would send a message out to nearby agents saying “if in range, die violently”. Messaging architectures are opposed to polling architectures where the agent is constantly looking through his environment to see if there is something to react to. Using the same example, the agent would constantly have to scan the nearby world to see if a grenade has exploded. Since grenade explosions are relatively rare, this would be a lot of wasted CPU. As you can see, messaging architectures are more efficient for event based situations.

In this case, the landing area of the grenade is roughly calculated. If the projected landing spot is near an agent, they react appropriately. They yell out warnings and move away if necessary.

However, the programmers would have faced a quandary as to when the grenade itself should send a message. If you wait until it lands or bounces off of something, it would discount the very possible scenario that they could have seen it flying through the air in the first place. If it is triggered when it is thrown, it creates the issue I observed above – that they know it is there despite the fact that they should not be able to see it at all.

One solution (albeit not a very efficient one) would be line of sight checks along the path of the grenade. Once a grenade launch has happened – and a message dispatched to the agent – the agent would now have to do a line of sight (LOS) check. If the grenade is visible, all is well – react appropriately. If it is not visible, the agent would have to change techniques to a polling architecture specifically doing periodic LOS checks at the grenade. When you consider that there may be 10 or 20 agents (friend and foe) in the area, multiple grenades in the area, and each agent-grenade LOS check would need to be done many times per second, the computational overhead adds up very quickly.

Another potential workaround would be to create a plane that is the intersection point of the path of the grenade and the visible area of the agent. At that point, the “see grenade” event can be triggered once the grenade passes that point. That seems to be not much of an improvement since there would be as many planes as there are agents and the grenade would now have to poll the planes many times per second. The overhead would be similar to the first example.

There is another way of handling that, however. Once the plane is generated, the distance from the source to the intersection of the plane can be calculated. Since the velocity of the grenade is near constant, the time delay until the projectile reaches the plane can be established. The message can be sent with a delay of x number of frames (or any other way that the game loop timer is built) so that the message is delivered and activated at the point that the grenade would have come into sight. No polling is necessary at this point. Just like before, a single event message is sent. The only additional overhead would be creating the planes representing the lines of sight for the agents in the area. In fact, if the LOS check is successful at the beginning of the throw, the plane creation is not even necessary. If an agent is going to yell to his squadmates about the throw, you don’t need to do LOS checks for them at all – they already know. It would take some testing to find out how much overhead this eats up, but the result would be that you could do stealth/surprise grenades in a much more realistic fashion rather than the current “I saw it coming around the corner” effect.

(More observations of “Call of Duty 2″ for Xbox 360)

Call of Duty 2: Omniscience and Invulnerability

Wednesday, November 28th, 2007

I was playing Call of Duty 2 on XBox 360 last night. I’ve already gone through the full campaign on the normal difficulty and am now about halfway on “Veteran” (“…you will not survive”).

Enemy Invulnerability:
“I can’t be bothered right now.”

I was on the level where your convoy through the town in Tunisia gets ambushed. I had to start it over and over because I was getting pelted by the guys on top of the walls. I finally got down where I should hide and who I should tag first. One thing kept bugging me. There was a string of guys appearing on one wall in order. Most of them were gunners shooting at me – but there was one that had a Panzerschreck. I would try to shoot him over and over and he wouldn’t die until he had fired at one of the tanks/trucks in my convoy and blown it up. In the mean time, another one would pop up and take me down. I realized that it was pointless to try to kill this dude at all. He wasn’t going to kill me and that tank/truck was going to blow up anyway. It was part of the level.

What annoys me, however, is that it took me a number of times before I realized that, until that guy got his shot off, he was invulnerable. As soon as he was done with his job (that I was trying to prevent) he was now able to be shot. I wasted valuable time and got very frustrated by the fact that the level designers had decided that this was such a required series of actions on the level that they would break the rules.

I have found other instances where I have tried to peg some dude that was threatening me and was dead in my sights only to find that he had some sort of mission that couldn’t be stopped. That really exposes the scripting in the game. I understand why the scripting is there – and, in general, it is very well done in the game. I very much love some of the actions that happen despite the fact that I can tell you the exact line I crossed in order to trigger the action – but usually they are too fast for me to react to or something that is meant to be just watched anyway. Don’t let me point at a guy and unload an entire clip into his spleen while he doesn’t seem to care that I am there.

To me, it seems like this is a case of the “anti-sandbox” concept. Sure, a somewhat linear game like CoD isn’t meant to be a sandbox – and there are certain things that have to happen to advance the plot. That’s fine, but I always feel cheated when I can’t change the series of events even if I do the right thing to disrupt it.

Enemy Omniscience:
“Am I wearing an orange hunting vest?”

The other thing that I noticed the other day is that, while I can sneak up on some people from the side once in a while – which is very satisfying – there are other times when someone will spin 45 degrees to point right at me despite the fact that I am peeking between some crates or barrels or sandbags. It is almost as if I just barely moved into a place where the ray-trace succeeded and told the enemy that I was now visible. This is fine if I walk around the corner or pop up behind something, but it seems odd when only 2% of my body is visible. The result is that it seems like the AI is omniscient (i.e. cheating) as to my whereabouts. In some respects it creates excitement, in others frustration. I know that they aren’t trying to create Thief or Splinter Cell. Stealth isn’t the focus of CoD. Still, there are times when hiding is a requirement of the game. Don’t cheat me out of those brief moments of respite.

A possible solution to this is to measure how much of me is visible and then combine that with a coefficient based on how far “off center” I am from their current direction of vision. Perhaps another factor based on movement. I know that is a bit expensive to calculate for each AI (since their fields of view would all be different).

Another potential solution is to cast multiple rays to different parts of my body – perhaps shoulders, head and a couple of lower torso spots. If more than ONE is visible, then I can be seen. Not knowing exactly what mechanism they are using, it’s difficult to know how to improve upon it.

This is even more alarming when it is obvious that there is a bias towards firing at ME. I may have 10 squad members all hidden behind objects and taking pot shots at the enemy – or even running around in the open, but damn it if the AI doesn’t want to fire at me instead. It doesn’t happen all the time, but it is obviously biased more towards me than my squad-mates. This is an obvious design decision to make it more exciting. I understand that. However, when combined with the omniscience above, it’s kinda creepy.

[note: I've realized that waiting to write one complete writeup on a game is sometimes prohibitive - so I will write as I think of things... and tag them by game so you can find all relevant stuff]

Add to Google Reader or Homepage

Content 2002-2015 by Intrinsic Algorithm L.L.C.