IA Logo

IA Information

Dave Mark's books

Post-Play'em - Observations on Game AI

Posts Tagged ‘Companion AI’

Gears of War 2: Cover vs. Running Around

Sunday, June 12th, 2011

The recent E3 (2011) reminded me that it was time to catch up on some games. Additionally, it reminded me that I really want to comment on the AI of the games that I play—something that I haven’t been doing as much of lately… at least not on here. So, after having not touched it for a while (2 years!), I picked up my copy of Gears of War 2. (Thanks again to Epic AI Programmer, Matt Tonks, for providing me with a copy of the game.)


The AI in Gears 2 actually uses cover in a reasonably intelligent manner.

It took me a while to get back into the groove of the control system—most notably how to use their cover system. At the time, using cover that way was a fairly fresh idea. It has been utilized in plenty more games since then, of course. Even better was how the AI was using cover. Unlike other games that I have played where the AI either subscribes to the “Doom model” of standing out in the open, or half-heartedly takes cover, the AI in Gears 2 actually uses cover in a reasonably intelligent manner.

For example, compared to how the AI in Mass Effect used cover as two ends of a shooting gallery path, the AI in Gears 2 seems to take advantageous cover and keep it. This really adds some nice depth to the game in that, often, the only way to solve the resulting stalemate is to flank them so that their cover is less useful.

I can’t just sit there and play Whack-a-Mole.

Their “pop-outs” to fire on me are quite unpredictable which is frustrating (in a good way). Additionally, if the cover is long enough for them to move behind it, they seem to do so. Again, this is one of those things that, unless I was running a video recording, it would really be hard to discern in the middle of a firefight. What I do know, is that I can’t just sit there and play Whack-a-Mole with someone who is in cover—waiting with my cursor positioned perfectly so that when they pop up, I can pop them. That has always been a tactic that my slow thumbs relished because it was so simple to defeat. If the enemies are moving and popping up at unpredictable intervals, I either can’t anticipate where or when they will do so, or I simply get anxious and move on to another target that is visible.

As a follow -up to that last point, it is often hard to switch from one target to another in time because their exposure is so limited. They truly do only “pop out” rather than “come out and remain a really obvious, easy-to-hit target.” All of this serves to create a very frenetic atmosphere in the combat… even the “trench warfare” combat that could normally get pretty stale.


Unfortunately, there is some issue I take with the combat AI. Sometimes, both friend and foe seem to get into a mode where they are dashing around in odd ways. This seems related both to target selection (and by this I mean not just who to shoot but where to go) and idle situation positioning. In the former, it seems similar to what I experienced with Dragon Age: Origins. In that case, enemies would run past one of my allies—completely ignoring them—in order to get to me.

It simply doesn’t make tactical sense to leave your group like that.

In this situation, it isn’t necessarily limited to the enemy having an unhealthy preoccupation with me. My allies will run off into combat way out in front of what are obviously established lines. This surprises me quite a bit when I’m looking way downrange from our location (even sniping) and suddenly see one of my allies appear in my scope. It’s not as if I am holding back—often there is combat happening directly in front of us. In a game where one of the mechanics is being “revived” by one of your allies, it is disturbing when one of them runs so far afield. Even without that mechanic, the nature of this game is one of squad tactics. It simply doesn’t make tactical sense to leave your group and go deep into a group of enemies like that.

The issue (both friend and foe) could be solved by a better decision selection algorithm that takes into account proximity better. This could be achieved in two ways — first by scoring individual choices better (e.g. including better opportunity cost for cover points), or on the whole by including influence mapping. Certainly, influence mapping would then be rolled up into the decision-making process, but you could manage better mathematical decisions without necessarily inducing the overhead of an influence map.

There is no concept of “personal space“.

When the characters are supposed to be idle, I’ve had a number of situations occur where my allies were simply not paying attention to me or other allies. They will bump into me in an open area obviously trying to get to a spot on which I’m already standing or at least near to it. There is no concept of “personal space” even to the point of them trying to be in my space.

Sure, much of this could be solved by the inclusion of simple steering and separation algos, but I have to wonder if the decision of where to go isn’t being locked in too early and therefore overriding the sensibility of how to get there. What was a legitimate destination when you started there became a poor choice by the time you were half-way. Therefore, you either look like an idiot going there in the first place or, once you get there, you immediately change your mind and go someplace else. This would certainly lead to some of the “running around” behaviors as well. The solution partially lays in updating your target location often. Sometimes, changing your target location is really not that big of a deal anyway. If your goal is simply a random walk, your destination is unknown to the observer and largely irrelevant anyway.

Working Together

One thing that I haven’t seen (yet?) is a good sense that the enemy is working together. I mentioned this in my First Look post on the game over 2 years ago. Having played plenty of Halo 3 and Halo: Reach, I’m used to seeing the “organized fall-back” behavior that was built into their enemy AI. Not seeing it here in Gears 2 is actually kind of startling by its absence. Certainly building in that kind of behavior goes beyond the AI brain of the characters. It needs to be designed into the levels, etc. That may not have even been an option for the Gears team much less a consideration. I just wanted to point out that I “felt” it not being there.

Anyway, I’m not done with Gears of War 2, so if anything else comes up, you will see it here.

Dragon Age: Origins – First Look

Monday, February 22nd, 2010

Just sat down with Dragon Age: Origin for a while. I know I won’t get too far into it anytime soon because, for a Bioware game, that usually means a commitment of a few months. I plan on blasting through more soon, but the 360 is on the same 43″ HDTV that the wife wants to watch Olympic stuff on. (I can’t begrudge her that since she usually doesn’t watch enough TV to get in the way of my gaming.)

To give a frame of reference of how far I am, I started as a City Elf and am only just about ready to roll out of town with Duncan. That has, at least, given me plenty of opportunity to go through both peaceful and combative encounters. Also, I haven’t yet dealt with the NPC behavior customization. I imagine that will be more important as I go foward, but for now I just wanted to live by the defaults.
Well Lookie Here!
One of the first things I noticed was when I was in a conversation. While I like fact that people’s eyes move and blink, I would like to see a little more movement. There is so much that can be said by breaking of eye contact, turning the head while still speaking, etc. I have written about this here and there before (and really should write a formal article about it). The point being, what they have done is a start, but it isn’t enough to really sell the character. As an example of dialog scenes that are well done, I point you to Assassin’s Creed (the original at least… haven’t seen AC2 yet). Previously, I wrote about how I liked that, in Assassin’s Creed, not only could you walk around during conversations, but that the NPCs did as well. They looked down, away, etc. I would like to see more of this. The walking around isn’t even quite as important as the head and eye control. Some of those can even be randomized — especially with parameters — so it’s not like it would take a lot of work or resources.

On the subject of head movement, however, one thing I noticed the first time that I was in a conversation with two people was that their head turns are NOT animated by hand but are automated. The reason I know this is that there is a very distinct start and end to the turn and that the turn rate is entirely constant during the whole turn. Put another way, if someone turned from looking at me to the person standing beside them, that 90° turn would go from a rotation rate of 0 straight to n, rotate that 90°, then change rate from n to 0 again. The rate, n, was always the same throughout one turn and on every turn. The overall effect made it look like one of those audio-animatronic characters in a Disneyworld exhibit. It was very mechanical-looking.
While I like the idea behind the technique, I believe a better way would have been to randomize the rate, a rate of change into the turn, and a rate of change out of the turn. This not only changes things up between one turn and the next, but also smooths out the starts and stops so it doesn’t look like little servo motors kicking in.
… one of those audio-animatronic characters in a Disneyworld exhibit.
Also, you don’t always have to look directly at someone. If someone standing next to you is speaking, you can turn your head part of the way and do the rest with your eyes. In fact, it is very common for humans to “split the difference” and look somewhere between the two speakers in that arrangement. Toss in the occasional look at other objects of interest in the environment (especially if you’re nervous, wary, etc.) and you can really sell a conversation without a lot of extra work.
Don’t Mind Me
She never acknowledged that I was even there.
While I appreciate that not every denizen of a place should attack you, I would at least like them to acknowledge my existence. For example, I was a little startled when I was storming the castle. In a couple of the rooms, I came upon a woman scrubbing the floor. She didn’t look up when I entered (if she was a servant, shouldn’t she care to see if it is a boss?), she didn’t look up at I moved around, she didn’t look up as I stood with my boots under her nose. She never acknowledged that I was even there. She didn’t do anything other than scrub the floor. It was a little startling, especially when everyone else in the place (friend or foe) recognized that, as an armed elf, I was seriously out of place. (Incidentally, I got a kick out of the fact that this matched up nicely with the theme of AIGameDev‘s AI Marmelade – their twist on the Global Game Jam. Their coverage of it was sub-titled “on little old cleaning ladies in stealth games.”)
I admit that this is likely a content issue. Developers would love to add more background characters to an area, but then you run into the issue of having them react in meaningful ways. Also, if those reactions are going to involve custom animation or sound, the expense goes up significantly. It would be nice to have a general list of generalized actions and interactions, however. Put people in there moving from place to place, carrying things, tending things, etc. When I come in, at least have them look at me. If I try to engage them, make them politely dismiss me. If I do something flashy like attack people, make them run. That much, at least, puts a little life into the place. Oh well… no one can ever say Bioware skimps on content. They just put it into length of the experience rather than the depth of it, I suppose.
Combat AI – So Far…
I was a little wary going into this first play session because someone I trust on AI commented that he was frustrated with his party members. As I said at the top, I haven’t gotten too deeply into the game and haven’t messed with the customization yet. However, there was nothing that I saw yet that was really “pants-on-head retarded.” I was a fighter as was my buddy. He attacked when I did or when we were attacked. I was too busy trying to do my own gig to see if he used a healing option when he should have. I don’t think he did because he “died” a couple of times during our great castle adventure. I didn’t think to give him a ranged weapon to see if he would switch at appropriate times, though. And now, of course, he is no longer in my party (taking some cool gear into the abyss of retirement with him). My next play session will have me leaving town with Duncan and likely teaming up with some dudes. We’ll see how things go then.
… it looked a lot like a tool that an AI programmer would make for a designer.
I only briefly looked at the customization system. From what I could tell, it looked interesting if you are an AI programmer or a designer. In fact, it looked a lot like a tool that an AI programmer would make for a designer. I would lay money that it’s origin is not too far from that. I can see how it would tend to quell the inevitable gamer uprising that would like more control of the autonomy of their party-members (is there a contradiction in there somewhere?). However, it was said by a few people that you actually have to tweak this system by hand to get people to work together properly. I really don’t like this idea. In my opinion, if it doesn’t work intelligently with the default settings, there is something wrong.
From a technical standpoint, the system looks like it is a parameterized rule-based system of some sort. I’ll have to play with it… for all I know, it is tweaking a behavior tree behind the scenes. Much of my conclusion on what is going on would be based on whether you can sort your rules in a particular order of consideration. Like I said, I only just glanced at it. More on this after I play. In the mean time, I refuse to look up any technical documentation on it just to see if I can figure it out on my own.
The enemies were pretty much standard “fight to the death” fare so far. They were all castle guards and punks, though. That would be right up their alley. I’ll have to see what other types of enemies offer later on.
Anyway, despite their similarities in production, the game is much more enjoyable to me than Mass Effect was — at least for now. I’d be pleased if that stayed the case.
Definitely keep an eye on the rss feed here. I’ll be writing more about this game later in the week.

Brütal Legend’s RTS elements

Tuesday, February 16th, 2010

Ok… now I get what Tara Teich is going to be talking about in her AI Summit lecture on Brütal Legend. She is going to be focusing on the challenges that they faced when they introduced RTS elements into a brawler-style game. When I did my first look, I hadn’t really seen evidence of how the RTS/brawler cross-over was happening. I just now played the first major battle (after you set up the stage for the first time), and I really see the mechanism at work.

Trying to manage all the different types of “units” you can summon makes for an interesting gameplay element. What’s more, as a player, you have to switch between solo brawling and teaming up with the other units. For example, you can get together with a group of headbangers and form a moving moshpit. At times, however, you have to make the decision to disband the group and go back to mashing solo, or team up with another group. Oh yeah, and you can still just issue general orders to units you aren’t teamed up with — orders like “come to me”, “stay and defend” or “attack that”. This can be done in addition to the local control as a team.
If it is difficult for me to manage what is supposed to be happening in the grand melee of units, I can imagine that trying to organize it properly from an AI standpoint must have been challenging as well. Because the units need to be nearby in order to be ordered or paired up with, I wonder if the units try to stay close to the player?

But what happens if there are legitimate people to kill that I’m not near? I can see frustration developing if the units were to just go off and attack on their own. Sure, it’s the right thing to do from a tactical standpoint, but the whole point is to give the player the sense of command. In a way it combines the autonomy of RTS units with a little bit of companion AI from the likes of Left 4 Dead or Mass Effect.
The other question is if they are doing combined tactics when they are on their own. I don’t mean advanced stuff like flanking, etc. Rather, are they just picking the nearest enemy or are they spreading out for maximum coverage?
On both of these points, I will have to do some more observation of what’s going on — and that usually means watching one of my kids play instead of doing it myself. They usually don’t argue much when I tell them it’s their turn.

Fable 2: First Look

Sunday, March 8th, 2009

Continuing with my pre-GDC blitz of “must at least look at” games, I have just putzed around with Fable 2 for about 90 minutes. I had a look at some of this material back at the 2008 GDC when Peter Molyneux was going through his pre-release hype. (Isn’t he always?) Therefore, I knew some of the things to look for.

Most importantly, on the AI front, is the inclusion of the almost legendary “dog” AI. To hear Peter talk, you would have thought this game was a bigger and better “Nintendogs” game… that the entire purpose of buying it was to play with your doggie and bond with him. While that isn’t exactly the case, I have to admit that the dog is a good point of interest. The dog certainly is convincing in many ways.

My Dog Runs Rings around Me!

I think one of the things that was startling to me was how it moves in a general area around you. Most companion AI tends to lag behind you. For example, my recent mucking about with the companion AI of Left 4 Dead has me convinced that I am the only person in the entire world with any initiative. This is obviously a logistic issue because very rarely can the AI know where you want to go or when. This is sometimes mitigated in linear games where there is only one place to go. In my short experience with Gears of War 2 today, I saw that exact phenomenon. My companions were able to move ahead of me and take up positions specifically because there was no other way to go.

In Fable 2 this is done via the “breadcrumbs” trail. Despite being far more “open world” than other games, if you have selected (or the game has selected) a destination, by extension, the dog knows where you are headed. Of course, if you turn around and backtrack or go off to one side for some reason, the dog has to react and come back to you… but this is what happens in real life anyway.

The result of all this is that the dog will playfully romp or explore ahead, behind, of off to one side or the other in a general “halo” around you. If you stop traveling for a period of time, he will come back and see what’s up. If you stand still for very long, he may even lie down for a bit… all the while keeping an expectant eye on you as if to say “now what?” It’s all very convincing.

I really noticed one particular aspect of the dog’s animation after I put the controller down and switched off the 360. I went to go feed our own dogs and, as I was walking from the food bin to the bowl, one of our dogs was walking a few feet ahead of me. However, he had his head turned slightly back toward me, keeping an excited eye on me and the cup of food that I was carrying. He did that very dog-like routine of glancing forward and then back at me over and over all the way to his bowl. It struck me that this is exactly what the dog in Fable 2 was doing as I walked. As he would criss-cross in front of me, he would keep glancing back as if to make sure that I was coming along with him. It was this sort of attention to detail both on the behavior and the animation AI that really sold me on this dog. I’m looking forward to experimenting more with it soon.

Look at Me When I’m Talking to Ya!

Interestingly, there were times with the villager AI that I felt that the eye-contact issue was the problem rather than a feature. I’m not talking about how, while in conversation, people will glance away or past you. This was a problem with where they were facing in general. If they started their spiel while I was in one place, and then I moved to one side or the other, they would continue to face the spot where I was. Only after a moment (usually after finishing a line of dialog), would they turn to face me again. It was all rather disconcerting. Compare this to how well games like Half Life 2 continued to address you throughout a conversation… even to the point of turning and walking backwards if necessary.

Dialog Pauses

Another problem I had with the villager AI was how many of the scripted dialogs had very unnatural pauses between lines. There was an awkward silence between many of their sentences that left me wondering for a moment if they were finished or not. It was one thing when it was in response to something I had selected. However, when they paused in conversation with each other, it was odd. When they paused between their own lines, it was horrific and very distracting. I have to imagine that this is because each line of dialog needed to be loaded and accessed separately, but I can’t see why this would be a problem.

Both the facing issue and the dialog pause issue may be related to the same root cause, however. It may be a result of waiting for an associated animation to finish and report back that it is done. If a line of dialog (and the code to “face the player”) are associated with a particular animation, they would each have to wait for the other to finish before they both advanced to the next segment. If the animation finishes first, it isn’t as tragic because the animation can sit in idle for that extra second. However, if the voice finishes first, the next line must wait until the animation is done — causing the unnatural pause. This is just speculation on my part for the moment. I figure that if I analyze it a little further I can make the determination down the road.

Individual Individuals

One thing that jumped out at me during the initial city quests was how the group of kids all reacted slightly different to my various gestures, poses, and flatulation. I would get more “love points” (or whatever) from some than I would from others in the group. Upon a little examination, I found that each person had their own likes and dislikes — that is, things that they were more impressed by or didn’t like. It’s one thing to have different “respect ratings” from different people. It’s quite another to have different models of what causes respect on an individual basis.

I found this rather amusing since I am doing a co-lecture on this very subject at GDC in a few weeks. (“Breaking the Cookie Cutter: Modeling Individual Personality, Mood, and Emotion in Characters“) It’s something I would like to see more of in games in the future — although maybe not in quite as transparent a fashion as what appeared on-screen in Fable 2.

Will You Treat Me Different When I’m Famous?

I think it will remain to be seen what happens when I’m more renowned than I am so early in the game. In the original Fable, it got rather annoying that everyone, everywhere wouldn’t shut up about you. The comments that they heaped upon you were not as many in number as there needed to be. (You can only hear “chicken chaser” so many times before you want to crack someone’s head.) With a nod to my friend and colleague, Adam Russell, who did the villager AI for Fable, that isn’t an AI problem as much as it is a content problem. (This is another bit that I will bring up at the AI Summit panel, “Characters Welcome: Next Steps Towards Human AI“… it doesn’t matter how many decisions our AI can generate if we don’t have enough art and sound assets to express it.)

Anyway, this is a bit long for a first look. I will tag on some more as I experience more of the game. On the other hand, I will be shipping this back to GameFly in the next day or so. I need to check out a couple more titles before GDC in a couple of weeks.

If you are just jumping into this one article, make sure you click the tag(s) below to see if I have written anything else about Fable 2.

Left 4 Dead: Companion State Changes

Wednesday, February 25th, 2009

Another interesting observation in Left 4 Dead. In this case, it is about the state machine that the companion AIs are using. First, the observations:

The first clip in the video below shows me getting ready to leave the safe house at the beginning of the level. My companions did their usual “grab some stuff” behaviors and then lapsed into “random wander” idle behaviors (I couldn’t hit the screen shot key fast enough to show Louis standing with his nose in the corner like a punished boy.) When I went down the stairs to the door, I was mildly perturbed that they didn’t follow.

I then opened the door and shot the zombie standing outside. They still had not moved to join me. A zombie rushed me, I leveled him, and my sidekicks were still admiring the walls upstairs. Only when I stepped across the threshold did they move to join me (with Francis doing a completely unnecessary walk on the railing… but that’s a future post). Something not shown in the video is something I have experienced before. Usually, when I step across the threshold of the safe house, they are in quite a hurry to leave the room, even to the point of pushing through me to do so. It probably would have happened in this case if we had not been under attack at the time.

In the second clip, we were at one of the intermediate buildings on a map. We had run inside, closed the door, stocked up on ammo and health packs, healed ourselves and whatnot. I opened the door and left the building. The video picks up as I look back inside realizing that my pals didn’t seem to want to leave. This was different than the first clip in that the trigger was not leaving the room.

There was a Boomer behind the building that I could hear. Even when the Boomer came around the corner and started waddling toward me, they didn’t move. Only when I fired my weapon did they decide it was time to rush out.

Now, for the explanation. It seems that Valve is using an HFSM (Hierarchical Finite State Machine) or another such tiered approach (a behavior tree can cause this as well). There is likely a high-level state that we will call “In Safe House”. When in this state, other lower-level states are things like “random wander,” and “random comment.” The only thing that seems to override it is if they see a zombie outside a door (many of the safe house doors have those barred windows). They will actually engage and kill zombies outside the safe house from inside the door. Therefore, there is an “engage/kill” state that is contained under “In Safe House”.

On the other hand, another high-level state is “In the World”. It is in this state that the AI spends most of its time. Apparently, “stick with the player” and “defend the player” are only included in this high-level state. That is why they would not follow me down the stairs while I was still in the safe house or defend me when I got attacked. However, once I crossed the threshold, a message was sent to them to change states to “In the World” at which point, they were free to analyze their usual parameters such as distance (move to the player) and threats (defend the player).

Note that this would not have been a big deal if I had simply stepped out the door. Alternately, in other safe house situations, the random wander location is in sight of the door. Therefore, when I got attacked, they would have likely seen the zombies and fought back. This particular arrangement did not allow for that.

Now, I don’t know what happened with the building in the second half of the clip. Because that isn’t an “beginning/end of level” safe house, I would suspect the above rules don’t apply. Why did they not leave, then? I have seen other behaviors where they don’t seem to follow me like I would expect, but I have usually found other explanations for that (another post on that later). In this case, it would have made sense for them to leave along with me just like they tend to stick close in other circumstances. That leads me to believe that there was an artificial state in play that led them to believe that they were supposed to be there (or rather had yet to convince them it was time to leave).

Regardless, in this case, the obvious trigger was me firing my weapon. This was not the case in the safe house example.

Neither of these issues is dreadfully wrong in a gameplay sense. They are only noticeable in certain circumstances. And certainly the companion AI in L4D is better than some we have seen. However, when issues like these happen, they do make us pause and ask “what are you guys thinking?” Therefore, while the logistics of the game may not be affected too much, the perception of the game is. It breaks that coveted suspension of disbelief by making us ask (in typical gamer parlance) “WTF?!?”

A simple solution would have been to pay more attention to what was under the HFSM state of “In Safe House”. Alternately, have more than one trigger to transition from “In Safe House” to “In the World” would have been better. For example, opening the door is an obvious trigger that it is time to go. Getting attacked certainly is urgent enough to warrant attention as well.

Additionally, abandoning the rigidity of a HFSM could be the answer as well. Much of the problem would be solved by using a system of free-floating priorities such as what I describe in my book, “Behavioral Mathematics for Game AI”. In that arrangement, you can generally dispense with the state/transition model in favor of one that always has all possible actions in play through a system of calculated utilities and priorities. Something similar to this is probably already in effect in L4D for things like target selection, action selection (fight, heal, reload, etc.) and other actions. Therefore, extending it to cover the situations covered above would not be terribly difficult, in my opinion.

Anyway, all in all the companion AI seems to be fairly decent. As we AI programmers know, companion AI is a beast simply because of how involved the companion is with the player. There’s more scrutiny, more options of what to do, and far more potential for the “WTF?” moment and the ensuing frustration. I think that companion AI is the next frontier of game AI that we are already in the middle of. L4D is in the vanguard of this movement and doing an admirable job of it.

Remember to click the tags below for more Post-Play’em observations on Left 4 Dead and other related subjects!

Also, if you liked this article, please take a moment to submit this link to StumbleUpon, Digg, or Reddit. I would appreciate it, as would many other game AI enthusiasts!

Add to Google Reader or Homepage

Content 2002-2015 by Intrinsic Algorithm L.L.C.