Confessions Of A Game Artist

When people ask me what it is that I do, I cringe a little inside.

Not because I hate the attention, or dislike talking to people, but because it’s so hard to explain.

No, I am not a secret agent with a briefcase full of conspiracy fodder, or some kind of rocket surgeon with more sheepskin than social skills.  No, I am a lesser understood animal than those.  I am an artist who happens to make video games.

When I tell people that I am an artist, most people smile, ask if I am a painter, or if I am in advertising. I suppose it’s a frame of reference thing. Sometimes it’s easier to say I am a farmer.  People “get” farmers.

But when they also learn that I work for a video game company, the eyes glaze over.  People ask if I play games all day long, and they want to introduce me to a nephew who “also loves games.”

So right off the bat, odds are that anyone asking me what I do for a living isn’t going to have the foggiest clue how to continue the conversation in a meaningful, social interaction kind of way. Neither am I, for that matter.  It’s pretty rare I run into anyone with enough of a bridge between the left and right brain who understands digital art, let alone savvy enough to grasp the deeper complexity of 3d. And rarer still is the time when I bump into someone who I can strike up a conversation about game development, because, where to start?

I suppose I got myself into this situation.

From my first Commodore64 back when I was 8 years old, I knew that this was more than a fancy calculator, it had art in it. I saw it there on the screen, shooting down at me from space, trying to get at my little blocky tank. I knew somehow it could be used to make art; I just had to figure out the right button to push.

The first digital artwork I remember making was on a 320×240 monochromatic display of an IBM PC some such a few years later.  I couldn’t even save the blocky pictures of robots and time machines I drew with such meticulous effort a pixel at a time.  I got older, tech got faster, and now, more than 20 years later, I find myself sitting on top of computing horsepower that would have made my 8 year old self frantic with enthusiasm, or possibly retreat in horror at how absolutely complicated this thing called digital art can be.

It’s really not that scary. Sure, the tools I use have come a long way from programming the turtle to draw lines on the Apple2G of yesteryear.  Nowadays, they include the latest versions of all the software suites, a Wacom, nice dual monitors, Top of the line PC (that’s right, I am a dirty pc user.) Yes, the latest 3DStudio Max would look overwhelming next to the first 3DApp I started out with so many years ago.  But for the most part, one might mistake my desk for any digital illustrator or photographer’s.  However, where we in the game industry start to be culled from the rest of the digital herd is when we start to push our pixels into the third dimension, and then shove them across time itself.

That’s right kids, we’re talking real-time 3d.  An arena where if it’s not fast, it gets cut by some schedule-thumping producer, and if you don’t know how to make what’s left over look good, you’re out.  This is a place where your artistic integrity is constantly fighting the uphill battle with memory budgets, material instruction counts and polygon limits.  An industry where the customer wants real-time reflections, hi poly models, explosions and all of it synched up to online multiplayer at 30 or more frames per second. With sprinkles on top.

I know digital art, by its very nature, is pretty complicated.  I suppose any art form is.  But in the game industry we don’t just push pixels around in 2D, we breathe life into these little colored dots using complex shaders constructed from node based math functions. We assemble scriptable and dynamic materials in our game engine that do things from simple animation to the really fancy stuff.  People in my line of work bend software and hardware to the breaking point and beyond to accomplish what we do.  We construct the process and systems, the rules by which the game is played. Not just in the “I got the power-up and won” sense, but a vast infrastructure of information, content sharing, outsourcing, and tech management that we didn’t just inherit from our predecessors, we’re carving it out of cyberspace one zero at a time.  .

You may not be aware of it, but under the hood of your average game console, there is a LOT going on.

For example, our textures have to be pretty smart.  All images imported into the game have to be “power of two” aspect ratio; i.e. multiples of 4 in the height & width 64, 128, 256, 512, 1024, 2048, etc.  And when textures get far enough from the camera, they LODs (level of detail) down one MIP level to half size every predetermined multiple of camera distance to save rendering time and memory. 

Diffuse Map

We have to have a texture for every slice of an effect we want, such as a diffuse map for color information; specular maps which control where the light catches on the model, opacity maps for translucent sections, emissive for glowy spots, and then the normal map.  This texture looks pretty strange to the outsider, but to a game artist, it’s just like the matrix, you learn to read it.  Normal maps take the surface information from a very high poly model, and use RGB space to define the direction those polygons face.  That information is then mapped to a lower polygon model and the result you get is a few polys displaying surface information from a much higher fidelity model.

Normal Map

We also have a new thing called a channelpack texture, whereby since texture memory is very costly, if we have something like a mask, as long as it is grayscale, we can take that information, and stack it in an RGB image.  For example, I could have a model’s opacity map in the red channel, the reflection mask in the green channel, and the color tint mask in the blue channel.  Basically I have three channels of information at my disposal in one texture that I can use to jam information into the game where before, I would have to pay the cost of three separate RGB textures.  Quite the trick for saving texture memory!

The textures are assembled into the material, which is where you plug each texture into the appropriate slot. This is also where you get into the fancy script-controllable and dynamic effects.  Unreal3 has a node based material editor, where by pretty much anything you can do in Photoshop you can do in the material, and animate it or control it in a certain way. I’ve used the particle editor “color over life” curve editor to feed vector information back to the material to drive panning textures, animated fades, pretty much any dynamic controller based on one or more vectors. I broke the color information into the 4 components, so I could use the red channel to pan a texture, the blue channel to rotate something, the green channel to fade in and out over life, and the alpha channel to lerp between to instruction chains in the material.

The material is applied to the geometry, which has to be streamlined, too. It must have efficient spread of polygons, taking care not to waste geometry. There’s a real art to making a convincing model using the least amount of polygons, knowing just the right amount of polys to define a curve while optimizing a different section with less polys. Knowing how the texture will cover up this or that, knowing what the camera will see, etc.  If the texture mapped pixel to screen pixel ratio is off or uneven, one part of the model will be sharp, but the polygon next to it will have a stretched or distorted appearance. 

The game gets painted a pixel at a time, each pixel has what’s called an “instruction count.” Meaning, for every pixel on screen, the CPU or GPU has to run through anywhere from 20 to 100 mathematical instructions inside the material applied to that model.  While it’s cooking those instructions, it’s calling textures from disk, multiplying, adding, linear interpolating, scaling, shifting coordinates, panning and animating textures, reading input, passing information back out.  Basically a churning morass of information all to get the effect we’re looking for, dynamically in real time, at speeds that will suffice.

  All this to say that you can’t just push a button and make it happen.  One thing constantly builds on another, effects another, and impacts things we usually don’t even think about until our once smoothly running game grinds to a halt. We have to temper our creativity with efficiency and the knowledge of what we do and how it impacts other aspects of the game.  I mean, we only have so many milliseconds to spare per frame.  I can’t go gobbling them all up just because I want my Browning Hi Power pistol model to have so much detail that it could be a portfolio piece by itself.

It really comes down to the tricks you can use to deliver the content at speed.  Half of the folks working in the game industry are crackpot inventors of some type, full of problem solving skills they use to shoehorn the vastly growing content through a narrow bottleneck of hardware that hasn’t caught up with what the software can do.  It takes creative problem solving and, dare I say, “out of the box” thinking to get competitive content running at speed.

And that’s what’s really important here; innovation.  If you can “MacGyver” your way through the pitfalls of this digital jungle, you can pretty much be tossed into any situation and figure your way out.  Employers dig that.  I can’t tell you how many people I see fail because they only know how to do one thing, only one way. I force myself to be knowledgeable in as many facets of game development I can, so that I know where one aspect will impact another completely different wing of development. Sometimes I might not know what I am doing, but I have found that so often the key to success is just the determination and unwavering will to figure out how to succeed.

I wish I could have told myself all this back when I was 8 years old, taking apart the Commodore 64 looking for the “art” in it, much to my mother’s surprise. It’s a good thing I had it back together and cooking up a bunch of space invaders to shoot at by the time my dad got home.  Little did I know that while I had avoided a good spanking, I had also set myself off on this digital journey. And that today, I’d be looking back on those early days of tinkering inside the guts of gaming and wondering how it’s gone by so fast.  I might also tell myself then, and now, not to let the fact that people might not understand your job keep you from enjoying the act of telling people about it.

So if you run into someone and they say they’re a game artist with a hint of hesitation in their voice, realize why.  Dig a little into that shell, though.  If you can convince them to slow the gears down a little, and stop cracking some problem in their heads, you might just be rewarded with some interesting conversation.  They’ll appreciate the effort.