A brief primer on Human-Computer Interaction

I’ve been reading several books on Interaction Design and how to design usable computing interfaces. I’ve read Alan Cooper’s About Face 2.0 and Klaus Kaasgaard’s Software Design & Usability. The former is a sort of entry-level book to the world of HCI, and it takes the reader through all the various stages of design – from the initial inquiry to prototyping and usability testing. The latter is an interview book where the author discusses issues central to HCI and Interaction Design with leading people in the field. Here I’ll try to sum up some of the insights of these two books that struck a chord with me – and maybe add some of my own.

** A goal oriented approach

Alan Cooper’s first central observation is that for most users, computing technology is only a means. Not a goal in itself. This observation is central to understand how computers are perceived and used. Many HCI models (and thusly most software documentation) focus on the various taks associated with computers: pointing, clicking printing, saving; Cooper argues that all good design should begin with making clear what the user would want to accomplish using the software.

As longtime HCI consultant Stephanie Rosenbaum tells Klaus Kaasgaard:
“For most people, the computers that they use as tools are not central to their goals, so they are not willing to spend much time to understand computers.”

Much software is technically capable of any task that the user might have to do, but still fails as its designers haven’t considered which of these tasks are essential in achieving the users’ goals:

Software that enables users to perform their tasks without addressing their goals rarely helps them be truly effective. If the task is to enter 5000 names and addresses into a database, a smoothly functioning data-entry application won’t satisfy users as much as an automated system that extracts the names from the invoicing system.

Cooper states that the users’ most important goal is “is not to feel stupid.” You do feel stupid if you have to enter 5000 names by hand. And you do feel stupid everytime you see a dialogue box telling you that the program “has encountered an error” or has “failed to notify the library” or couldn’t “query the database” without telling why or what to do about it, and only offering a “OK” option.

What?! My program just crashed, it’s not ‘okay’!

** Contextual inquiry

Cooper, and most other interaction designers with him, argue that in order to identify user goals and thereby streamline design to fit the user, it is necessary to study the users’ daily routines. Observing and discussing the daily goals and tasks of the users in their own work environment can help the designer understand the needs and perceived needs of the user.

This sort of “contextual inquiry” is a very ethnographic encounter. A sort of “collaborative exploration” of the workplace over an entire day. The user tells of his routines and goals in a discussion subtly led by the designer to touch upon on the relevant aspects that he has identified. By encouraging story-telling and show-and-tell the designer can develop a better understanding of how work flows in the workplace.

Cooper warns that the designer shouldn’t discuss product-related technology with user (such as “what kind of wireless networking would you prefer?”), and avoid making the user a “co-designer” (by asking questions like “how would you like it”) as the user rarely have a complete conceptual model of the work he performs (Cooper argues that this is what the designer should focus on constructing).

Instead, questions should focus on the actual daily use of technology, preferences, motivations and priorities of the users:

What are the most common things you do with the product?

What do you like about it? What drives you crazy?

What is most important to you in your work?

What do you enjoy most about your work?

What do you do when you encounter a problem?

** Design paradigms

Once you have a clear idea of what the users’ goals are (having interviewed more than a few users in the manner described above), you can begin designing – that is the openended, creative process of finding a solution to the users’ problem – ie. the goals they want to accomplish.

A lot has been written about design in general but the essential element here is to base the design in the observations you’ve made, constantly testing your ideas against them. Many of the designers and consultants that Klaus Kaasgaard interviews state that it is very important not to begin coding a prototype of any of these early designs, because most programmers will grow very attached to any code that they have produced, even if it is just a 1000 line mockup.

Instead, it is best first to decide on a form before any coding is done at all: Use storyboards, dramatizations, mockups to present how you envision the use and feel of the program so that there is a final goal that the programmers can work towards.

According to Alan Cooper, there are three dominant paradigms in the conceptual and visual design of User Interfaces:

1) Implementation-centric
based on understanding how things work. The design mirrors the actual construction of the program. It often demands understanding of the program. Command line programs work in this way, and it is the way that is preferred by programmers themselves as they often find an inherent elegance and simplicity in the fact that the interface design reflects the system. I guess this makes it easier to fix interface bugs, as well.

2) Metaphoric
based on intuiting how things work. The design mirrors well known real life interfaces that the user is already familiar with, and therefore has some intuition from which to extrapolate how to use the interface. One of the most developed examples of such design is the desktop metaphor with its iconic trash bin, in and out boxes, files and folders. Other examples are music and video players that copy the interface from CD or DVD players.

3) Idiomatic
based on learning how things work. Idioms aren’t intuitive or sensible but they stick once they’re learned. We understand idioms like “kicking the bucket” or “beating around the bush” not by understanding or intuiting it (if you hadn’t been told, how would you figure out what “cannot hold a candle to” means?) but by memorizing it. Most GUI conventions are idiomatic – such as resizable windows and endlessly nested folders. Most computer games use idiomatic design as well.

In an implementation-centric paradigm, users are most often simply expected to know what a given program is capable of (or be able to read the documentation to find out). But since most users would rather be successful than knowledgeable, it is required that the functions of programs under both the the metaphoric and idiomatic design paradigm are easily usable without much prior knowledge – that the affordances of the program is made evident.

Affordance is a term introduced by the nestor of UI design, Don Norman, and is defined as “the perceived and actual properties of the thing, primarily those fundamental properties that detemine just how a thing could possibly used.” That is: how obvious an item’s use and mode of use is to the user. The way that things can communicate their use through their design.

For example af door bell usually has button the size of the tip of a finger, at just the right height to be easily pushed. According to Norman, it is basically begging us to push it. The same with power buttons on PC cabinets, they tend to be so big and tempting, anybody who has had their computer accidently turned off by a toddler will know the lure of the power button whispering “Push me!”.

With user interfaces on the computer, it is slightly more tricky. GUIs tend to be two-dimensional, grey and boring. But by using simple shading techniques, buttons can appear more pushable, windows can appear more draggable, by hinting that it can be manipulated in new ways, the user will often try, and thus learn new ways of using the computer.

I find that the best use of the idiomatic design paradigm takes place in computer games which often use tutorials to good effect, showing off the conventions and idioms of the game one by one. Often, the best part of the game play is where the learning curve matches perfectly the pace of play. Conveniently introducing new elements of play just as the player has memorized the old parts. Games like Prince of Persia: Sands of Time or Civilization get this learning curve just right. I think it should be perfectly possible to make tutorials for most programs in much the same way.

Also, see Seymour Papert’s interesting discussion of the failure of “edutainment“.

** A brief discussion of semiotics
I’ve found that the semiotic theory of Charles Peirce is helpful here in order to understand the creation and navigation of the signs used in GUIs.

Semiotics is the study of signs, not merely visual signs such as road signs or symbols or paintings – but also words, sounds and even body language. Peirce says that anything can be a sign as long as someone interprets it as ’signifying’ something – referring to or standing for something other than itself. Peirce found three main types of signs, categorized according to the relationship between the sign and the signified object. In a GUI that would be the relationship between a button and the associated function.

These three types of signs are

1) Icons
are signs that are perceived as resembling or imitating the signified objects (recognizably looking, sounding, feeling, tasting or smelling like it) – being similar in possessing some of its qualities. Icons are a central part of any GUI – for instance the use of the trash bin to signify deletion; an envelope to signify sending a message or a house to signify Home (whether it is a home directory or a start home page in a browser).

2) Symbols
are signs that do not resemble the signified object. Instead it is fundamentally arbitrary or purely conventional – so that the relationship must be learnt. Most GUIs use some symbols as well: The “refresh” button in the browser or the alert icon (the exclamation mark in a triangle).

3) Index
are signs which are not arbitrarily but directly connected (physically or causally) with the signified object. This link can be observed or inferred, such as footprints, echoes or a pointing ‘index’ finger.
Indexical signs in GUIs are often related to user actions. For example marking a selected object in a different colour or the cursor changing shape when associated with a certain action – such as copying or moving a file.

For a comparison of GUI icons in various modern operating systems, check out this wonderful chart.

The problem with these signs usually arise when the user interprets a certain sign in a markedly different manner from what the designer expects. Designers often call this confusion on themselves by using the same signs for several different functions. For example the magnifying glass which is both used as an icon for zooming in and out, but also as an index for searching – probably due to a cultural connection with Sherlock Holmes, cf. the Mac OS 9 search application named Sherlock.

For more on the use of semiotics in relation to GUI design, check interface designer Luke Wroblewski’s homepage and blog and this interesting essay on the issue.

** Actual implementation
When you have decided on the design paradigm you want to base your design on, and have developed the appropriate idioms or metaphors, affordances and signs to make the program easy to learn and use. And you have visualised how the user is going to use the program, you can begin the actual implementation of the code. Cooper has a fair few pet peeves and pointers on how to develop the GUI for any program:

- Avoid unnecessary reporting: Don’t use dialogs to report normalcy!
Ex. When you put something in the trash, you expect it to go in the trash. Don’t ask me if I’m sure. Just make sure I can undo it later.

- Ask forgiveness: Don’t hold up procedures but finish them and tell us of any errors afterwards!
Ex. If I copy several hundred files all at once, and one of them have a permission problem, then don’t stop the entire program to receive response on that file. Instead, copy all the other files with proper permissions and gather up the problematic ones in one error report afterwards. Extra points if there’s an offer to solve the permissions problem with that report.

- Modeless feedback: Offer relevant information immediately and dynamically.
Ex. Many people use the ‘word count’ function all the time. Why is it hidden away in a menu and a dialogue box when that information could be directly accessible from the normal document view.

In general: Minimize excise!
Excise is the extra work that doesn’t contribute to reaching a goal but are necessary to accomplishing it all the same. Excise comes in many forms, but especially these are worth noting:

- Flowstoppers: Remove flowstoppers such as dialog boxes, passwords, permissions, errors and unresponsiveness. And if flow stoppers are absolutely necessary in a given situation – make sure to make it as clear and as helpful as possible. For a comprehensive list of un-helpful flowstoppers, see the Interface Hall of Shame.

- Navigation: Reduce the number of places to go. Simplicity in presentation is apparent: Don’t force the user through submenus and dialog boxes to reach a certain function. Place functions according to use. Provide an overview of the functions. Avoid hierarchies (I, for one, continue to have great expectations to the use of tagging and integrated searching utilities. When can we expect an OS based on tagging rather than a hierarchical file structure?)

** Getting help
Once you have your program prototyped, or even finished, you can worry about documentation. Alan Cooper argues that documentation is mostly read by what he calls the “perpetual intermediates”.

He argues that since most beginners work relatively hard to acquire the basic computer skills and understand the basic conventions of the GUI, they quickly learn enough to succesfully accomplish the goals they want to achieve, thus becoming intermediate users. Most users are thus intermediates, only a few of which will have the drive and the interest to become proper computer experts.

These intermediate users tend to have an idea if something can be done through a given application, and are they will try to use the documentation to find out how. Cooper’s advice is to optimize documentation for these intermediates. And again, it is important to think in goal-oriented solutions: When users consult the documentation it is usually with a “How do I…” question on their lips but usually they will lack the proper technical vocabulary to know where to look.

As the usability expert Jakob Nielsen says to Klaus Kaasgaard: “Anytime a user goes looking for help, it is a cry of desperation, almost.”

Most users usually try to figure stuff out themselves before consulting the documentation when this fails they’re already frustrated, and as Nielsen states: “From the moment you ask for help, until you get it you start forgetting what you’re doing.”

Therefore, documentation needs to be as gentle a ride as at all possible. It is essential to have an index with lots of cross-references and a brilliant search function. The sort of search function that can guess that when you enter “make picture brighter” will refer you to the entry on the brighten-contrast tool, or when you enter “show time on message” will refer you to the entry on the timestamp function.

Also, to help and encourage the user to use keyboard short-cuts, these should be both easily learnable and usable (preferably with just one hand – but no space cadet keyboards, please!). All of these short-cuts should also be available as a list from the help menu, so that it is easy to look up in case you forget one.

** Conclusions…
With so much thought and deliberation on the matter of Human-Computer Interaction Design, it seems relevant to ask how it can be that there continues to be companies that produce software without thoroughly considering the way it will be used.

The computer scientist Terry Winograd touches upon this in his discussion with Klaus Kaasgaard: There is a constant rush to bring new products to the market. Just as quickly as you have something usable, it gets marketed, packaged and sold. No matter whether it is ready or not.

The programmer-turned-novelist Ellen Ullman reflects on this mode of constant hurry and turmoil which defines the entire computing industry. She says:

“We build our computer (systems) the way we build our cities: over time, without a plan, on top of ruins”

She argues that the strain between backwards compatibility and the integration of new features leaves little room for concern for polish and usability. New ideas develop so quickly that we have come to expect “new” all the time. We absorb change so quickly that we tend to forget how short a time much of this technology has been with us.

One illustration of this can be found in Kaasgaard’s 1999 interview with Winograd. Winograd had supervised Google founders Larry Page and Sergey Brin’s doctoral work at Stanford as part of a project under the Digital Library Initiative, and could recommend the new start-up Google as an example of a promising new technology. Kaasgaard later notes that “it does seem rather efficient.”

But Google’s phenomenal rise to fortune and verbdom isn’t simply because of its technological capabilities. Its popularity might well also be related to the fact that it makes the computer simpler and easier to use. Just type your query, and Google will find what you’re most likely looking for. Automagically.

It is that kind of simplicity that appeals to user. The computer itself is only a means. It is the goals that it allows you to accomplish that are interesting. The less central the computer is to the user, the less patience they will have with it. This is not necessarily a bad thing. Sometimes you learn more from the doubting late-adopters than from the ones willing to change their routines to accomodate new technology.

Some Interaction Design thinkers, such as Anne Galloway and Jean Burgess argue that we need to slow down this accelerating technology race. That calmness and even boredom also are relevant aspects to consider about interaction design.

Slow or not, design of Human-Computer Interaction does require attention. Perhaps most basically, Interaction Design requires a change in attitude towards software production in general. Perhaps even a turn away from crufty compatibility concerns or fancy features to a focus on useful simplicity.