Successful product development involves understanding the market, setting appropriate business goals, designing a product or service—an “offering”—that delivers expected value to stakeholders while satisfying and even delighting users, implementing the offering, and delivering it to the market with support throughout its life cycle. Unfortunately, too often one or more of these crucial phases is overlooked, taken for granted, or not invested in adequately. Market research firms and industry consultants have documented and quantified the problems that are often the result.

A section of a Forrester Report with the heading, “Haphazard Design Produces Poor Results,” cites data from a study of 158 Web sites. It states that two-thirds of potential buyers using these Web sites give up and abandon their attempt to purchase, and that companies do not establish measurable goals for site redesigns.¹ Such problems cost businesses millions of dollars in immediate lost revenue, and they frustrate and alienate users, causing potential future business to be lost as well.

With respect to interactive systems, consultant Alan Cooper concludes, “The single most important process change we can make is to design our interactive products completely before any programming begins.”² He further advocates a process of “Goal-Directed Design” in which precise descriptions of users and their goals are discovered and documented.

User Engineering (UE) is a significant evolutionary advancement in the process of developing products that satisfy and delight users, as well as the stakeholders who invest in bringing a product to market. It requires an understanding of value propositions—the values that users seek, how a product will provide those values, and the values sought by the business delivering it. The goal is a balanced design that provides value for the business, stakeholders, and users.

User-Centered Design (UCD) is a well-proven approach for delivering value to users and is a cornerstone of UE. Figure 1 shows the relationship between UCD and UE. UCD is based on the following six principles, which are shown in the figure. (1) Setting business goals includes determining the market, users, and competition. (2) An understanding of users is the driving force behind all design activities. (3) Everything a user sees and touches is designed as a total user experience. (4) The design is evaluated often by gathering user feedback, and this feedback drives design and development. (5) There is a relentless focus on the competition and its customers. (6) Managing for users through user feedback is integral to product plans, priorities, and decision-making. In addition, UCD specifies a multidisciplinary team of key design skills, and it identifies proven methods for activities, such as task analysis. Implementations of UCD, such as the practice within IBM, typically provide tools for activities that include heuristic evaluations, user surveys, and task modeling.

Figure 1

In addition to the fundamentals established in UCD, UE is based on delivering measurable value to the business, stakeholders, customers, and users. Delivering measurable value begins with understanding business goals, specified in a formal business model (e.g., a spreadsheet), and identifying design elements that have an impact on business measures. UE prescribes roles and the required skills for each role, along with activities to be performed and work products to be produced by each role, during each phase of a project. All design is based on a thorough understanding of users, their goals, success criteria, current tasks, and other factors, as discovered through in-depth user research. The design proceeds in formal stages from general direction, to conceptual design, to detailed design. Conceptual and detailed designs are evaluated with users before implementation. User research and design activities use computer-aided system engineering (CASE) approaches and tools, such as the Unified Modeling Language (UML**) and Rational Rose*, to create rigorous and precise design specifications. The design specifications address the total user experience for each identified user group. User evaluations are conducted at each stage of the design, using agreed-on measures against targets established to deliver desired values. Formal process assessments for each project provide valuable feedback to improve and calibrate the process over time. Finally, business executives are motivated to be involved and invest in the process as a result of the direct connection between design and achievement of business goals.

This paper provides an overview of the UE process—what it is in terms of its key elements, including the theory and definition behind the key concepts. UE is important to ease-of-use practitioners, product developers, and business managers, and the results of UE are important to users. Much of this relevance should be obvious within the paper. For example, the business model element of UE demonstrates the relevancy to business managers. Also provided are case-study examples that demonstrate benefits of using a rigorous, engineering-like process to design the user experience. However, a description of how UE is done in practice, which involves various methods, measures, and tools, is beyond the scope of this paper. The intent of this paper is to provide an overview of the fundamental concepts involved and an appreciation for the engineering-inspired rigor involved. For additional detail and ongoing updates as the process evolves, see the IBM Ease of Use Web site.³ For practitioners, in-depth tutorials are provided periodically at the ACM SIGCHI and Make IT Easy annual conferences. The tutorial materials are included in the proceedings of each conference. In addition, formal classes in UCD are currently offered by IBM Learning Services, and these classes are being updated to the UE level.⁴

Throughout this paper the words “product” and “offering” are used as synonyms and are meant to include hardware, software, services, and Web sites. User-Centered Design applies equally well to all types of offerings. For example, within IBM, UCD is actively practiced in the development of personal computers and servers, as well as middleware and Web sites. Frequently, a single offering will be comprised of all of these elements. An IBM ThinkPad* is a hardware offering that includes specific software for setup, configuration, and assistance, as well as a Web site for sales and support.

Historical perspective

For decades the dominant paradigm for product development was “design-implement-test.” In the software community it was more specifically “design-code-unit test” (DCUT), typically followed by some form of “functional” and finally “system” verification testing. The user experience aspects of a product were evaluated in these latter stages of development through “usability” testing. The problems with this build-test approach are now widely recognized, with some reports citing cost factors as great as 100:1 to fix a problem after a product is made available versus finding and fixing it early, during design.⁵

At the same time, there has been a growing awareness of the business value that can be derived from creating a truly usable product, be it hardware, software, a service, or a Web experience. For example, studies show that for every dollar a company invests in developing usable software, it might receive $10–$100 in benefits.⁶

These factors led to the recognition that problems should be avoided during development and that the design phase is where “usability” should begin. Thus, UCD came to the forefront as a superior approach to simply doing usability testing at the end of development.⁷

The practice of human factors engineering within IBM spans decades. This practice evolved, and, in parallel with advancements in the field, was formalized as UCD within IBM in the early 1990s.⁸ UCD is now practiced by many project teams throughout the company. It has contributed to a number of significant product successes, yet there were situations in which UCD was reportedly practiced and significant product problems still resulted. Upon close examination, it was determined that problems were frequently caused by incomplete or inadequately skilled staff performing the process “on paper” but not in earnest, lack of understanding of what was really required to perform the process correctly, and various other causes indicative of not actually performing User-Centered Design. In some cases, development teams hired a human factors engineer, performed some usability tests, and concluded that they had followed UCD.

In parallel with the recognition that more rigor was needed in the adoption and application of UCD, competitive and market pressures have emphasized the importance of the user experience. Customer satisfaction surveys typically rank ease of use in the top three of product attributes desired by users, and strong correlations have been shown between brand loyalty and customers who are highly satisfied.

Within the IBM design and development community, initiatives have been underway since the early 1990s to develop more rigorous, thorough, and complete design methods focusing on the user experience. UE is concerned with providing value to users of products and value to developers of products. Approaches such as user object modeling, initially used in development of the Common User Access (CUA*) and OS/2* Workplace Shell* interfaces,⁹ were further developed into the Object View Interaction Design (OVID) method.¹⁰ The OVID method draws from software engineering and related tools. It utilizes UML¹¹ diagrams to precisely specify the key aspects of a user experience, such as the user objects, their properties and relationships, and views of those objects that enable performance of user tasks. Because OVID models the user experience, it is not limited to software offerings. In fact it has been used for hardware, software, and Web site design projects.

Concurrent with its adoption as a key method within UE, OVID has been extended to support modeling of user groups, their goals, and their success criteria. User goal models are becoming widely recognized as being crucial to satisfying users' expectations, even more so than traditional task models.

IBM uses a comprehensive development process called IPD (Integrated Product Development), in conjunction with an equally comprehensive marketing management process that identifies opportunities and specifies attributes for offerings. Within the last few years, UCD has been fully integrated into IPD and provides the foundation on which the practice of UE is now evolving. IPD and UE in particular are supported by a variety of design, development, and project management tools. The current UCD Workbench is evolving to become the UE Workbench, which will provide tools supporting the entire UE process from requirements gathering, goal modeling, and task analysis, to prototyping and evaluation by users. Reporting tools that clearly depict current status in terms of measures against established targets are also being developed. By the end of 2003, UE will be widely deployed throughout IBM. It will be fully integrated within IPD and will be supported by a productive set of tools.

UE is evolving from UCD to firmly root the total user experience and the process for creating it in a rigorous, repeatable method, based upon integration with business and user goals. UE is expected to provide more consistently successful product results, providing tangible return on the business investment to create a highly satisfying user experience. A simple example of such tangible return is shown later in this paper in the description of a business model. For IBM, it became clear that in order to establish ease-of-use leadership as an aspect of the IBM brand, UCD had to evolve to become a critical success factor, rather than an optional or encouraged activity. UE offers incentives to entice business managers to seek the benefits it can provide, both to users and to the business. For example, in one recent project using elements of UE, additional revenue of 1.3 million dollars was realized almost immediately, and in another project, $200000 in cost savings was realized in the first year. These and other examples are included in the section on the case study later in this paper. Details of the current deployment of UE in IBM are also provided later in this paper.

Overview of UE

UE is concerned with providing value to users of products and value to developers of products. User value can be expressed in terms of how well a product satisfies users' desires and expectations for function, reliability, ease of use, security, and a myriad of other attributes. IBM has a long history of identifying such attributes, along with users' expectations and current satisfaction, specific to certain types of markets, products, and users. User research identifies users' goals and success criteria. Users' expectations and current satisfaction are assessed for each attribute using surveys, and are measured in terms of weighted means, yielding the Net Importance Index (NII) and the Net Satisfaction Index (NSI), respectively.

Product developers seek to fulfill user expectations, and they seek to satisfy their own value expectations, such as selling more products, gaining market share, establishing brand recognition, reducing and avoiding costs, and building a base of loyal repeat customers. These expectations can be seen as an equation with users on one side and the product supplier on the other. Benefits of use and return on investment are a balance between the two sides. The role of design is to understand the values on each side and to define products that provide value to both as much as possible.

UE defines a specific product development process that involves market and business planning, product management, design, implementation, deployment, and support on the supplier side, and on the consumer side provides products with user experiences to customers and users. The role of product management is to invest in an offering, and the goal is to achieve a high degree of success with the lowest possible risk. The role of product design is to define an offering that achieves business goals and that creates a high level of user satisfaction by enabling users to achieve their goals with ease. The role of implementation is to create the offering with fidelity to the design, thereby ensuring a successful product. The deployment phase verifies the product in the user's domain. The support phase is tied to the deployment phase but does its own report after the product has been released for several months.

The distinction between customers and users can be an important one. Customers purchase a product. Users live the experience of using it. Sometimes they are the same individual; frequently they are not. Customers and users can have distinctly different values, constraints, and goals, which must be understood and addressed as part of creating an offering and bringing it to market.

The key to developing successful products involves understanding the value propositions for all parties: product developers, customers, and users. UE involves identification of measures, establishing meaningful targets, and rigorous ongoing evaluation to assess position with respect to the targets at every step.

Product developers are normally a business and therefore employ business measures to manage their process and establish success or failure. Business measures typically involve revenue and cost. Revenue factors often include aspects such as price of the product, volume, and rate of adoption. Cost factors might include development, service and support, and sales channel costs. Measures of success are often stated in terms of profit, market share, and user satisfaction, which are ultimately reflections of return on investment (ROI), market leadership, and customer loyalty, respectively.

UE specifies a series of activities to be performed and work products to be produced at each phase of a development project. The development process within IBM is IPD, which consists of these phases: preconcept, concept, plan, develop, qualify, launch, and life cycle. UE parallels these phases with the user experience design activities appropriate to the IPD phase. OVID comprises a set of methods for documenting user research (discovery), creating and documenting a conceptual design (abstract design), and creating the final design specification (realization design). Figure 2 shows the relationships between these phases. A more detailed explanation of the key elements shown in Figure 2, for example, the six UE phases, can be found in the section “Key Elements of UE,” later in this paper.

Figure 2

The theme of measures and targets is pervasive in UE. It underpins every phase and is reflected in all major activities. It is one of the most significant advancements over prior design processes. Business goals are reflected in business measures that drive the process. They provide incentive for business executive involvement, investment, and commitment. User goals are reflected in user-based measures that evaluate progress at every phase. User-based measures include satisfaction and various task-oriented measures such as task completion rate, task time, error rate, and assistance rate. These measures provide early in-development indications of the likelihood of success, allowing time for corrective action and avoidance of costs that escalate rapidly when problems are found after implementation. Ultimately, they reflect success in the marketplace and foretell customer loyalty.

UE utilizes approaches and tools inspired by the engineering disciplines. Precise notation and specification of the design is accomplished through creation of models and the use of UML. UML is utilized in the creation of goal models, task models, use cases, user object models, and view diagrams. Figure 3 shows a portion of a UML diagram for capturing and modeling users' goals. UML-based techniques specified in the OVID method are useful in the design of software and hardware offerings as well as Web sites, the out-of-box experience, and other aspects of the total user experience.^12,13

Figure 3

UE also prescribes methods and tools to achieve rigorous and comprehensive user research, to perform ongoing evaluations of an evolving design with actual users, and to report ongoing measures against established targets.

A significant advancement in design discipline. The practice of UE is expected to provide a significant advancement over current User-Centered Design approaches, and early case-study results support this expectation. Specific opportunities for advancement include the following:

• Achievement of business goals—Business goals drive the process, and the process is focused at every phase on achieving business targets, which include satisfying users' requirements.
• Executive support and investment—Executive involvement is facilitated because executives can assess a direct impact on the bottom line, which in turn makes them more likely to invest in the process. This is a key component of success. Without this commitment the project team may lose focus.
• Key skills required to perform activities—Precise roles are defined in terms of core skills required, activities to be performed, and work products to be produced at each phase of the process.
• Rigor and completeness are inherent in the process—The overall process is rigorous and structured through specification of activities performed by roles, which are precisely defined by their skills. The activities produce precisely defined work products, including measures to assess success. Measures are assessed against targets at each step, and corrective actions are initiated as needed.
• Proven approaches are specified in documented methods and are facilitated by tools—Activities are supported by detailed descriptions of proven methods, and in many cases are further facilitated by the provision of tools that implement the methods. Through iterative refinement, these methods and tools are improved over time for the benefit of all project development teams.
• Success is continually assessed against established targets—Measures are identified at the outset, and targets are established that satisfy business, customer, and user goals. Business measures might involve factors such as percentage of revenue increase and cost reduction. Customer and user measures typically include satisfaction, task completion rates, error rates, and so forth. The entire process is then driven to meet these targets at various phases throughout the life cycle of a product.
• Iteration facilitates success and improves the process—The process is based on a high degree of iteration to meet targets, including iteration on the process itself to successively refine methods, measures, and the ability to set useful targets. Through calibration of models and methods, and recalibration over successive cycles, steady improvement of the process can be achieved over time.

The key elements of UE

UE employs several key elements derived from User-Centered Design, engineering disciplines, and project management. This section describes the key elements, including: guiding principles; team, roles, and skills; market, business, and user requirements; use of a business model to drive design; design of the total user experience; and the design process, with specifics on project phases, activities, measures, and targets.

Guiding principles. UE is based on a set of principles relating knowledge, design, and process. Additional detail relating to many of these principles is included later in this paper. The principles pertaining to knowledge are:

• Business—Business goals must be well articulated and identify how a design will win in its intended market. There must be absolute clarity on business goals and vision for an offering, such that a UE team can be focused and directed.
• Market—An intended market must be well understood in terms of users, how they value offering attributes, and competition. Market segmentation must be well-defined, with a clear statement of particular market segments to be targeted. Various groups anticipated to be users of an offering must be identified and described. A primary competitor must be identified so that the UE team knows which offering to match or surpass.
• Users—An intimate understanding of what users want to do and how they envision doing it is required. Each group of users must be identified and characterized, including specification of their goals and how the goals are assessed. The total user experience must be defined for each user group. Details of the tasks performed by each user group must be recorded together with all important measures.

The principles for design are:

• Skills—A multidisciplinary UE team creates the design with continuous user participation. A UE team must have the required skills to produce a leading design, and these skills must be in place early in a project. Required skills are associated with specific roles.
• Measures and targets—Design of a total user experience must progress against specific measures with challenging targets. Business and user measures must drive a design. These measures, and their associated targets, must be established during the initial phases of a project. Measures must be collected at each stage of a design process and evaluated against the associated targets for an offering.
• Evaluation—A design must be evaluated against business, market, and user requirements, and against their respective targets, at each phase of the process. Design evaluations must be performed at each stage of a design process to ensure that the design is in conformance with business and user expectations. Evaluation results are used to drive iteration and refinement. Where measures are not tracking against targets, the team must take appropriate corrective action.
• Attraction—A physically stimulating and enticing experience encourages user adoption and regular use. Every physical aspect of an offering must be in harmony with users' operation and style expectations. Users must feel very comfortable with an offering. They should find it attractive and totally in keeping with their life styles and personal images. This affinity should be so strong that users are proud to be associated with an offering and demonstrate great loyalty.

The principles pertaining to process are:

• Information—Specific work products created through proven methods must be used to convey a design through successive phases of a project. All aspects of a design process must be documented through work products. Activities must be performed to create work products, which are specified for each phase. Specific methods should be specified for performing many of the activities.
• Collaboration—All work products, including measures and targets, must be easily accessible and be used by the entire project team. A design must be readily accessible by all members of a project team, who must be advised of all significant changes, including availability of completed work products, and specifically, status with respect to measures and targets.
• Improvement—A formal mechanism must be employed to capture project experience and enhance the UE process. A UE team must record issues and concerns that the members experience while performing the process. Before project completion, the experience must be analyzed and recommendations for improvements made.

UE team, roles, and skills. A properly skilled and staffed design team is crucial to the success of a product. This subsection describes the UE team roles, how skills relate to the roles, and how the UE team relates to an overall product development team.

UE roles. The UE team is a multidisciplinary team consisting of the following roles: user experience leadership, market planning, user research, user experience design, visual and industrial design, and user experience evaluation.

Each of the UE roles has assigned responsibilities for performing activities, leading the production of work products, and assessing specific measures. A role is not a job title and often may not equate to a single person. An individual may perform one or more roles, providing the person has the required skills for each role. On a large project, multiple individuals may be required in the same role, in which case one of them should be assigned overall responsibility for that role.

User experience leadership. The role of UE leadership has the overall responsibility for superior user satisfaction with the total user experience. The purpose of the role is to ensure that the intended users are effectively represented and accommodated in all aspects of the project, especially design. Most important, this role ensures that users are fully considered in all discussions, especially where trade-off decisions may be made.

The user experience leadership role sets the overall vision from the user's viewpoint, identifies desired user experiences for each aspect of the total user experience, establishes and leads the UE team, ensures that the design is on schedule and meets targets at every phase, and initiates corrective actions when deviations are observed.

Individuals in this role must have great passion for appreciating users' needs and fully satisfying them. They must be totally familiar with the design process. They should be proven leaders in setting direction and motivating, as opposed to having primary skills for activities that are oriented more toward project management, such as ensuring resources and funding.

Market planning. Individuals in the market planning role provide the market and business requirements to be addressed by the project. They define the primary goals, provide a high-level description of the offering and market segmentation, and position the offering against competition. These activities form the backdrop and reference point for all design activity.

The market planning role must provide sufficient facts and information for the other members of the team to identify and engage actual users and to balance business objectives against a prioritized set of market attributes. The evolving design must continually accommodate this balance. Market planning also establishes clear user-oriented targets, such as user satisfaction, that must be achieved for the project to succeed. Throughout the project, market planning provides strong linkage with the production of marketing deliverables as well as with the service and support organizations. Individuals in this role develop the marketing messages at an early stage and ensure that these are reflected both in the offering launch activities and marketing collateral. They also formalize all branding requirements, which must be fully accommodated by the evolving design.

User research. Those in the role of user research gather and record all user requirements, including who the users are, what they need to do, how they wish to do it, and their success criteria. They document the manner in which users currently perform their tasks and describe the various groups of users, along with the tasks that each group performs. They strive to understand how users might wish to perform their tasks, especially if there is no previous implementation or a new approach needs to be taken. This gathering of information is done with users, frequently within their current work environment, through observation, interviews, and other techniques, and it is done with individuals and with groups of users.

For each task, those in the user research role qualify the level of expertise required for successful completion and quantify the time users would expect the task to take. This activity records the entire user context, including social, cultural, and emotional factors. User research also helps establish objectives against which the offering will be evaluated. This is critical to the establishment of measures and targets for the project. Although user research activity is greatest during the early phases of a project, it continues throughout design and development to provide clarification of users' expectations.

User experience design. User experience design creates a design that satisfies the business and user requirements received from market planning and user research, and at the same time incorporates the feedback received from the user experience evaluation role. It creates a design direction statement that provides a very high-level picture of the envisioned design approach. Individuals in this role clearly define the major parameters that will drive the design and the priorities that will be used when making trade-off decisions. Based on an approved design direction statement, they create a conceptual design that uses scenarios and stories provided by the user research role, and they begin user evaluations by working with the user experience evaluation role.

User experience design creates a user model that will form the hub of the design. User models must be readily recognizable by the intended audience, who should immediately appreciate the behavior and interaction. The user model includes definition of user objects, relationships, views, and task flows. Low-fidelity prototypes, such as paper sketches, are created to support user evaluation of the conceptual design.

Based on user evaluations, the conceptual design evolves into the detailed design. This evolution involves designing the physical presentation of the objects and their attributes, provision of mechanisms for interacting with the objects, and the overall navigation through the user space. High-fidelity prototypes are produced to evaluate all critical aspects of the detailed design. The resulting design documentation is the specification for the implementation team. It should be thorough and complete, such that the implementors rarely need to obtain clarification.

Visual and industrial design. The role of visual and industrial design provides the overall appearance, form factor, layout, style, and balance of the offering, within the context of branding expectations. The physical appearance of an offering cannot be added at a late stage in a project. It has to be fully integrated into the conceptual design. Those in this role work in partnership with user experience design, and their work addresses everything the user sees and touches, including the consistent signature of the advertising, packaging, and product design.

When the project is part of a broader brand offering, visual and industrial design ensures that there is total harmony with the overall branding scheme. The individuals in this role rely on branding and other presentation standards, which are typically defined within style guides, and in some situations, augment or even create these style guides.

This role takes in a specialized understanding of visual, aural, tactile, safety, and convenience criteria, with concern for the user, as well as a practical concern for technical processes and requirements for manufacturing, marketing opportunities, cultural differences, economic constraints, and distribution, sales, and servicing processes. This role ensures that design recommendations use materials and technology effectively and comply with all legal and regulatory requirements. It is active throughout the project and makes strong contributions at every design phase.

User experience evaluation. Individuals in the role of user experience evaluation plan and conduct user studies throughout the project to continually assess progress of the design in meeting users' goals and targets. They organize and perform studies to evaluate designs using well-proven methods. During conceptual design, it is important to evaluate alternatives and perform iterative design.

These individuals collect both qualitative and quantitative data, including such measures as task completion times, success rates, and user satisfaction. Using the results and their relationship to project targets, they summarize the findings and make recommendations for the design team to address. They take into account costs, project schedules, and, most important, satisfying users' expectations. UE is based on the principle of informed design.

This role also performs competitive evaluations, which must be conducted whenever there is a significant change by competition, such as the availability of a new product, or at the beginning of a project if there is no up-to-date competitive evaluation. These evaluations are performed from the user's perspective and take into account all aspects of the total user experience.

Relationship to the project team. A project often consists of multiple teams, each addressing a specific aspect such as the business, design of the offering, implementation, or support. A project team typically coordinates these subteams and manages the project as a whole. In this setting, the UE team is the design team, and the person performing the user experience leadership role is a representative on the overall project team.

Other individuals on the UE team may have roles on other teams as well. For example, the market planning role will typically be a member of the business team, and the visual and industrial design role may be a member of the implementation team.

Skills definitions. Each role is defined in terms of activities to be performed and skills required in performing them. In general, all members of the UE team are expected to have mastered a set of common skills, including a comprehensive understanding of the UE process, methods, and tools. Because of the rapidly expanding worldwide e-business environment and the broad focus on assisting users with special needs, UE also specifies a common level of skill in aspects of both globalization (refers to worldwide cultural considerations, such as language and translation) and accessibility (refers to considerations for helping users with special needs, for example, by providing assistive devices).

In addition to the common skills, role-specific skills are specified in terms of abilities, formal education, and experience. Abilities identify general types of activities that the role should be capable of performing. Formal education identifies specific university degree programs and other certifications that are generally good indicators of the ability to perform. Experience level assesses an individual's knowledge and capability to perform with and without assistance, and is described using a six-point scale. Over time, UE will include project profiles and tools for assisting project managers in deciding on the number of people required for each role and their required experience levels.

Key work products to inform design. UE is based on the principle of informed design. A design must be informed about opportunities in the market that the offering intends to address, the goals of the business sponsoring the offering, and the wants and needs of intended users of the offering. Four key work products are provided during the initial phases of the process specifically to inform the design. They are market requirements, business requirements, user requirements, and the business model.

Market requirements. Market requirements provide a comprehensive outline of market factors that will drive the design with the objective of providing market insights as the basis for user research. The market requirements characterize the market in terms of opportunity, trends, and anticipated users, and identify the key value propositions in terms of factors such as price and value, availability, packaging, performance, ease of use, customer assurance, life-cycle costs, and social impact. Buyer characteristics, including behavior, wants and needs, decision processes, and buying criteria, are also described. The competitive landscape is documented, including key competitors, their perceived strengths and weaknesses, and positioning of the proposed offering within this landscape. Finally, environmental considerations, including economic, industry, demographic, political, and social factors, are identified.

Business requirements. Business requirements specify the business goals for the project, including financial, volume, geographies, markets, audience, and target metrics. This information establishes the basic business parameters that need to be successfully addressed when designing the total user experience. Business requirements define the business goals relevant to the marketplace and the envisioned offering, and describe a worldwide market segmentation schema, listing prioritized markets, countries, and offering or purchase attributes. This work product specifies how business goals are to be satisfied, including branding strategies, and identifies anticipated user groups along with their key characteristics, such as skills and experience. The proposed offering is positioned within the market segmentation schema, identifying gaps and overlaps for current offerings, the envisioned offering, and competition. User satisfaction attributes and their priority order are identified, along with current customer satisfaction and targets for the envisioned offering. The market planning role is responsible for gathering and maintaining all market and business requirements.

User requirements. User requirements provide a clear articulation of how users currently work, what they expect to be able to do, and how they wish to do it. This ensures that the design is based on a comprehensive understanding of users' expectations. User requirements identify user groups and their key characteristics in terms of skills, abilities, experience, and special needs. This work product describes users' working environments, the size and importance of each user group, and their goals and success criteria. Inter-relationships, tasks, common task strategies, and task-based measures and targets are also identified. Tasks are ranked by importance to users, and scenarios and stories are recorded to clearly communicate and demonstrate the users' requirements. Finally, the overall user environment is characterized, including the physical, cultural, and social context, and candidate measures for evaluating the design throughout the project are proposed.

The user research role leads a series of user-involved activities that provide this information. These activities may include observation of users in their current work environment, interviews with individuals and groups of users, and Decision Support Center (DSC) sessions. In a DSC session, a group of users provides anonymous answers that the entire group can see, thereby facilitating the consensus, brainstorming, discussion, and decision-making that informs user research.

Business model. A business model is a crucial element of the overall business requirements. It uses a spreadsheet to identify key measures and quantify the business proposition, positioning baseline data against competition and targets for the offering. It enables a detailed financial evaluation of the offering and supports prioritization for design trade-off decisions.

A simple example for an e-commerce Web site is shown in Figure 4. The goal of this Web site is to enable users to make a purchase. The number of users who can successfully find what they want and complete the purchase process is clearly a significant factor in business success. The model starts with “site visitors per month” with a baseline value obtained, for example, from actual site statistics. The “wish to purchase” value indicates the percentage of those who visit the site who actually wish to purchase something. This value might be derived from user surveys. The next three values, “find what they want,” “acceptable delivery date,” and “able to complete purchase,” reflect specific tasks that the design can impact. Based on existing site statistics and estimates from broader industry surveys, some baseline values can be established. The result is the total revenue that reflects the conversion rate, which is the percentage of those wishing to make a purchase who are actually successful. Competitive data are then plotted against the baseline. These data might be acquired by estimates based on industry surveys and reports or in-house evaluations. Finally, targets are established for the planned offering. It becomes relatively obvious how designing to enable key user tasks can ensure business success.

Figure 4

Although this simple example is based on a Web site, the business model approach can be applied to all types of offerings, including software, services, and hardware. The measures will be different in each case because they reflect the value propositions identified in the business requirements. Value propositions might include revenue, cost savings, cost avoidance, market share, and others, as appropriate to the business and type of offering being developed.

A model such as this can be extremely effective in driving a design and achieving business goals in return. Designers understand quite well the measures involved in achieving tasks such as “able to complete purchase,” and they know how to design to achieve specific targets. Common measures such as task completion rate, time on task, and error rates are direct reflections of the design. They can be effectively evaluated even at the design prototype level, allowing for design iterations to meet the targets prior to implementation.

Even if such a model is not perfect the first time, it can be calibrated over time. Through several cycles of a project, each time applying some degree of recalibration, a model should become an accurate predictor of the ability of a design to meet business goals.

Total user experience. UE focuses on designing the total user experience (TUE). This experience consists of all aspects of a product or service as perceived by users—from a user's initial awareness of an offering, through additional discovery, ordering, fulfillment, installation, initial use, day-to-day use, service, support, upgrades, and end of life. These aspects are depicted in Figure 5.

Figure 5

The aspects that make up a TUE typically vary for different types of offerings. The TUE shown in Figure 5 would be typical for a personal computer, for example. The TUE for an e-commerce Web site would be different. Although it would still have an “awareness” aspect, the remaining “Acquire” aspects and others under “Use” and “Upgrade” would likely not apply. Each project must define the TUE aspects appropriate for the offering and its intended groups of users.

Similarly, specific users may not experience all aspects of the TUE, so each aspect must identify its intended user group. From the example of a TUE for a personal computer, it can be seen that in an enterprise environment, the users of PCs are rarely the purchasers. In this case, there may be two or more user groups across which the TUE aspects are distributed. UE prescribes how these details are understood and how they affect the resulting design to benefit all users.

Often, product design is performed without adequate consideration for supporting elements, such as promotional material, packaging, and support services. However, the goal must be for all users to have a consistent and supportive experience with everything they see and touch. In other words, all elements of an offering with which users interact must be the focus of explicit design activities.

UE addresses the total user experience of each group of users by incorporating many well-established practices from User-Centered Design, by requiring a multidisciplinary design team, and by specifying activities and work products that address and communicate all aspects of the user experience throughout the design process.

The design process. UE specifies a design process consisting of three phases: design direction, conceptual design, and detailed design. Implementation is based on the final design specification after significant user evaluation. The user experience design role and visual and industrial design role are responsible for defining and communicating all aspects of the design. A thorough understanding of the market, business, and user requirements is the prerequisite.

Several of the methods prescribed in UE are adopted from the OVID design method, an overview of which is shown in Figure 6. As described earlier, OVID is actually a set of methods for discovery, abstract design, and realization design. These OVID-based activities directly parallel UE phases, as shown previously in Figure 2.

Figure 6

The user research described previously is captured in the form of UML diagrams. Class diagrams are used to specify user groups and their characteristics, goals, and success criteria. Activity diagrams describe the key decisions and activities involved in the way in which they currently achieve their goals (their current tasks). These aspects are the primary components of a user requirements work product.

The design team carefully analyzes the user requirements and develops use cases that specify the functions to be designed. Use cases are identified by employing heuristics such as function reuse, and by looking for logical checkpoints in an overall task flow. The use cases are the specification for and primary driver of the design. Design direction statements are developed, using prose, diagrams, scenarios, and stories, to communicate the design vision enabled by the use cases.

The conceptual design develops a user object model, various task models, and user views, at an abstract level. The user object model defines objects that users will perceive and use in accomplishing their tasks, such as a hotel composed of rooms, guests who reserve rooms, and a reservation and folio that capture the relationships between the hotel and guests. Task models identify which users interact with these objects and describe their interactions in accomplishing tasks, such as making a reservation, checking into the hotel, and checking out. Other UML diagrams may be employed to specify crucial interactions and object state transitions. At this level the design is platform- and device-independent. However, it can still be evaluated with users through the use of low-fidelity prototypes, providing very useful early feedback and enabling design changes to meet users' expectations.

The detailed design evolves from the conceptual design by adding platform- and device-specific details, including physical appearance and interaction. Medium and high-fidelity prototypes are developed during this phase to further evaluate and iterate on the design.

The final result of the design process is a set of diagrams and physical appearance details that serve as a specification for implementation. This rigorous process ensures a complete and concise specification; moreover, it ensures that user expectations and business goals are met.

Phases and checkpoints. Each project is made up of discrete phases during which specific activities are performed. These activities result in work products that capture the work and transport it between phases. Figure 2 provides an overview of the phases.

Development projects progress through a set of discrete phases in which specific aspects of the project are accomplished. These activities result in well-defined work products. The phases identified for UE correspond to those of most development processes, but are given titles that focus on the design activities performed.

Each phase is normally completed with a formal checkpoint in which all progress is reviewed, including that of design. Apart from checking that the key work products for a phase have been successfully completed, achievement of target measures is also verified. At the end of the checkpoint review, a decision is made to continue to the next phase, perform remedial work, or cancel the project.

During most phases, it is usual for design and user evaluation to iterate until the targets for key user measures, including satisfaction, have been met. All projects should make allowance for this iterative approach and ensure that it is accommodated in the overall schedule. Concern that iteration will consume precious cycles of the project is not justified because the availability of modern techniques permits several iterations to be performed in a short period. More important, the project will proceed more smoothly in later phases with more likelihood of overall success.

UE is highly focused on the early phases of a project in order to minimize cost and risk and to create the best possible user experience. This focus allows a more complete design to be transferred to the implementation team.

Following are six discrete phases, which are depicted graphically in Figure 2 and discussed in detail below.

1. Business opportunity—marketplace opportunity related to business goals
2. Understanding users—comprehensive understanding of users' needs
3. Initial design—conceptual design that reflects every aspect
4. Development—complete definition of actual design
5. Deployment—final validation of implemented design
6. Life cycle—user feedback from final product

Business opportunity. The business opportunity phase identifies a marketplace opportunity and relates it to business goals. An initial UE plan is created to outline resources and schedules.

The business opportunity phase consists of an analysis that results in a marketplace segmentation schema and prioritized investments, captured in a business plan. This business plan provides the fundamental information necessary to frame, initiate, and drive a development project that results in a successful business venture. The business plan should be based on a sound understanding of market conditions, customer and user requirements, and the competitive landscape.

Customer and user requirements may be described in terms of price, value, channel delivery, packaging, applicability, ease of use, assurances (delivering on product claims), life-cycle costs, social influences, standards, and sanctions. These aspects affect customers' competitive buying decisions and resulting market share.

It is necessary to understand how customers and users rate the importance of each of these attributes. This importance, in conjunction with business goals and competitive data, is used to prioritize product attributes for the project. It also establishes measures that drive a winning position for a new offering in the marketplace. These metrics are used throughout the project as targets to be achieved at every phase.

Understanding users. The understanding users phase establishes user requirements in accordance with the business and market requirements. The UE plan is finalized, and the design direction is set.

This phase establishes the initial approach, risks, costs, schedules, and financial impact of the proposed offering in response to the opportunity articulated in the business plan. It assesses project potential in terms of appropriateness for the marketplace and accomplishment of business goals, and culminates in a decision to proceed, defer, or cancel.

The complete UE team is established along with the project schedule. The user experience leadership role should be assigned to a key member of the project team. Schedules are set, consistent with the overall project plan.

The user research role collects user requirements and documents them in a user requirements work product. This work product identifies user groups and their characteristics, their goals, and success criteria. In addition, the work product identifies worldwide language, cultural, and accessibility needs. It also identifies competitor strengths and weaknesses and sets ease-of-use objectives and validation metrics.

Based on these user requirements, the design team creates the design direction statement for the offering. This statement documents the envisioned design direction, proposed architecture, and ease-of-use marketing messages. The user requirements and design direction statement are validated with users.

Initial design. The initial design phase establishes the conceptual design, which is exemplified through low-fidelity prototypes. These prototypes are evaluated with users, and appropriate changes are made before creating the detailed design in the next phase.

The conceptual design addresses the total user experience, including the user model, task models, task flows, scenarios, and abstract views, plus advertising concepts, outlined terms and conditions, candidate purchase options, packaging mock-ups, and the out-of-box experience. The design team should also ensure that it satisfies worldwide language, culture, and accessibility requirements. These aspects must be evaluated by users along with the rest of the conceptual design.

Close collaboration within the UE team ensures a truly integrated and coherent design, with complementary relationships across all aspects of the offering. The entire design team should attend design walk-throughs as well as user evaluation sessions.

Development. The development phase establishes the detailed design, exemplified by high-fidelity prototypes to support user evaluations. Any necessary design changes are made, and the completed design is conveyed to the implementation team.

The key design activity is transformation of the conceptual design, validated through low-fidelity prototypes, into a concrete and fully detailed design, including high-fidelity prototypes. This level of the design is communicated to the implementation team. The goal of low-level design is a complete and unambiguous specification of all aspects that affect the user experience.

Although the user experience design role leads these activities, the entire multidisciplinary team participates in decisions about details that lead to a complete specification of the user experience across all aspects of the project. For example, these activities include transition planning, such as scenarios for upgrade and migration of current users, as well as adoption by new users. A detailed specification allows the implementation team to focus on implementation issues and ensures that issues affecting the user experience are made by those with the knowledge and skills to do so.

Throughout implementation, the design team assesses fidelity, identifies deviations, provides clarification, and directs corrections. During formal testing, the design team prioritizes problems and helps keep the project on target. All members of the design team participate, at least as observers, in user validation studies. Key members of the implementation team should also observe users' experiences with the offering.

Close collaboration within the UE team ensures a truly integrated and coherent design.

This phase concludes with the successful completion of all validation tests against both business goals and user criteria.

Deployment. The deployment phase verifies the implementation in the users' domain and addresses any problems resulting from user experience. In addition, an evaluation is made of the effectiveness of the UE process, and improvements are adopted.

The deployment phase includes all activities and deliverables required to successfully announce, deploy, and support the new offering. They include the early customer support programs that provide invaluable experience, as well as the availability of all marketing collateral. Briefing materials for marketing communications and advertising agencies should explain key user objectives, innovative features, and design elements.

Early support programs, including such arrangements as beta releases, should be employed to gain first-hand experience of the distribution and support processes, as well as initial customer reaction. Customer and user feedback procedures should be fully functional, so that the service and support organizations can be responsive to early input. Marketing should seek strong testimonials that can be included in their promotional activities.

In order to ensure a smooth deployment, an emergency response team may need to be prepared to identify quick solutions or workarounds. Mechanisms should be in place for gathering early user experience, and this feedback should be acted upon quickly, with prioritization of issues that need to be addressed immediately versus those that might be handled in a subsequent delivery.

Depending on the competitive landscape, a competitive benchmark assessment may be appropriate. The results from all post-development evaluations should be correlated with design and development evaluations and used to recalibrate processes and metrics as appropriate. The operational support structure should be validated through assessments of customer satisfaction with service incidents, and this information should also be correlated with design and development approaches to achieve improvement over time.

The design team will complete their contribution to the project at the end of this phase, so it is appropriate to do an assessment of the project and highlight achievements. The user experience leadership role should represent the design aspects in an overall project assessment. Naturally, any highlighting of achievements should be strongly influenced by actual user feedback.

Life cycle. The life-cycle phase manages the delivered offering to maximize its business potential. It also captures relevant user experience feedback that can be used as input for subsequent projects.

The life cycle phase is aligned with actual use of the final product. The total user experience is put to the ultimate test through orders, installations, and real use. The user will be the final arbiter, who may become a loyal and delighted fan at one extreme or a dissatisfied and frustrated enemy at the other extreme. The product may gain market share or make little impact. The user experience will most likely have much to do with the product's reception and its future.

A simple and accessible user feedback mechanism should be provided, in addition to gathering data from service and support, so that valuable information starts flowing from a growing community of users. This information is of great value and needs to be given detailed attention. Customer complaints may need immediate response to maintain customer satisfaction. All of this information should be used to increase market understanding, which will aid future development. Also, these users, who have devoted time to provide input about the product, may well be great test subjects for future development projects.

Although all design is complete at this phase and the team will have moved on to new projects, team members may well be called upon to review any significant user experience issues. Thus they may need to assist service and support where their specialized skills would be invaluable. This involvement benefits the customer and ensures that real-world experience is fed back into future designs.

After several months of actual use, service and support gathers all user experience incidents, performs a thorough analysis, and creates a report. This report summarizes the main design issues and brings forward proposals on how they should be addressed. Should there be a follow-on project, this report will be absolutely vital as input to the understanding-user phase.

A user satisfaction survey is performed during this phase. It is based on a statistically valid sample of the market and seeks to gain insight into how real users are interacting with the finished offering. Both quantitative and qualitative information is collected to compare to the original user targets, as well as those measures obtained through each of the user evaluations.

Activities. Activities consist of the work performed by a role during a phase to create and contribute to work products. UE describes all activities performed by a specific role for a specific work product. Also identified are any roles that assist in performing the activities and the phase in which the work is performed.

Each activity is closely correlated with a specific element of the targeted work product. Verbs, such as “create” and “specify,” indicate the specific type of work to be done in creating the related element. Accepted and proven methods for performing the activities are specified in many cases. An overview of these activities is shown in Table 1.

Measures and targets.

UE is about providing value to both shareholders and users. As described earlier in this paper, shareholder and user goals are explicitly modeled, their measures of success are determined, and target values are identified.

Goals are established and results are assessed by using various measures throughout the UE process. Measures identify aspects of an offering from both the business and user's perspectives. Targets are established for key measures early in the process, and progress is tracked against these targets throughout the process. Targets and measured values are recorded in the various work products produced in each phase.

A measure is defined in terms of the aspect to be evaluated, the calculation or technique used in performing the measurement, and the units in which results are recorded. Also included is the role performing the evaluation, the activity during which the evaluation is performed, and the work product in which results are recorded. Targets are specific values that represent desired results. Measures in one phase are sometimes used to establish targets for subsequent phases. For example, measures obtained during benchmark and competitive evaluations are typically used to set targets that ensure subsequent offerings surpass prior versions and competition.

An example of shareholder measures and targets is shown in the business model of Figure 4 earlier in this section. User-oriented measures are typically both qualitative and quantitative. For a software offering, as an example, user satisfaction is assessed by using a survey technique, allowing users to rate a number of factors such as capability, usability, performance, reliability, installability, maintainability, documentation, service, and overall satisfaction. Quantitative measures typically include task-oriented assessments such as time to perform a task, percentage of users who successfully complete a task, and error rates while performing a task.

Although these measures are common to many types of offerings, UE dictates that each project defines measures and targets appropriate to the business and to users' goals.

UE deployment at IBM

UE is being rapidly deployed in IBM via upgrades to existing development processes and tools and active evangelism across all development divisions. IBM uses the Integrated Product Development (IPD) process. This process consists of a series of development phases from initial product concept through design, implementation, deployment, and life cycle. Preceding IPD is a market management process that establishes the marketing and business requirements necessary to launch a development team for an offering. Gating each phase is a decision checkpoint, during which the results of a phase are assessed and a go or no-go decision is made. The IPD process is documented in great detail, and tools are provided to facilitate its use.

UE is being tightly integrated into the IPD process, with regard to roles, activities, work products, and measures. It is becoming an integral component of the checkpoint process used to create IBM offerings. In addition, existing tools such as the UCD workbench are being uplifted to integrate UE methods and tools. For example, the workbench already supports tools for building and conducting user surveys, such as WebSurveyor**, and new tools are being added to plan UE projects, conduct heuristic evaluations, and perform user task analysis. The use of standard workbench tools not only improves productivity; it ensures that standard measures are captured such that trends and comparisons can be assessed between projects and over time within a project. The data from all projects using UE will be maintained in a database to facilitate monitoring and tracking of progress over time.

The definition and deployment of UE and its key methods is being pursued both in IBM and outside. Within IBM, presentations are being given to practitioners at each of the key development locations, and pilot projects have been undertaken. Outside of IBM, the methodology has been presented through such means as the annual Make It Easy conference,¹⁴ tutorials and booths at ACM SIGCHI conferences,¹⁵ and articles in IEEE Software¹⁶ and the International Journal of Human-Computer Interface.¹⁷

Many of the elements of UE have already been used in practice in various IBM development projects. The following section highlights some of these instances.

UE case study

The use of elements of UE during actual development projects has provided opportunities to preview the impacts and benefits of the process. This case study consists of anecdotes that come from an eight-month Web project to redesign the PartnerWorld* business partner Web experience on the IBM Web site.

IBM conducts a quarterly survey of business partners to assess their satisfaction in working with IBM. When their satisfaction did not meet established targets, IBM responded by assembling the proper organizations and executives to determine which aspects were the most problematic and critical to address. Improving “Web navigation,” a commonly voiced desire of business partners, emerged as a high-priority initiative.

The organization responsible for the business partner Web experience engaged an internal IBM team to apply elements of the UE method to the problem of business partner Web experience. The UE team was engaged in March 2002, and final user testing of the design was completed in early November 2002.

Impact area examples. At the time of this writing, the user-validated design was not yet operational. However, the project has already realized a positive impact on the business in the following areas, each of which is then further described through the use of examples:

• Precise (architectural) thinking—done by both the business teams and the content creation team (the creation of valid, clear entities and relationships)
• Organizational processes and teaming—long-term repetition of quality solutions and ability of diverse subject matter experts to work more effectively together
• Business efficiency—time to market
• Business impacts—reduction of operating costs and increases in revenue
• Content creation and integrity—capturing and creating content once to make it accessible in many ways, and encapsulating content
• Corporate assets—creation of shareable, long-term intellectual assets that capture an essential body of knowledge about IBM customers and users

Precise (architectural) thinking. The use of UE elements has resulted in more “architectural thinking” by the business and content owners. In a company as large as IBM, one way to get things done quickly is to work in isolation, focusing on a single item or aspect of the total user experience (a business program, a service, a new offering, etc.) without reference to the bigger picture. One consequence of this approach is that the relationship of that single item to others is not always clearly defined. Ambiguous or “fuzzy” business concepts usually cause problems when they are ultimately manifest on the Web. The Web experience exacerbates the problem of business complexity. Ideas and offerings must be very clear and self-explanatory. There are no longer “blue suits” around to spend time explaining things to IBM's customers. IBM's offerings, services, and business concepts need to be clear and crisp, fundamentally defined by the relationships between items. The application of UE required the UE team to ask the business for definitions of items as we worked to understand the relationships. As a result, the business owners were “immersed” in architectural thinking—having to think precisely and in terms of clear distinctions, which helped everyone to see the impact of what they define as the total user experience.

Organizational processes and team building. The application of UE is also driving change in the efficiency of creating an effective and satisfying Web experience. Historically, large-scale Web sites have relied to a great extent on individuals with ambiguous qualifications to drive the design of the total user experience. Thus, design efforts have been using a process of “design by a committee of opinions.” Basically, there has been a lack of rigorous process brought to bear on the creation of Web experiences. The design-by-committee method of arriving at a design can be very tedious, rambling, and ultimately ineffective, even though a team can stumble upon a reasonably satisfying experience. This design approach cannot be sustained. Eventually the customer's goals may change, and the Web experience team may not be aware of it. The application of UE caused all of the former committee members to begin to understand that the user drives design decisions, based upon a true understanding of users' goals, tasks, objects, and preferences by the team.

IBM is characterized by many highly driven, articulate, intelligent individuals, and with that comes much diversity of subject matter expertise and perspective. As mentioned above, design efforts that use a process of committee decision-making are at the mercy of these individuals' ability to work effectively together as a team. If individuals involved are not able to transcend the differences of subject matter expertise and personal perspective, it is extremely difficult to move design solutions forward. By contrast, UE provides a user-centered framework to which subject matter expertise can be applied without pulling the design away from the core set of user-centered concepts. In other words, UE provides the topics, issues, process, principles, measurements, and so forth that are to be addressed systematically with a common set of perspectives and understandings. When team members share an understanding of influences, components, and repeatable processes, they are able to participate more effectively together to create a satisfying design. At its core, UE is also driven by measures, which help the team members to focus on the targets that have been defined.

Business efficiency. UE impacts business efficiency by bringing new, long-term, more satisfying designs to the marketplace (i.e., released live to the Web) in a timely manner. It might appear that the PartnerWorld redesign effort was the antithesis of “time to market” in that it took eight months to do a major user experience architectural redesign, and in “Web time” that is extensive. The reality is that PartnerWorld had a serious set of Web experience problems that only a complete change to the fundamental “user perceivable” architecture would fix. If UE had not been applied, the kind of change that is being proposed might never have happened. The business was in effect waiting for the technical infrastructure to change before attempting to create a singular user experience for our partners. Instead, UE created a solution that is rooted in what the user perceives as a single unified experience, and a critical solution to a well-established problem was created two years before the technical infrastructure would be ready. The long-term impact to the business should be substantial because the redesign should remain stable for many years as the business improves its offerings, changes the technical infrastructure to meet the needs of partners, and, over time, changes the nature of what is offered.

Business impacts. UE called into question any element that was not rooted in users' needs. It was clearly understood that the business had goals as well as users, and work was done to articulate the user-business goal relationships. The UE team was the only team close to users and provided the business with additional service by asking for a closer look at elements that might not be needed. For example, one result of this activity was the potential to accelerate the sunset of much content that was no longer needed and that was costing the company millions of dollars to host. Eventually the hosting costs will be reduced to zero as the content is incorporated into the Web environment. The business impact in this case is a reduction of operating costs.

Content creation and integrity. It is difficult to say where user experience design stops and content starts. The UE effort resulted in the creation of an architecture that had reusable content components that were presented in various contexts. The actual authors of the content now have a holistic context in which to write the actual text for each element. Without a user experience architecture, content creation can result in more but less effective content. The integrity of the content also increases when it is clear that the content author is providing the appropriate value propositions and essential nature of the content to ensure that users can use the content in a Web experience.

Corporate assets. The work products that are created as a result of UE have long-term application and should be thought of as corporate assets. The capturing and documenting of users' contexts, goals, tasks, and so on will provide information for many projects that take place over time to improve the total user experience for our partners. Any form of documentation of critical knowledge can be thought of as a corporate asset. The modeling work products that result from UE may someday support automatic code generation; thus, they play a substantial role in the future efficiencies of creating user experiences.

Anecdotes. The following were selected from a set of at least 15–20 anecdotes that tell the story of the application of UE to this real-world project. They were chosen to illustrate some of the more complex topics with high impact on the business if not addressed appropriately. Although these anecdotes are primarily Web-oriented, key elements of UE, such as the OVID method, have been successfully applied in development for other environments, including hardware devices.¹²

Anecdote 1. Conceptual structure of a Web experience. This structure is reflected in the top-level navigation system. Business owners first felt the impact of UE principles when this topic was introduced. Determining the heuristics for structuring the top-level categorization of a Web experience that enables site navigation is an interesting ongoing subject of many Web design efforts. In the case of the PartnerWorld Web experience, there was a legacy of having the Web structure reflect the PartnerWorld business partner program—what IBM offers business partners. It seems reasonable at first to assume this 1:1 correspondence between a structure of program content and a Web experience that delivers the benefits defined in the program to partners. Through a very specific UE activity—goal modeling—we quickly discovered a more useful basis for structuring the Web experience.

The first user activity performed on the project was to gather user requirements, including user goals. We gathered user goal statements from 88 business partners in two different regions (United States and France). Through goal analysis and modeling, we created a top-level navigation structure that reflects users' goals. We tested the goal-based site structure in our first prototype (performing a cognitive walkthrough), and the structure stayed very stable throughout the subsequent two prototypes. Getting the structure correct early enabled us to use only three prototypes rather than the four prototypes we had planned.

Net outcome. Goal modeling was instrumental in determining the structure of the site and focusing the business owners on the Web design to reflect the user's goals, not just what IBM was offering. It was also very helpful to do a “mini” competitive assessment to determine how benefits are portrayed by our competitors.

User testing outcome. The benefits section is rating very high on satisfaction, as is the overall site. If we did not do this work properly, the site might still reflect benefits categories rather than focusing on user priorities and conceptual models.

Business impacts. More partners should be able to find out about benefits and join the PartnerWorld program. In theory, more partners may result in more revenue to IBM. (The other part of the equation, however, is: how are we helping partners increase their revenue, and what role has the Web in that?)

Architectural thinking impact. The part of the UE stakeholder team that owns the creation of the business partner program experienced the UE team's architectural thinking and thus will carry the approach forward as they perform subsequent work.

Anecdote 2. Determining what to do when a business requirement has a conceptual model embedded in it. One of the UE team's most challenging issues came about halfway through the project as we began to concentrate on one of the sections of the site design. The team presented conceptual designs to the technical content owners. The design was derived from task analysis (a card sort activity) and from results of the first iterative test (a cognitive walk-through). The UE team felt that there was an appropriate level of understanding by users behind the proposed design solution. In the review it became immediately apparent that the UE team and the business team were very far apart on thoughts about the design of the section. We realized that we had a substantial problem because we were fairly far into the project and not at all close to an agreement that the business requirements were being fulfilled by the proposed design.

The UE team proceeded to determine where the disconnection was between the proposed design and the business requirements. We thought about all of the following possibilities:

• Was it a question of “design preference,” that is, the business team was trying to design the experience and disagreed with the UE team and user-testing results?
• Was it that we did not have a true and valid set of business requirements?
• Was it an authority problem, that is, who really owned the design of the user experience—the UE team and its client, or the content owners of the section?

We quickly realized that the apparent problem seemed to be that the business requirement had a conceptual model embedded within it! The UE team attempted to work with the business team to restate the business requirement in a “solution-neutral” way, which meant identifying the business measurement that the business or content owner was really trying to achieve. The UE team arranged a meeting with the owner and our client executives to educate everyone on the role of the conceptual model in a user experience. This meeting allowed the issue to be addressed with a clear understanding of why it is so important to have the business requirement expressed in a solution-neutral way. During the meeting, we discussed a number of topics, and as we worked through the issues, we suddenly realized that the root of the disconnection was actually the goal of the section. The business or content owner wanted the section to educate users on the technical services, whereas the UE team needed to ensure users could use the technical resources they are entitled to. It was a revealing discussion, and in the end both goals were met: satisfying both the users' needs and the business's needs. It was a critical win for the user experience. If we had not been able to resolve this issue with users' needs at the forefront, it could have adversely impacted the business. Technical business partners are the majority of business partner users, and their user experience needs are very specific. If the UE team had missed the mark, it would have had a very detrimental effect on the overall user satisfaction with the proposed design.

Net outcome. Placing the issue in the context of goals enabled the UE team to produce a very useful section design that appeared to be a valid design to test. The business's goals were met and the user's goals were met.

User testing outcome. The technical section tested very well, including a welcome guide that was created to achieve the business goal of educating new users.

Business impacts. Improved satisfaction with PartnerWorld by the largest group of users should result in increased loyalty to the IBM brand, as well as revenue increases, by enabling this technical group of users to obtain the information they need to create solutions that they are selling. Goal modeling was instrumental in determining the structure of the Web site.

Organizational processes and teaming. The ability of the UE team and the business team to work together to understand the disconnection resulted in an extremely satisfying and useful user experience. The UE team and the business team worked hard to understand the topic from their individual perspectives and now have a frame of reference against which to understand all future potential issues.

Lesson learned. From a UE perspective, the UE team plans to assess whether additional goal modeling could have prevented this type of disconnection. It is an extremely interesting topic and one that will help shape the evolution of UE such that the UE methods can uncover topics of potential harm before the teams encounter them.

Anecdote 3. What “a single site” meant to different parts of the company and to users. The most challenging topic that the UE team faced concerned the new vision of the unified user experience that was proposed early in the project. Because the team did not report to the same organization that owns the PartnerWorld user experience, we had the defensible audacity to propose a new user experience that was founded on users' goals, tasks, and conceptual models. The UE team had user study data to back up the proposed design and bore primary responsibility for presenting the vision, recognizing it went against one of the set of requirements, which was to maintain a current separation of Web experiences based upon the type of business partner.

As the UE team presented the vision to the business, it became apparent that there was a major disconnection over the definition of a site. To the business it meant either: (a) technically one site—all created with one technical infrastructure, or (b) owned by a single organization—a single set of resources managed under the same leadership. Each of these definitions created extreme emotional responses to the proposed design, when in effect what the team was proposing was a single site from a user perspective. Users should not have to know or care that the site is created through a variety of content management systems and is hosted by a variety of services. Likewise, how the business chose to organize itself to deliver the design is not a user concern either. With the clear focus on what the user perceives and needs, the UE team was able to fight the fight that mattered most, with confidence in the ability to position the topic from the user's perspective, which is also fundamentally a business perspective. Without the ability to identify and do what is right for customers, IBM will not achieve the business gains being sought.

Net outcome. The business will provide a user experience that business partners desire, two years before they thought it was possible!

User testing outcome. The overall user experience tested very well, but one component that users desire requires a technical solution that is not currently possible. Until it is possible, this specific issue will continue to be a user experience problem.

Business impacts. Significant potential positive impact on business partner revenue and satisfaction will occur as business partners see that IBM is listening to them, and business partners will be able to more efficiently create their solutions.

Architectural thinking. It is critical to point out the “ripple effect” of presenting a critical topic from an architectural perspective. The UE team's point of persuasion was successful because the solution was presented from the users' context. Architectural thinking establishes understanding and clearly expresses context so that ambiguities are avoided or minimized. An application of UE with architectural thinking provided the team with the elements to win the argument.

Lesson learned. Believe in the audacity of a proposal when there is confidence among members of an experienced UE team in how the solution was created and continued testing proves it is desirable to users. A UE team must continually portray designs in the context of what is in the best interest of the business, but designed from the user's perspective.

Early quantifiable results. While the practice of UE as a complete process is evolving within IBM, many of the elements of UE have been in use and have delivered measurable results. Although much of the data are proprietary, two specific cases illustrate the kind of results that might be achieved.

Elements of UE were used in a redesign of a portion of the IBM Web site, the purpose of which was the sale of certain types of servers. The redesign efforts yielded these significant improvements in early 2002:

• A 15-point increase in user satisfaction with the ordering process
• A 10-point increase in ease of use
• A 10-point increase in goal achievement

In addition, the user research identified a need for a new function that led almost immediately to 1.3 million dollars in revenue.

In the latter half of 2002, another section of the IBM Web site was redesigned with the following results:

• A 10-point increase in goal achievement
• A 12-point increase in users easily achieving their goal
• A 50 percent increase in revenue, with record-setting revenue the first month of deployment
• A decrease in annual maintenance of nearly $200000

Case study summary. There are more anecdotes of substance to share, and the UE team looks forward to measuring the results and long-term impacts of the PartnerWorld project. The team was cautioned at the beginning of the project that the wide variety of stakeholders on the project could be a challenging proposition as the project progressed. The team found that the stakeholders seemed to understand the UE process was resulting in a design that not only would meet users' needs but also would solve some of their long-standing problems. Some stakeholders now have substantial changes to make, but they all see the value in doing the work and believe that the end result will make a big difference in overall business partner satisfaction.

The UE team commended the stakeholders for their willingness to see issues from the user-centered perspective, and for participating in the UE project in a fully engaged and positive manner. The result of everyone working together should mark an important turning point in business partner satisfaction in doing business with IBM.

Conclusion

Engineering is the application of rigorous principles and methodical approaches for practical purposes. UE has been developed from this heritage as a discipline for designing user experiences that match users' expectations and that return expected value to stakeholders.

The success of an offering is determined by how well it is received by its intended audience, and success typically results from a systematic design process that involves the intended users in creating the solution. It also involves satisfying the total user experience, which consists of all aspects of a product or service as perceived by users—from a user's initial awareness of it, through additional discovery, ordering, fulfillment, installation, initial use, day-to-day use, service, support, upgrades, and end of life.

UE addresses the total user experience, incorporating many well-established practices such as User-Centered Design. It utilizes a multidisciplinary design team to create this experience in partnership with its intended users. This team fulfills a set of well-defined roles and performs defined activities during specific phases of the design process. Activities result in work products that capture and communicate all elements of the design. Key elements of the process are thorough research and specification of measures and targets, following a rigorous process, using proven methods, facilitated by tools, with measures and targets that reflect the values sought by stakeholders as well as users. Practitioners of UE should achieve a significant improvement in their ability to produce offerings that are not only usable, but that achieve and potentially surpass users' and stakeholders' expectations.

^*Trademark or registered trademark of International Business Machines Corporation.
^**Trademark or registered trademark of the Object Management Group or WebSurveyor Corporation.

Cited references