IBM business transformation enabled by service-oriented architecture

The business value of new technologies must be demonstrated to ensure their acceptance by business owners and information technology (IT) practitioners. IBM views service-oriented architecture (SOA) as a key enabler of business transformation, one that can lead to cost-effective IT implementations which can be incrementally deployed.

The on demand transformation of IBM is business driven, guiding IT in the creation of new business capabilities. New techniques to create or assemble solutions are continually being defined, and business information is made accessible as easy-to-use services, thus allowing people to easily find the right tools or information.

Reuse also plays a major role in strategies to maximize the return on investment (ROI), to increase resource sharing, and to minimize the need to build new functions.

SOA is positioned as one of the foundation layers for the IBM internal transformation strategy. The following is a working definition of SOA¹:

The second part of this definition emphasizes the need for a loose coupling between applications, which helps to ensure that a request to use a business function is not tied to a specific technical implementation of that function. This is critical in the IBM heterogeneous environment, which includes custom applications, third-party products, and integrated business-partner systems.

This paper discusses the use of SOA as a key element supporting business transformation at IBM. Following this introduction, the IBM internal SOA directions and strategies are discussed. A diagrammatic depiction of this strategy illustrates how SOA contributes to the implementation consistency, decoupling, and interoperability of component applications.

The section that follows covers implementation. The IBM internal IT environment is first portrayed as a medium into which SOA is incorporated. A combination of top-down and bottom-up approaches to implementing SOA are next described. From a top-down perspective, SOA cannot stand alone; it must be infused into the IBM enterprise architecture. Corporate SOA governance is discussed as an integrated facet of IBM's overall enterprise architecture governance which is an essential tool to manage the complexity of creating an effective collaboration environment across multiple business units with a diverse set of business goals. The bottom-up approach is equally important; thus, the activities to promote SOA within the IBM internal development organization are also discussed. This organization is known as IBM Global Account (IGA). To understand the organizational implications of using SOA to support transformation of IBM, a brief discussion of the various internal organizations and their ability to support enterprise-wide services is provided. Next, the internal SOA Center of Excellence (CoE) is represented as a virtual organization that consists of several internal organizations devoted to the advancement of SOA.

In the section that follows, several IBM internal SOA case studies are summarized. They deal with support for regulatory compliance, cost reduction through the elimination of IT redundancy, the use of centralized and trusted data, and the outsourcing of business functions.

Then, a set of high-level business and IT lessons learned from such SOA projects are briefly described. The paper closes with some final comments.

In this section the strategic steps taken to use SOA in support of IBM business transformation are discussed and illustrated. The IBM SOA strategy is the result of a collaborative approach among corporate, business area, and key development organizations.

Business drivers, such as customer satisfaction, changing market conditions, competitive threats, government regulations, and IT cost management, increase the need for improved IT responsiveness, speed, and flexibility. Other overarching business goals, such as creating a “business on demand” and shifting resources rapidly to growth opportunities, are also important elements that influence the IBM internal SOA strategy. The following represent the strategic steps taken by IBM within its internal business transformation activities to satisfy these business drivers:

1. Business and process driven—The IBM Chief Information Officer (CIO) office views IT responsiveness and flexibility as a high-priority item. To accomplish this, it has to ensure that IT is process driven and responds quickly to changing business conditions. Business process management (BPM) techniques are used by several internal projects, which import process models directly into workflow modeling tools. This helps to ensure that a business analyst's intent is correctly realized when workflows are created. New customer requirements can be quickly added to process models and input into workflow models. For improved flexibility, externalized business rules described in process models allow parameters to be simply changed without manually changing the code.

2. Promote information as a service—A service orientation for information access has been adopted within IBM. The IBM Enterprise Business Information (EBI) Center of Excellence has defined common sources of enterprise data to be shared by the entire corporation. The mission of EBI to manage business information assets includes using SOA for a common approach to access and distribute trusted information, and to establish an enterprise vocabulary for business data used by SOA solutions. Information-as-a-service implementations have often required some level of enterprise logic with business rules and policies as a layer wrapped around data sources. Such systems provide services with added business value when compared with simple, raw data. As an example, a unified common view of customer information can be provided by a Web service that makes aggregated customer data accessible; this data originates from multiple customer “touch points” in several internal business areas. The common view of client information may include background information, previous purchases, and support history and leads to improved customer satisfaction.

3. Foster reuse—SOA enables reuse through the “build once and leverage” approach (e.g., by using already existing functions instead of building new ones, thus eliminating redundant development and support costs). As the inventory of business services in the IBM internal SOA asset repository increases, more IT functions can be created as composite business services, which involve the assembly of existing services. As market conditions change, solutions can be assembled to create new business capabilities on demand.

   Reuse of common IT functions at build time (development) and runtime (service access) becomes part of the IBM community culture. Although some progress is visible, further changes in the IBM internal IT culture are needed for project managers and IT practitioners to always consider reuse (reusing existing functions and building functions with improved reusability). This occurs from a top-down perspective through the enterprise architecture, as well as a bottom-up perspective, especially in IGA.

   As an example of SOA-enabled reuse, with the availability of a common enterprise tax service, global tax calculations are now performed consistently. Compliance with government regulations is supported by Export Validation Service, which is described in the section “SOA showcase highlights.”

4. Leverage and modernize legacy applications—IBM legacy systems are often difficult to change, but have lingered longer than anticipated, since they continue to provide returns on their initial investments. Because a flexible application environment is critical for developing on demand solutions, several legacy applications have been upgraded through the use of service encapsulation techniques. Thus, upgrading existing legacy applications instead of replacing them allows limited IT funds to be redirected toward new business opportunities. SOA patterns have been developed to simplify reuse of legacy systems (e.g., message and protocol transformations) and to encourage incremental migrations to current technologies. This helps minimize the impact on the applications that consumes these legacy applications by shielding them from the migration activities. For example, COATS (Customer Order Analysis Tracking System), the order preprocessing function for hardware manufacturing, was upgraded with new SOA features on top of the existing legacy PL/1 code, which enables its incremental migration to a state-of-the-art replacement.

5. Incorporate third party products—A variety of sources for information systems and applications are used to maintain a balanced portfolio of best-of-breed IT capabilities. Strategic application architectures account for applications that are (1) completely customized from code, (2) customized, but built using IBM products, (3) adaptations of legacy systems, and (4) created by using third-party, off-the-shelf products. Third-party products can be used in either a preferred “out-of-the box” fashion (as much as the IBM internal business processes can tolerate), or through varying levels of customization to adapt to IBM unique business requirements. Considerations to use third-party products for IBM internal functions now include evaluations of their SOA capabilities, making use of their service interfaces (and for these products to use services provided by other components). As an example, the configuration product from Selectica, Inc. now provides configuration services to the internally deployed commerce engines that allow IBM customers to order products and services.

6. Enhance B2B (business-to-business) transactions—SOA-enabled improvements in interoperability with IBM partners are possible by creating an extended enterprise, in which IBM and its business partners use each other's services for conducting B2B transactions. SOVA (Single Order Validation Application) is a Web service that is externally available to IBM business partners validating hardware product configuration (files) attached to order submissions. It has increased the number of configuration validation checks before order submission by allowing a partner's enterprise resource planning (ERP) systems to automatically verify configurations. Due to the improved quality of submitted orders, order management costs are contained.

7. Simplify outsourcing—Outsourcing allows IBM to shift resources to core business growth opportunities by using vendors for cost-effective management of IT functions that have become commoditized in the industry. If an internal application is already decoupled through SOA and open standards, while still located within the IBM infrastructure, then it can be more easily outsourced to a vendor outside of IBM firewalls. IBM systems and Technology Group recently outsourced a semiconductor testing facility to Amkor Technology, Inc. The IBM internal system was down for just four hours during the cutover from IBM to Amkor. In another example, IBM employee authentication at the Web sites of external vendors to whom previous internal functions were outsourced is now possible with the IBM Intranet Password External (IIPX) Web service.

8. Automate sense and respond—The coordination of key performance indicators is a critical IBM activity that provides insights into operational processes. Event management capabilities have been deployed to the IBM shared infrastructure, allowing the implementation of business dashboards that collect structured business events emitted from a variety of IT sources, such as workflows and Web services. Business managers then have a ready source of operational information to mitigate production errors and to ensure that products and services are delivered on time to their customers.

9. Manage incremental deployments—Incremental deployments are preferable to the costly and disruptive single deployments of a large IT function. In addition, they allow incremental business value to be realized, rather than waiting until the end of a long development cycle. Several IBM projects have created strategic interface layers that hide their transformations from the components they serve, while allowing new functions to be introduced gradually.

10. Enable a new integration platform—Many technologies and approaches have come and gone within IBM, along with their promises of improving enterprise-wide, end-to-end process integration. Some technologies, such as CORBA (Common Object Request Broker Architecture), were excellent concepts that were not able to gain sufficient traction, either due to perceptions of high complexity or lack of support products. SOA integration capabilities are tightly woven into the IBM enterprise architecture blueprint, which helps provide a cohesive view two to three years into the future of how the systems will interact to enable business capabilities in an on demand fashion.

In summary, major transformation efforts within the company are directed toward making it simpler for customers to do business with IBM and to increase its posture as a globally integrated enterprise. SOA, as both an integration platform and a way of approaching the creation of new IT solutions, plays a major role in IBM business transformation.

Figure 1 represents a view of an SOA-enabled IBM infrastructure that includes a mixture of IBM custom and vendor applications.

Figure 1

Service and workflow components
A number of large-scale applications, such as Application 1 in Figure 1, can be decomposed into smaller units, designed as service components (represented in the diagram as the boxes labeled 1, 2, and 3). These service components represent modular functions that can be designed, developed, tested, and deployed independently for improved flexibility and speed to market—in comparison to overly large applications with longer cycles for development, change, and testing. Workflow-oriented code in other applications, such as Application 2 in Figure 1, can be decomposed into service components and migrated to workflow-engine-based platforms. Although the workflow engines are depicted for illustration purposes as being external to the service components, they are generally implemented as an integral part of a service component, especially because they can be accessed through service interfaces. Corporate guidelines instruct workflow development projects to handle business process models created by business analysts as direct inputs into their modeling tools.

Enterprise Service Bus
The Enterprise Service Bus (ESB), which plays a prominent role in the IBM internal SOA vision, allows for the virtualization of services though a middleware intermediary that performs transportation functions (e.g., secure and assured delivery), mediation functions (e.g., service routing, protocol and data transformations), and rudimentary forms of event management. ESBs have been deployed as a middle layer between service requestors and service providers, and add value by relieving providers from the burden of supporting various functions. The ESB strategy includes the creation of domain ESBs, which support the service consumers and providers within each of the IBM major business areas, and an enterprise-wide ESB backbone to interconnect the domains.

Common messaging specification
Represented in Figure 1 is a common messaging format, EIMS (Enterprise Integration Messaging Specification), which has been successfully deployed by the IBM Enterprise Business Information team to support a number of internal applications. It defines and creates reusable message constructs and data vocabularies. These Extensible Markup Language (XML) message constructs provide a common data structure and common data semantics for IBM internal messages. EIMS extends the Business Object Document specification of the Open Application Group,² and has been documented in the IBM internal strategic SOA recommendations for application-to-application communications.

Third party product integration
The major vendor applications in Figure 1 (e.g., SAP AG, CRM Siebel, and I2, Inc.) of which several are from IBM strategic alliance partners, are being incorporated into the IBM SOA infrastructure. In general, for all vendor packages that are being considered for adoption, standardized third-party product review processes include questions about their SOA capabilities. Integration considerations include WS-I-compliant (Web Services Interoperability-compliant) Web service interfaces, portal interoperability (e.g., Web Services for Remote Portals for remote portlet invocation through services), process-modeling integration techniques, and workflow engines. Vendor business objects can sometimes be made accessible directly with Web service interfaces included in their products. However, it has often been necessary to define strategic “front-end” service components with IBM WebSphere* products that reside in front of these third-party objects—to aggregate the operations of multiple business objects when producing a higher level of transaction or information than is normally available directly from the vendor's product. Because the service requestors only have visibility to the service layers and are generally oblivious to the mechanisms behind the layers, the middleware intermediaries and strategic service interfaces hide wholesale or incremental changes to the ERP packages. Service interaction is defined for both directions—vendor applications can also use services that reside externally, in addition to providing services. However, if multiple physical instances of a specific vendor application exist and need to interact with each other directly, they will be allowed to use their most efficient communication protocol, even if it is proprietary and does not have a service orientation.

Internal SOA-related standards
In a business-as-usual fashion, projects using SOA techniques are still subject to IT standards and corporate instructions for security, privacy, business interactions, technology usage, project management, processes, data, and applications. For an additional SOA focus, internally published SOA design criteria promote an effective separation of concerns and isolation to enable simple business-partner integration and potential outsourcing opportunities. Internal SOA-related standards and formal guidance documents are based upon industry standards and specifications to enable service interoperability with business partners external to the boundaries of an enterprise, in addition to promoting a consistency of application interactions within IBM.

In summary, the IBM internal SOA strategy is a critical ingredient in its on demand business transformation efforts. SOA is thus a key integration platform for fusing business, information, and application architectures.

In this section, the implementation of the IBM internal SOA strategy is described. The topics covered include: the internal IT environment, the IBM enterprise architecture, managing project development, the internal organizational structure in support of SOA, and the SOA Center of Excellence activities.

The IBM IT landscape is a heterogeneous mixture of legacy information sources and applications, new custom functions incorporating the latest technologies, and third-party products with varying degrees of customization. This juxtaposition of various application styles requires a commitment to integration, which is evident in many of the current IBM internal architecture definitions and strategies, both at the enterprise and business unit levels.

Several common enterprise- and business-unit-specific IT functions have already been deployed and reused on a large scale, but some did not use a services interaction framework. Others, however, have been implemented with SOA and are recognized as enterprise or business unit service components (applications with service interfaces) in the IBM enterprise architecture.

Application and information integration have played an important role in IBM internal IT architecture over the past decade. Although this integration has included user front ends, middleware, and back-end systems, implementations were often not technically consistent or easily interoperable. Unique interface requirements for a mixture of legacy, custom, and off-the-shelf applications have at times dictated unique integration solutions, which can result in longer development schedules and higher costs.

A shared physical infrastructure has been deployed for commonly used IT platforms, such as application servers, e-mail servers, databases, Web portals, messaging queues, and shared file systems. This shared infrastructure is distributed over several global locations, and supports standard profile capabilities in clustered environments (to maximize economies of scale on shared resources), in addition to unique software and server installations for projects with requirements that do not fit with the standard profile.

The IBM enterprise architecture is designed to ensure effective linkages between enterprise business and IT deliverables. It is a means to integrate business strategy, process, data, applications, and infrastructure. Enterprise architecture governance attempts to unify design approaches with a set of published principles, architecture criteria, standards, and guidelines. SOA is directly integrated into the process of managing architectures within IBM. Because the IBM enterprise architecture provides a top-down view of IT that supports the alignment of IT with business objectives, incorporating SOA with it also allows SOA to share this top-down viewpoint and business alignment. Internal Web-service standards, as well as ESB and service component design criteria, are documented as part of the IBM enterprise architecture governance.

The end-to-end integration of process, data, and application architecture is reflected in the IBM enterprise architecture blueprint, which also provides a view of IBM internal architecture two to three years into the future. The IT focus of the blueprint that is defining key information systems and their relationships is complemented by the illustration of key business functions. Lead architects from each of the business areas document their architectures in the blueprint and provide road maps that link existing architecture implementations to projected models.

Figure 2 illustrates a high-level view of the enterprise architecture blueprint. Other diagrams are available that depict details for the major IBM business areas (e.g., Software Group, Systems and Technology Group, Global Business Services, Global Financing, Client Process Transformation, Integrated Supply Chain). Note that all these diagrams are Web enabled and interactive on the IBM intranet, in order to provide a variable level of detail.

Figure 2

The service components are depicted by the light brown boxes with a Unified Modeling Language (UML)** component icon located in the upper lefthand corner. Selecting each component provides information about its business function, business owners, data and process categorizations, the systems that use it, the services it provides, operations per service, and other details. Clicking on the arrows from the service consumers to these service components reveals which service is being invoked, and which specific operations are of interest to the consumer.

Figure 2 contains a mixture of strategic and tactical information systems and data sources, some of which have already been deployed, whereas others represent projected capabilities two to three years into the future. These systems and data sources have been defined as critical ingredients necessary to support the IBM internal IT transformation. Because the transformation is business-driven, these elements are positioned within the major process areas (yellow boxes in the diagram), such as Market Solutions, Products and Services, Manage Client Relationships, Manage Post Sales Technical Support, and Fulfill Solutions, Products and Services. These major process categories are further decomposed into several subcategories. The IBM enterprise architecture contains mappings of the information systems and data sources to these subcategories, which provide visibility into potential redundancies of capabilities (i.e., more than one system or database supporting the same process area).

All boxes with double-line borders, regardless of color shading, are information systems (applications). They are implemented by a variety of means, some custom built, others leveraging third-party software packages such as Siebel CRM from Oracle Corp. or mySAP ERP from SAP AG. The database entities in the diagram are data sources required by the various information systems. Users can select each information system or data source to obtain information about its business function, business owners, data and process categorizations, dependencies on other systems or data sources, and so on. The orange boxes at the left and right sides of the diagram represent parties that interact with the information systems. They can be employees, business partners (people or their systems for sales, fulfillment, or procurement), or IBM systems in customer locations. The arrows indicate information flows of data, which can occur between information systems, between information systems and data sources, or between data sources. Users can select these information flows to reveal a description, the type of data, and required data sources.

SOA governance has been deeply embedded into the IBM internal enterprise architecture governance from the inception of a corporate focus on Web Services and SOA. The IBM strong internal enterprise architecture model served as a vehicle to incorporate key SOA governance policies into project architectures. Internal service standards and SOA design criteria were published and enforced with existing enterprise architecture governance practices. Like SOA in general, SOA governance was not a stand-alone concept that needed to be addressed separately from normal project management activities. The CIO office recognized the need to minimize impediments to successful project deployments, which led to leveraging existing internal enterprise architecture governance mechanisms to drive key SOA directives—rather than defining a completely new and independent governance structure.

CIO funding guidance for the IBM business areas was established in the third quarter of 2006 to promote more SOA-related projects in 2007. In 2007, a minimum of 2 percent of the development spending of each business area must be invested in SOA-related capabilities. Since all of the areas have met or exceeded this target, the minimum investment criterion was raised to 10 percent for 2008. Supplementary specifications were provided to describe the allowed SOA activities, such as the development and consumption of services, ESBs, sense-and-respond monitoring functions, and automated workflows.

Several factors were identified that support the need for internal SOA governance and have a primary goal to maximize the ability of SOA as a key enabler of the IBM internal business transformation. These motives can be summarized as follows: First, coordinate the integration of SOA into the enterprise by defining a set of enterprise policies and agreements for service ownership, funding, charging, and usage mandates and publishing SOA compliance criteria to promote a consistent SOA infusion into information and application designs. Then, simplify the IT environment to easily create composite business services assembled from preexisting common services by:

• enhancing the process to identify and fund new common SOA assets,
• identifying services that support business capability needs,
• eliminating service redundancy and promoting service reuse,
• integrating data from multiple sources using a common business data vocabulary,
• ensuring that common service assets are integrated into the IBM enterprise architecture,
• enforcing the use of accepted enterprise and business-unit service components, and
• cultivating the ability to “sunset” costly legacy applications and incrementally migrate to a service base.

Internal IBM enterprise architecture governance is represented by the collection of four major governance domains: data, application, business process, and infrastructure. The IBM enterprise architecture target blueprint represents a view of internal architectures with a two-to-three year outlook into the future, and is one of the primary mechanisms to publish SOA-related governance. The blueprint contains internal SOA principles and design criteria to guide project architectures. Its depiction of enterprise and business-unit service components defines the commonly shared services that assist with the quest to maximize service reuse. The blueprint description of which applications share which common services helps to interlock service responsibilities in a federated manner throughout the enterprise. In addition, the blueprint illustration of key processes associated with all of its information systems (including service components) supports the need to ensure that service definitions are process driven.

The IGA team, which is part of IBM Global Services, has established a systematic reuse process involving a community of participating practitioners dedicated to identifying reusable project assets. Members of this community create SOA scorecards for their development projects that illustrate the degree to which a project is using SOA approaches (e.g., services, workflow, and ESBs). Also, an SOA asset repository based on the Reusable Asset Specification has been deployed and migrated to the recently released IBM Rational Asset Manager. It encourages reuse by providing visibility of internal SOA-related assets, such as process models, workflow models, service interfaces, data models, and externalized business rules. It has been integrated with the enterprise architecture blueprint to allow a combined view of the blueprint's future (target) service components along with in-progress and deployed assets. It was also integrated with the internal WebSphere Service Registry and Repository (WSRR), to allow the promotion of a service in the asset repository to the WSRR, which is used to dynamically discover service end-points during run-time.

Many new technologies are first embraced by the grassroots movement of practitioners who are eager to learn and utilize them for their development projects—often before a top-down focus is initiated. SOA is not an exception, as was illustrated in its early phases during the exploration of Web services within IBM. Isolated activities within the IBM technical community established several service proof points and demonstrated the strong level of interest that helped to justify the resources for corporate top-down activities.

A strong partnership with development organizations is critical to accelerate the adoption of SOA and make it a pervasive way of life for practitioners. This is especially effective if project development activities within a company are allocated to a single development division that is centrally responsible for enterprise-wide development efforts. Although many companies have individual development teams that are “owned” by each of the business units, the central organization model improves the efficiency of sharing resources across multiple projects. This model also has the advantage of managing consistency for project development practices, such as a common SOA approach and a common set of SOA modeling tools, which increases the possibilities for code and service reuse.

The IGA team responsible for IBM internal development has made substantial investments to build the SOA skills of their practitioners and has implemented a systematic process to accelerate solution deliveries (for which reuse plays an important role). As part of this process, an SOA scorecard was deployed that quantifies the SOA penetration in projects, such as services, workflows, ESB creation and use, and process modeling, as inputs into workflow modeling tools. This scorecard is normalized to account for project size (e.g., development budget) to allow a comparison of one project with another. A few of the business units have small groups of their own development teams that are not part of IGA, but have also made improving the SOA skills of their practitioners a priority.

IGA has created a team of reuse engineers who are embedded in projects and whose job is to identify assets that can be reused elsewhere, as well as to introduce assets from other projects for reuse. They are also trained to develop assets in a manner that improves their chances for later reuse and to work with project developers toward that goal.

Overall project-management approaches for SOA-related projects have remained relatively similar to more traditional IT implementation methods. Regardless of the IT design, projects must still undergo the same phases for business justification, requirements, and scheduling. However, SOA has introduced a few additional considerations that have been built into project activities, such as:

• the IGA SOA scorecard mentioned earlier,

• the service-oriented modeling and architecture (SOMA) method, which is used for SOA-related projects to identify the services that a project will produce or consume from other sources,

• process modeling (of existing and planned processes), which is heavily emphasized as a prerequisite for automated workflow projects (as a direct input into workflow modeling tools),

• identifying SOA-related development skills, and

• relying on the IBM enterprise architecture blueprint and the internal SOA asset repository for discovering reusable SOA-related objects.

The organizational structure of several key areas within IBM has a direct bearing on the enterprise-wide common services that can be developed and supported to enable business transformation. Figure 3 illustrates the relationships between the organizations owned by the Business Transformation Executives (BTEs), the Process Transformation Executives (PTEs), and the CIO-led Enterprise Investment Review Board (eIRB), which is an executive forum that coordinates and reviews the funding of the various business areas.

Figure 3

The PTEs, whose mission is to directly support BTE objectives, are the primary drivers of enterprise transformation and are accountable for transformation results. They promote common worldwide, cross-enterprise processes, lead end-to-end process transformations, and manage the process application portfolio. Because their focus spans the enterprise, the common services developed by PTEs are often the best candidates for enterprise-wide service components.

In comparison, the BTEs are responsible for their business-unit objectives and associated activities. They define business-unit requirements and they collaborate with the PTEs on cross-enterprise solutions. The IBM BTE areas are: Software Group (SWG), Systems and Technology Group (STG), Global Services (IGS), Global Financing (IGF), Sales and Distribution (S&D), and corporate functions (CORP). Common services created by the BTE areas often apply only to that business unit. Hence, many of the BTE common services are described as business-unit service components.

The IBM internal SOA Center of Excellence (CoE) is a virtual entity that consists of the CIO team for the SOA initiative, the SOA Guidance Council (including business-unit representatives—the SOA lead architects for their respective business units), and the Technical Leadership team in the IBM Global Account group.

In 2003, the CIO office funded an initiative to promote and coordinate SOA throughout the enterprise as part of the IBM collaborative effort to accelerate internal IT transformation. The business case to justify this expense was based on the future value associated with the SOA expected benefits, such as improved flexibility, speed to market, incremental deployments, extension of legacy ROI or ease of legacy migration, and improved productivity. Some initial estimates of development savings (e.g., 25 percent reduction of workflow updates from early proof of concepts) were also considered, along with the need to showcase an ability to effectively employ SOA for efficient IT solutions.

The SOA Guidance Council generally focuses on more technically oriented topics and consists of a volunteer representative from each of the various internal business areas. The following is a sampling of SOA CoE activities:

• Jump-start SOA projects by providing SOA technology support with skilled practitioners
• Conduct SOMA workshops to identify services that projects or initiatives require
• Manage SOA assets with an SOA asset repository and service registry
• Document metrics, reuse, and business value
• Define and deploy SOA infrastructure support
• Communicate, educate, and document internal SOA references and best practices
• Define and maintain SOA governance
• Promote SOA technology innovation

The virtual SOA CoE has been very successful in raising the awareness of SOA, and how it can be incorporated into project designs. A growing number of SOA-related projects are evident as the business-area lead architects and IGA architects affect the adoption of SOA. However, in some cases, initial overambitious recommendations to migrate to automated workflow or ESBs were later reversed in favor of simple service designs. The recognition of the schedule impacts of these changes led to the publication of decision guides describing when the use of certain SOA characteristics is not appropriate.

The following IBM internal deployments are just a few examples that highlight the variety of applications that leverage SOA. The first three are examples of enterprise service components that have been so designated in the strategic blueprint of the IBM enterprise architecture. The fourth example, “Manufacturing in a box,” is an example of a business-unit service component, to be reused within the business unit.

Compliance with government regulations has received an increased level of attention as the penalties for failure become more punitive, financially and personally (incarceration). The IBM Integrated Supply Chain (ISC) is frequently involved with export management and must comply with government regulations restricting companies from doing business with certain individuals, companies, or countries. The IBM Export Regulation Office publishes export control lists from the United States Government, which are related to the types of products being exported (e.g., products that can be used to support weapon or security systems). ISC created a Web service³ that enables applications to determine whether a potential customer (e.g., during an Internet commerce interaction) is on one of these lists. Names, addresses (physical and Internet Protocol), company, and country information are examples of the data submitted to the service. ISC also provides a manual support process that evaluates “false positives” (due to a required fuzzy search algorithm) and prevents them from recurring.

Some services support a combination of several IT domains, such as security, employee productivity, and outsourcing. The best example of this is the IBM Intranet Password for External Vendors (IIPx) service,⁴ which was one of the first B2B services at IBM. IIPx enables IBM employees to use their existing IBM intranet password to securely access third-party vendor Web sites (e.g., to book trips using commercial travel agencies). It is based on the IBM internal user authentication system, which allows each IBM employee and contractor to have a single set of credentials (user ID and password) to access internal applications. IIPx extends the internal authentication feature to external vendor Web sites to which previously internally managed functions have been outsourced. Employees are first authenticated through their credentials on an IBM internal system, which then redirects their browsers to the targeted external Web site at a vendor's location. A digitally signed and encrypted XML document is transmitted by the browser redirect action to the vendor's Web application, which then uses it to access an externally exposed IBM Web service to verify its signature and the user's identification. The ability of this service to extend IT functions beyond IBM firewalls is an example of the Extended Enterprise Pattern.⁵ In summary, this service improves the ability to outsource internal IT functions by allowing IBM employees to use their internal credentials, rather than create an additional set to access an outsourced function.

The centralized management of customer information is essential to providing a view of “one IBM” to clients and business partners. The Centralized Customer Management System (CCMS)³ was developed by the EBI team to provide a strategic, single create-and-update resource for all IBM customer information. CCMS is an aggregation point for multiple customer data sources that originate from various countries and business-unit transactions. Its cleansing, deduplication, and matching algorithms for these various customer information inputs are combined with company data from the D&B Corporation (Dunn & Bradstreet) to provide a comprehensive view of any customer. CCMS has employed workflow on WebSphere products to drive several of its back-end processes and has provided a Web service to allow applications to query its database on customer information. In addition to Simple Object Access Protocol (SOAP)-based service interfaces, Representational State Transfer (REST)-style interactions have also been explored. CCMS employs several e-business and SOA-related patterns, most notably the Information Aggregation and Process Integration patterns.⁶

By presenting a common view of client information, the application simplifies client interactions. This allows customer relationships to focus on new business opportunities and leads to enhanced customer service.

The decoupling effect of SOA can promote efficient outsourcing, thus containing the cost of IT functions that have become commoditized. If an application is implemented by using SOA and open standards, then it can more easily be outsourced to another company outside of IBM firewalls. When IBM Microelectronics Group outsourced an internal semiconductor testing facility to Amkor Technology, Inc., the IBM facility was down for just the four hours it took to complete the switchover. A self-contained system, the Multisource Data Integrator (MDI), communicates with the “outside world” through standard interfaces and uses the WebSphere Message Broker and a set of services to manage its functions.⁷ MDI is installed at the vendor's physical location to manage the interactions between IBM requests and operations performed by the vendor. IBM Microelectronics calls the MDI a “manufacturing in a box” system, but from an SOA perspective, it is also a virtual “services in a box” system. Its services are generally utilized by internal MDI processes, but they are also available externally for use by the vendor. This is another example of the Extended Enterprise Pattern⁵ that incorporates SOA. Its specific use of the WebSphere Message Broker is an excellent example of the ESB Collaboration Hub and Interaction Direct Connection and Interaction Broker patterns.⁸

The following represents a high-level summary of lessons learned from activities to promote and accelerate the use of SOA within IBM. Some are the result of successful strategies carried out during the initial phase of this journey, while others illustrate the consequences of incomplete transformation activities or newly discovered problem areas.

Here are some of the business lessons learned from SOA activities.

Start with business objectives—do not lead with SOA IT solutions
It is now understood within IBM that SOA is not a “silver bullet” to solve all business problems; instead, it must be applied appropriately to maximize its value. The standard approach of starting with business problems and business goals still applies to all IT projects, whether SOA is involved or not. However, the emphasis of SOA on defining strong linkages between business drivers and IT solutions (e.g., by way of process and workflow modeling) helps reinforce the IBM practice of focusing on business issues.

It is difficult to “sell” SOA based only on marketing promises
The direct approach to SOA-based business value involves the savings in development and support activities that result from reuse. This has been repeatedly demonstrated in several projects by using Poulin's reuse value techniques,^9,10 primarily for service providers with many consumers. However, SOA-driven improvements in flexibility and responsiveness are often difficult to quantify. In a few cases, the gains have been calculated by allowing a reduction in the effort to update code that has been redeployed onto SOA platforms, such as workflow engines. Model-based development methods, which integrate process, architecture, and workflow models and show promise for improved productivity and development efficiency, are being deployed at the time of this writing. A focus on decoupling and decomposition methods to derive gradual business value through incremental deployments (as opposed to an all-at-once approach) has also been helpful to calibrate expectations for the usefulness of SOA. Although the business side is less affected by the SOA hype and is often neutral with respect to the technical implementation methods (as long as the business requirements are met), new SOA skill development costs and additional ramp-up time must still be justified as part of the business case for using SOA.

The additional cost of development for reuse
Experience shows there is an additional cost when developing for reuse as compared with the single-use case. Some groups have successfully incorporated cost-recovery models for funding the development of new services and new service capabilities, as well as ongoing support. However, deploying reusable services is only part of the effort required, because additional effort is expended to migrate consumers to the new services. Also, an asset repository, which supports a single, common view of reusable SOA assets (both actual and potential), is required in order to avoid duplication of development effort.

Assigning ownership of planned, but not yet deployed, common services
The internal IBM structure, which includes organizations corresponding to the various product brands and support functions, results in common service ownership responsibilities spread across many organizations. For required, but not yet deployed, common services, it is often not easy to assign business ownership for enterprise-wide responsibility if a group with a mission that includes the scope of the service does not already exist. This may happen when a business area is narrowly focused and does not already have the mission and the funds to provide enterprise-level services to the entire corporation. The project scope (i.e., mission, schedule, resources, funds, longer-term support needs, and specific business requirements) often does not allow it to create services that can be reused by other parts of the business.

Resource limitations impede wider use of existing services
IBM groups can often provide their services to other IBM groups through creative funding arrangements that allow them to recover the costs when their mission includes the business scope of a service, but does not quite extend to providing it at no cost. In other cases, however, resource restrictions and schedule constraints limit the ability of the organization to make these existing services available to other groups.

Here are some of the management of IT lessons learned from SOA activities.

Ensuring availability of business experts during SOMA sessions
The SOMA method is used as a systematic means to identify the process flows of a project, the services to be created, and the services to be consumed from other sources. Its use reinforces the need to start the transformation activities with the business drivers because models of existing and future processes are crucial inputs to this exercise. Although it is critical to involve business experts as early as possible in these sessions, due to demand they are at times not available. Advanced planning for the availability of participants is extremely important for ensuring that the right skills are represented during the SOMA sessions.

Process modeling with WebSphere tooling has been adapted throughout the enterprise, and is recognized to be a prescriptive element for a successful project. However, due to occasional project resource constraints (e.g., lack of trained analysts or budget limitations), process models of existing or to-be states were not always available. The SOMA method proved sufficiently flexible in the use of alternative documentation for describing current and planned business activities from which service requirements could be derived.

SOA requirements integrated in project management
To ensure the complete incorporation of SOA into architectural practices, SOA has been made an integral part of the comprehensive IBM internal enterprise architecture, which coordinates and integrates the business area architectures. By incorporating SOA recommendations and guidelines into existing enterprise architecture management processes and documentation, SOA concerns are no longer a separate task to be managed and thus the disruption to project management activities has been minimized.

SOA governance established early in the transformation process
SOA governance was integrated into the IBM internal enterprise architecture governance. The developing Web services internal standard was released as one of the first SOA governance publications and helped with the consistency and interoperability of services deployed. Additional corporate guidance, such as EIMS (Enterprise Integration Messaging Specification) and the SOA design criteria of the enterprise architecture blueprint also promoted SOA consistency.

Establishing organizational roles, such as SOA leads, as soon as possible was important for coordinating and improving collaboration during SOA activities. It was critical for each of the business-area architecture review councils (which provide guidance to their respective business areas and review their project architectures) to incorporate SOA-review checkpoints into their own governance processes. This was especially important to ensure project compliance with corporate SOA standards and design guidelines.

SOA acceleration through top-down and bottom-up activities
A partnership was built between the corporate staff and the development organizations that support internal projects. The IGA collaborates on a regular basis with the corporate SOA staff. A common view of an SOA penetration timetable between these two organizations was developed to coordinate activities and to avoid redundant efforts.

The bottom-up embracement of SOA by developers must be nurtured beyond the initial interest in isolated project teams. Experience has shown that development teams must be involved early in the architecture process to acclimate them to new SOA approaches. During the initial phases of SOA adoption, a lack of skills and training programs resulted in slow project startups, and occasionally, in SOA avoidance. Although IGA and other internal development teams have invested in training their practitioners in new SOA-related technologies, it can be difficult to determine who should be trained and when—especially given the uncertainties involved, such as those associated with the timeline for future SOA-based projects.

IGA deployed an excellent reuse tracking program to track and reward reuse by their practitioners. Service-asset registries are essential to promote reuse and are an integral part of the IBM internal enterprise governance. The IBM internal asset architecture board promotes a federation of asset repositories that satisfy the needs of different business areas. As part of its accelerated solution delivery program, IGA also maintains a catalog of reusable assets that includes more than just SOA categories.

SOA needs to be embedded into the practitioner community and culture. IBM Global Services has created a successful internal SOA technical user community to leverage volunteer efforts. New accomplishments are publicized by showcasing successful SOA deployments. However, a company like IBM, which places a high value on innovation, can still have development groups that are slow to embrace reuse. IGA has modified its developer incentive program after experience revealed that the encouragement to reuse existing assets could be more productive than incentives to create reusable assets (which may or may not be reused in practice). Innovation is also about creatively reusing something that already exists and not just creating something new.

Additional interoperability standards
From a service consumer's perspective, some level of consistency in interfaces is required to facilitate interactions involving multiple services, such as when composite business services are involved. The Developing Web Services internal standard was mandated to increase the interoperability and consistency of deployed Web services. This internal standard uses WS-I (Web Services Interoperability) industry standard as basis for extensions that address IBM unique requirements. REST (Representational State Transfer) style services are also being incorporated into the internal standards. As an example, additional requirements and guidelines are being written into this IBM internal standard to ensure a consistent description of REST services, because this style is not governed by industry standards (such standards do exist for SOAP-based services, namely Web Services Description Language). Messaging standardization is achieved with the IBM internal Enterprise Integration Messaging Specification (EIMS).

Plan for enabling the infrastructure for SOA
Infrastructure teams need to be involved early and collaborate on determining the set of SOA products that need to be deployed onto the infrastructure in advance of a project rollout. IBM has internally shared infrastructures that provide servers and standard software platforms (e.g., WebSphere Application Server, DB2*, and WebSphere Process Server) to applications. However, if a project requires new software products or versions not yet deployed onto a shared environment, then it could be subject to early adopter penalties of one-off, higher-cost servers for unique requirements.

For new and evolving technologies like SOA, it can be difficult to determine which products should be deployed onto shared infrastructures. A focus on business value helps to guide selection processes to define which products need to be added, with the goal of benefiting as many applications as possible.

The IBM internal SOA strategy is to collaborate to make SOA pervasive throughout the enterprise and to ensure that its value—to impact internal business transformation efforts positively—is realized. Significant progress has already been made to reach this goal, and work continues to embed SOA approaches completely into the consciousness of the business, by creating a set of self-sustaining activities to incorporate SOA into IBM projects and initiatives.

We thank all IBM employees who contributed to the work described in this paper and extend thanks especially to Catherine Winters for her enterprise architecture leadership, to Jay Strosnider and Jim Penney for their efforts to institutionalize SOA in IGA, to Don Willenborg and Jamshid Vayghan for Enterprise Business Information strategies, to LeeAnn Kania and Greg Terwilliger for the Export Validation showcase, to Akram Boughannam for the Customer Information showcase, to Steve Pike for the Manufacturing-in-a-box showcase, to Geoff Meissner for his SOMA extensions, to David Chen for his ESB+ reference architecture, to Patrick Rooney for EIMS, and to Dick Panko for the SOA initiative's project management activities.

*Trademark, service mark, or registered trademark of International Business Machine Corporation in the United States, other countries, or both.
**Trademark, service mark, or registered trademark of Object Management Group, Inc., in the United states, other countries or both.

Accepted for publication June 18, 2007; Published online November 7, 2007.