Improving service delivery through integrated quality initiatives: A case study

Service providers often undergo multiple, separate diagnostic efforts, such as ISO (International Standards Organization) 9001:2000 audits for quality management, ISO 27001 audits for security, ISO 20000 for IT service management, and eSCM-SP (eSourcing Capability Model for Service Providers) evaluations for determining the capability of a service source. The integration of these diagnostics and service delivery processes is critical to measuring the performance and quality of service delivery, and this integration presents a challenge to the service-management and service-sourcing communities.

Most global service providers use processes based on the best practices from multiple standards. Thus, separating the evidentiary material needed for any one diagnostic is counter to the way of doing business at these organizations. In addition, separate diagnostics are expensive in terms of preparation of process documentation to meet requirements for evidence and the disruption of work that is necessary to enable audits to occur.

The types of evidence collected for each of these diagnostics include process documentation, interviews, and demonstrations to prove that the implementation of these standards is occurring. This paper describes the feasibility and reliability of reuse of evidence for multiple diagnostics, which has not been empirically shown prior to the study described in this paper. We present this case study in an effort to examine some of the issues involved in process and diagnostic integration.

A case study¹ was determined to be the appropriate research method to use, given the complexity and qualitative nature of the data to be collected. We describe the service quality initiatives under way in a specific organization, a delivery center of a major global service provider. We summarize briefly the history and demographics of the organization and present detailed data regarding the reuse of documentation and data for two diagnostics of this organization. The first was an internal ISO 9001:2000 audit,² followed immediately by a capability determination (self-appraisal) using the eSCM-SP.^3,4

How does a major global service provider, such as Accenture, Electronic Data Systems Corporation, Hewlett-Packard, or IBM, assure its customers of service quality? One method is to use industry standards for quality. Until recently, there were no specific quality standards for strategic outsourcing. Historically, many companies have used the most widely recognized quality standard, ISO 9001:2000, to measure their outsourcing service quality. While ISO 9001:2000 is an internationally recognized standard for quality management, it is a generic standard that assures the customer that the organization has deployed a system to manage quality, and does not reflect the best practices that are specific to the outsourcing industry.

With the advent of a standard specifically for IT-enabled sourcing, eSCM-SP, a new direction was made possible for service quality management. eSCM-SP offers several benefits:

1. It is specific to IT-enabled sourcing, and incorporates best practices for that core business;

2. It includes a capability model that offers certification at increasing capability levels. This creates the opportunity for differentiation in the market and also provides a road map for quality improvement;

3. Unlike ITIL** version 3 (Information Technology Infrastructure Library),⁵ which is the framework for best practices in IT service management, eSCM-SP provides full coverage of the sourcing life cycle and governance of IT service management; and

4. This standard was created with the recognition that successful outsourcing requires capabilities that are complementary for both the service provider and client. The standard does this by defining two models: eSCM-SP for the service provider and the eSourcing Capability Model for Client Organizations (eSCM-CL,⁶ a separate but complementary set of best practices) for the customer.

It is our intent in this paper to describe a case study in applying eSCM-SP and ISO 9001:2000 in a service system. In so doing, we hope to provide information that will be helpful to service providers who are expanding their global quality framework from one which is limited to ISO 9001:2000 to one which encompasses eSCM-SP as well.

The case study took place in a global service delivery center that began operation in January 2005 and provides infrastructure services and solutions such as server administration, user ID administration (logical access), storage and communications management, asset management, and help desk operations. This organization employs approximately 1000 personnel in an offshore facility that supports IT services operations for six regions and for several market sectors. The primary business goal of the organization is rapid growth in the number of accounts it services. To assist in the achievement of this goal, the organization embarked on a series of quality evaluations designed to instill an organizational culture based on the use of consistent and repeatable processes and measurement throughout the organization.

ISO 9001:2000 registration was pursued by this organization to implement and enforce this process structure. The organization passed both an internal and an external ISO 9001:2000 audit, resulting in ISO 9001:2000 registration two months prior to the beginning of the eSCM-SP capability determinations. The internal eSCM-SP assessment was conducted as a readiness check and was used to collect much of the data reported in this paper. It was followed by a third-party external evaluation resulting in an eSCM-SP certification for the organization.

The ISO audit examines compliance with ISO 9001:2000 quality management system requirements. The internal audit includes examination of documents and interviews with responsible parties and is conducted by trained and authorized ISO auditors. These requirements are stated in ISO 9001:2000 in the form of five major clauses: (a) quality management system; (b) management responsibility; (c) resource management; (d) product realization; and (e) measurement, analysis, and improvement. To become registered, an organization must demonstrate that it conforms to the requirements of all the applicable clauses of the standard.

The eSCM-SP diagnostics consist of a rigorous examination of guidance documentation (policies, procedures, guidelines, job aids, etc.), interviews (done individually and under the terms of a nondisclosure agreement), implementation reviews (demonstrations, unobtrusive observations, etc.), and a review of artifacts (minutes of meetings, completed templates, etc.). All of these sources of data were reviewed and verified by small teams and then considered and rated by a five-to-seven person team of trained and authorized evaluators. The 84 best practices from the eSCM-SP were used as the reference model and the data we present here shows reuse for 78 of these practices.

The best practices are grouped into 10 capability areas. Six of these are ongoing throughout the sourcing life cycle and focus on the management of knowledge, people, performance, relationships, technology, and threat (e.g., to security or intellectual property). We do not address the remaining four capability areas (namely, contracting, service design and deployment, service delivery, and service transfer) in detail in this paper.

The notation used in the next section is known as the “short identifier” for each of the practices. For example, knw01 is “Knowledge 01”. See Reference 4 for details of these practices. In the following, the organization of interest is denoted as “DC1 (Delivery Center 1).”

The following analyses examine three factors: (1) the reuse of diagnostic evidence by ISO 9001:2000 and eSCM-SP for DC1; (2) ISO 9001:2000 coverage between ISO 9001:2000 and eSCM-SP as defined in a previous technical report released by Carnegie Mellon University;⁷ and (3) the process level examined in the eSCM-SP DC1 external evaluation.

These three factors provide information to the greater organization regarding the effort required in moving from ISO 9001:2000 to eSCM-SP certification. Table 1 displays each practice for the three factors. The table is followed by an in-depth analysis and reporting for the six capability areas mentioned previously. Further discussion of notable practices and opportunities for improvement follows in this section. Improvements may be in either the organization or in the eSCM-SP implementation effort.

The concept of coverage versus reuse is significant to this discussion. eSCM-SP is designed to complement other quality models. Figure 1 illustrates the relationship between ISO 9001:2000 and eSCM-SP. For each of the 10 capability areas, the blue shaded region indicates how much of eSCM-SP is covered by ISO 9001:2000. Coverage refers to the correspondence between the clauses of ISO 9001:2000 and the practices and capability areas of eSCM-SP. A full discussion of coverage is contained in Reference 7. Reuse, in this case study, refers to the relevance and sufficiency of documents needed to satisfy the evidentiary requirements of the diagnostic teams for both the ISO and eSCM-SP efforts.

Figure 1

In this section, we describe the findings of our case study as regards the level of reuse between ISO 9001:2000 and eSCM-SP and the use of global versus local processes for the eSCM-SP best practices in the six capability areas we focused on, as mentioned previously.

The knowledge management capability area comprises eight practices that relate to managing information and knowledge systems in support of the sourcing environment. DC1 showed the highest level of reuse between ISO 9001:2000 and eSCM-SP in this capability area. This supports previous findings that showed an 80 percent overlap between the knowledge management capability area and ISO 9001:2000.⁷ Five out of eight practices showed evidence of global process use. In two of the practices, a global process was available, but not used. The creation of a different local process was appropriate and even preferred to the use of a higher-level process. In one practice, there was potential for the local process to be used at a higher level to improve the higher-level organization. Notable processes within knowledge management are discussed in the following.

The practice knw01 focuses on the establishment and maintenance of a knowledge-sharing policy. While ISO 9001:2000 does not explicitly require a knowledge management policy, it does assign management responsibility for the establishment of a quality policy as well as communication and review of customer quality objectives. On its face, this ISO 9001:2000 clause appears not to be linked to knowledge management. However, the intent of ISO 9000 management responsibility is met in part by the requirements of knw01 to encourage communication for the purpose of learning and improving performance. Reuse of documents between this practice and ISO 9001:2000 supports their linkage.

knw02 focuses on providing required information to personnel so that they can successfully perform their jobs. The organization used both global and local processes; however, the primary process was a locally created process. The use of the local process for this practice is most likely due to the relative newness (2 years) of the delivery model that this delivery center operates within and points to an area for improvement in which the same process could be used for all delivery centers within this delivery model.

knw05 focuses on analyzing and using knowledge that is gained through working with customers and the use of a “lessons learned” methodology for obtaining, analyzing, and using customer information. DC1 had a local procedure in place to capture lessons learned. This practice could be satisfied with a global process, but the analysis of the local procedure provides more value regarding the performance of the organization of interest. ISO 9001:2000 fully covers this practice as it applies to the gathering and analysis of information relating to customer satisfaction, product quality, and continual improvement. However, eSCM-SP expands on these functions to include profitability and productivity.

knw06 specifies the reuse of work products within the organization across different customer accounts, such as the reuse of policies, technical designs, processes, or methods. DC1 met this practice with prolific cross-client reuse of many global templates, processes, and enabling tools. ISO 9001:2000 does not require the reuse of work products; however, several clauses within the product realization clause support a minimal level of reuse of processes, methods, and procedures. Within DC1, there was a higher level of reuse of evidence for the satisfaction of ISO 9001:2000 and this eSCM practice than was expected. This is because the evidence used in the eSCM-SP evaluation to demonstrate the reuse of work products in the organization was the evidence that demonstrated for ISO 9001:2000 the establishment and use of a quality management system. This included templates for processes, templates for the quality control books, and documentation used to support the processes. This level of evidentiary reuse was an artifact of the way in which ISO 9001:2000 was implemented in DC1 (e.g., the use of templates) and may not be applicable to other organizations.

knw07 focuses on version and change control of work products. DC1 adapted the global processes for document and record control. This practice is notable because it is one in which ISO 9001:2000 directly contributes to the satisfaction of eSCM-SP. ISO 9001:2000 requires procedures be in place that address how the organization will control change to relevant documents and records. ISO 9000 also specifies that the organization provide for product or service identification and traceability throughout product realization. This practice can be satisfied by the implementation of document and record control procedures as well as mechanisms to provide tracking and identification of services (for example, configuration management). One hundred percent reuse for this practice demonstrated the connection between this practice and ISO 9001:2000.

knw08 focuses on resource consumption. DC1 has a local process in place to manage resource consumption, and this was used for the eSCM-SP evaluation. It uses global tools to manage resource consumption. Usually, global tools and procedures are sufficient to meet the need for resource consumption. However, the local process represents an improvement this organization made because of its demand for extraordinary growth and the need for close scrutiny of resource consumption to manage the growth. ISO 9001:2000 covers the provision of resources rather than resource consumption. Depending upon how an organization, such as service organization, measures its process performance, the consumption of resources could be indirectly covered. The practice does not require prediction of resource consumption, but does not preclude it. The reuse of documentation between the two diagnostics did include predictive data. This is a case in which a quality management system (QMS) that includes resource consumption predictive indicators will result in higher reuse between standards, even though ISO 9001:2000 does not require it.

The people management capability area contains 11 practices related to organizational personnel practices that underlie successful sourcing. The organization showed a very high level of global process penetration in this area. Nine out of 11 processes were satisfied by global processes. This is the result of a companywide focus on innovation and IT capability and skill. Emphasis on employee retention and development is crucial to IT capability.² There is moderate (60 percent) coverage of the people management capability area by ISO 9001:2000.⁷ The moderate reuse of people management shown by the organization in the ISO 9001:2000 internal audit supports the coverage finding.

The reuse of evidence was affected by the human resource management focus of ISO 9001:2000 versus eSCM-SP. ISO 9001:2000 focuses on personnel in the context of ensuring that the right skills are in place to ensure product or service quality. eSCM-SP includes this and additionally assesses that mechanisms are in place to ensure employee satisfaction and retention. Because ISO 9001:2000 does not look at these factors, the reuse of evidence was lower.

The practice ppl01 focuses on establishing a policy to encourage innovation. In the eSCM-SP evaluation, one program, which is an annual global online event to encourage and implement innovative ideas, was cited. However, the global organization has a number of global programs beyond the cited program. Innovation is highly prized in this organization and is part of its legacy. It is fostered through a number of programs focusing on innovation, especially in the area of rewards and recognition. The fact that only one program was cited in the evaluation is most likely due to the relative newness of the organization. As the organization develops in expertise, it will be able to contribute and take advantage of other available programs. This growth was in evidence during the evaluation as the organization was embarking on participation in a global quality improvement initiative designed to produce innovation in service delivery workflow. This program could not be used in the evaluation because the work was planned but not yet under way, but is notable because it indicates potential for future organizational innovation. ISO 9001:2000 does not require policies or programs directly focused on innovation. Even so, there was reuse between ISO 9001:2000 and eSCM-SP. This can be attributed to the inclusion of innovation programs as part of the quality management system.

ppl02 focuses on obtaining the participation of personnel in decisions about their work commitments. This ensures that commitments are made that are informed by the knowledge of the staff. Employee participation in work commitment decisions is only minimally covered by ISO 9001:2000 (see clause 6.2.1, human resource-general and clause 6.2.2d, competence, awareness, and training), but is crucial to employee satisfaction and retention.

ppl03 focuses on establishing a work environment that allows employees to work effectively. This practice covers establishing an appropriate physical work space and also addresses the handling of disputes in the workplace. ISO 9001:2000 clauses 6.3 and 6.4 address the need to establish, determine, and maintain the infrastructure and a work environment appropriate to meeting requirements. ISO 9001:2000, does not, however, cover a system for handling employee disputes.

ppl04 focuses on assigning roles and responsibilities to people based on their competencies while ppl05 focuses on the definition and communication of roles, responsibility, and authority of employees. ISO 9001:2000 clauses 5.5.1 and 7.3.1 contribute to the coverage of these practices by requiring that roles, responsibilities, and authority be documented and assigned in the organization. Although reuse is lower than one would anticipate, it is high in the context of the documentation of the actual roles and responsibilities. The documents that were used by eSCM-SP but not by ISO 9001:2000 related to the procedures by which roles and responsibilities are assigned. ISO 9001:2000 did not require these procedures, only the demonstration that roles and responsibilities were established and communicated. ppl04 used a local procedure for the assignment of roles. This procedure took outputs from a global transition procedure that identifies needed staff. However, the growth of the organization dictated the use of additional local procedures to control resource assignment.

ppl07 focuses on providing a cross-account training function that analyzes organizational needs, provides training solutions, tracks training, and improves training effectiveness. Although a local training procedure was cited in this audit, training planning and provision is performed though a global function that was not part of the audit scope.

ppl10 focuses on career development. While DC1 satisfied this requirement by using the global employee development process and tool, there is additional global support that was not cited in the assessment which is created and maintained by the corporate human resources department. It includes career paths for globally recognized job roles. Although ISO 9001:2000 does address skills development and training in support of meeting quality requirements, career development for long-term employee and organizational goals is not addressed.

The performance management capability area focuses on setting objectives, measuring and improving performance, and meeting customer requirements. What is most significant about this capability area for this case study is the low percentage of practices that were satisfied by global processes. Of 11 practices, two practices related to capability baseline establishment and benchmarking were not evaluated because of insufficient evidence collected during the eSCM-SP evaluation, seven practices were satisfied using local processes, one practice was satisfied by global processes, and the remaining practice was satisfied by a mixture of global and local processes. The reason for this is not the lack of global processes, but rather the special circumstance of this organization. The organization is growing at such a high rate, it is appropriate and desirable for it to use a local performance management process that can ensure that quality is maintained while growth is pursued. Eight of the practices had the majority of their activities satisfied by a highly structured performance review process instituted by the local management.

ISO 9001:2000 coverage of this capability area is 70 percent. Reuse at the practice level was quite high (with seven practices having more than 80 percent reuse) for this capability area. This is because performance is managed by DC1 primarily through the same system that manages quality.

The practice prf05 focuses on reviewing organizational performance. Although the reuse findings appear to be lower than the coverage would lead one to expect, an evaluation of the evidence shows that this practice was primarily satisfied by processes that had been used in ISO 9001:2000 compliance. In fact, evidence for ISO 9001:2000 compliance could have satisfied this practice without the addition of other evidence. Thus, while the reuse percentage appears low (57 percent) for a practice that is fully covered by ISO 9001:2000 compliance, this is an artifact of the audit. The eSCM evaluators found evidence beyond what would have satisfied the practice that was not used in the ISO 9001:2000 audit.

The practices in the relationship management capability area focus on managing interactions and relationships with clients, suppliers, and partners. There are eight relationship management practices. Reuse of evidence for ISO 9001:2000 compliance was 58 percent, which was the lowest reuse of all of the capability areas. This stands in contrast to the 65 percent coverage reported in Reference 7. The reason for this is that eSCM-SP focuses more on the formal relationship management of clients, suppliers, and partners, while ISO 9001:2000 focuses primarily on ensuring that goods and services procured from suppliers meet quality requirements. The majority of processes that satisfy relationship management requirements are global, since the global organization puts a high emphasis on relationship management. In each of the processes for engagement, transition, and steady-state service delivery, there are process steps and tools to capture the personnel authorized to act on behalf of each party, their roles and responsibilities, and the communication channels and methods.

rel01 focuses on managing interactions with clients. This practice looks for a defined customer contact channel, defined roles and responsibilities, and a communication plan that provides an outline of the communication vehicles which are used to ensure clear communication with the customer. DC1 used a mixture of global and local process to satisfy this practice. Reuse in this practice was 50 percent, even though the practice is fully covered by ISO 9001:2000. The evidence that satisfies ISO 9001:2000 clause 7.2.3, providing for customer communication, covers the requirements for rel01. The primary business acquisition and transition processes that satisfied this practice were used in ISO 9001:2000. The tool used for account management was cited by both ISO 9001:2000 and eSCM-SP. However, there were additional steady-state relationship management procedures used in the eSCM-SP capability determination that were not used in ISO 9001:2000.

rel03 focuses on supplier and partner management, rel07 expands the management of suppliers and partners to multiple accounts. For both practices, reuse was very low (20 percent) between the evidence used for ISO 9001:2000 compliance and eSCM-SP. This is because eSCM evaluates more of the mechanism by which suppliers and partners are managed than ISO 9001:2000, leading to greater evidentiary requirements to satisfy eSCM than ISO 9001:2000.

re108 focuses on identifying opportunities for creating value and communicating them to the client. ISO9000 does not cover value creation; however, eSCM-SP evidence for value creation was reused 100 percent in ISO 9001:2000. This is because the value creation program of the organization is a component of the quality management system, causing it to be assessed in the ISO 9001:2000 audit.

The technology management capability area focuses on ensuring an adequate technology infrastructure is available to meet the requirements of service delivery. It includes change, release, and performance management. It also includes licensing and systems integration between service providers and clients. The organization had limited responsibilities for some technology management practices, so the findings regarding process reuse and globalization may not generalize to all other cases. Legal and regulatory issues limited the transfer of software and hardware to the organization. However, it retained responsibility for change management, integration, and optimization. Coverage of the technology management capability area was 45 percent in Reference 7, which was second lowest compared to the other capability areas. Reuse results supported this finding (61 percent). The low findings for reuse and coverage are due to differences in requirements between ISO 9001:2000 and eSCM-SP. ISO 9001:2000 addresses only a portion of the infrastructure scope that is specified in eSCM. ISO 9001:2000 coverage of infrastructure is focused primarily on facilities that are used to achieve conformance with product requirements. eSCM expands on this to include the technology used to deliver outsourcing services.

The practice tch 03 covers technology change management. It is notable for its low reuse between eSCM-SP and ISO 9001:2000. The organization used the global change process and tool, but the ISO 9001:2000 audit reviewed only the tool and not the process. ISO 9001:2000 requires maintenance of the infrastructure, but does not specify, as eSCM-SP does, the maintenance of a documented change management process.

tch06 focuses on proactively introducing technology. Although ISO 9001:2000 does not cover the proactive introduction of technology, there was still 100 percent reuse of evidence. This is due to the role that management review and the quality management system play, in this organization, in identifying potential technologies for use and tracking the introduction of those technologies.

The threat management capability area focuses on security, confidentiality, organizational and engagement risk, and disaster recovery. ISO 9001:2000 does not focus heavily on this area, relying on the ISO 27001 standard⁸ to cover security. Reuse was low for some practices in this capability area.

thr01 focuses on having a policy in place for risk management. This practice is notable not for its low level of reuse (as one would expect) but because of its use of local processes in addition to global risk management processes. Because the organization is growing at such a fast pace and its core business is supplying skill for IT infrastructure management, most of its risk management activity focuses on ensuring adequate skills and staffing for accounts. This focus resulted in mostly local processes being scrutinized in the eSCM self-appraisal, rather than the more generalized global process.

thr07 focuses on disaster recovery. ISO 9001:2000 does not cover disaster recovery or business continuity, leaving it to the ISO 27000 standard.

There was significant reuse found between the two diagnostics and the processes that were examined, as shown in Table 1. The high reuse appears to be related to the maturity and the design of the DC1 quality management system. Many eSCM-SP practices that ISO 9001:2000 does not cover (e.g., re108 and tch06) reused ISO 9001:2000 evidence. Much of this reuse related to innovation or proactive measures that were documented in the quality management system. ISO 9001:2000 requires a quality management system that focuses on continual improvement of quality, and preventive and corrective action. It does not require innovation or programs that focus on proactively improving efficiency or performance. However, an organization that has a focus on innovation, performance, and efficiency as part of quality management will see a higher reuse of evidence between eSCM-SP and ISO 9001:2000.

The difference in requirements between ISO 9001:2000 and eSCM-SP regarding innovation and proactive performance and efficiency programs also points out the higher sensitivity of eSCM-SP as a diagnostic quality tool in an IT organization. This is significant for customers and service providers who are interested in improving their IT quality. ISO 9001:2000 is a generic standard for quality management systems and does not differentiate between an organization that is using reactive management of IT service delivery and one that is using innovative and proactive management. eSCM-SP does make this differentiation. Given the strong emphasis by many leaders on innovation, this suggests that eSCM-SP is a useful diagnostic for measuring the capability of higher-capability service providers.

Another conclusion that may be drawn is that the high reuse of evidence coupled with the high utilization of global standard processes by DC1, if consistent across the larger organization, should ease the movement of the global organization from ISO 9001:2000 to eSCM-SP. Table 2 can be used to guide the organization as to where further research needs to be done and may provide guidance on how parts of the organization can be tested for process consistency and integration.

Many quality models include requirements for a “plan-do-check-act” (PDCA) cycle⁹ and this is true of ISO 9001:2000. Implementors of quality management systems that comply with ISO 9001:2000 are expected to collect measures which fall into three categories: (1) customer satisfaction (clause 8.2.1); (2) process quality (clause 8.2.3); and (3) product quality (clause 8.2.4). The term product, in this context, also includes service. The requirements in this standard are generic and may be applied to any size and type of organization, including service organizations. The purpose of measurement in this standard is to demonstrate product and process conformity as well as continual improvement of the quality management system.

Service delivery organizations implementing eSCM-SP are encouraged to use a “goal/questions/metric” (GQM) approach to measurement, which has an underlying premise that the measures should provide insight into attaining business goals. Recommended measures for IT-enabled service delivery organizations implementing eSCM-SP include the following kinds of measures: (1) cost/effort; (2) status/progress; (3) nonconformance; and (4) performance/satisfaction.

Future work is needed to support the effective process integration, co-registration, and co-certification initiatives that are important for the service provider community. These efforts should include cases documenting reuse for additional quality registrations and certifications, in particular, ISO 20000 (ITIL), ISO 27001 (Security), and CoBIT** (Financial/IT Controls).

On the basis of empirical studies, standards organizations and the provider community should be positioned to make fact-based decisions about co-registrations and co-certifications that could lead to significant cost savings while ensuring that critical certifications to ensure service quality are enabled. While some would suggest that organizations should choose one quality framework or standard, we find that providers' clients expect them to have implemented multiple standards rather than any single one. It is this market reality that leads the authors to the conclusion that process integration and co-registration or co-certifications are needed.

**Trademark, service mark, or registered trademark of the Office of Government Commerce or the Information Systems Audit and Control Association and the IT Governance Institute.

Accepted for publication August 15, 2007; Published online January 24, 2008.