Mauro Martini
“Comparison” and the “benchmarking”
Differently
from a usage, sometimes incorrect, of the term in Public Health environments,
the “Benchmarking” can not be taken as a mere comparison of activities or
processes between two health services firms. It is necessary to premise that
we don’t want to refuse the absolute importance of a mere “comparison” between
activities and processes, which also is of remarkable managerial utility,
but yet different from the benchmarking.
Besides, it is useful to notice that the “comparison” is not necessarily realized
between two structures of different health services firms; it is possible
to draw profit even from a comparison, in time, of the production processes
of the same service. In this sense, the comparison is meant essentially in
two typologies (see table 1).
First
of all a “cross section” comparison, which is necessarily synchronic, between
two services, or departments, or hospitals (of the same, or of different health
services firms), whereas the processes are comparable. The “cross section”
comparison does not have to be referred to a “best in class” structure: remember
the regional reports in Italy, where data from the various health services
firms are compared. Instead, the most important problem is the comparability
of the data. About it, Casati reminds the need that the necessary homogeneity
for a correct comparison has to be: structural (similar dimension); functional
(comparable case – mix); countable (same survey methods and elaboration of
the costs).
The second comparison type, on the contrary, is diachronic, “over time”, that
is a monitoring (such has to be considered this case) of the modifications
that happen in time, in terms of output and of outcome. In that case the choice
of a unity of time is imposed: month, year, etc. Even if it seems pleonastic
to remind it, it is opportune to underline that, except for different indications
imposed by the circumstances, the comparisons over time happen regularly between
similar periods of time; for example: the first quarter of two or more following
years, or the second semester of two or more following years, the annual data
for a certain number of years and so on. (Table
1) The diachronic comparison suggests the trend of the results, within
the limits where suitable attention to the homogeneity of the absorbed resources
and of the operational conditions was paid. The synchronic comparison involves,
on the contrary, above all the < relationship between the human, financial
and instrumental resources used and the quantitative, qualitative and economic
level > of performances, even though we must underline that it is < very difficult
in public administration to have comparable data > (Macrina and Ravaioli),
since often the method of collecting data is not properly shared, in order
to obtain a reliable limit of homogeneity, that is even before the elaboration
and interpretation of the resulting numerical values. It is clear that the
comparison can concern: quantitative aspects; qualitative aspects; economic
– financial aspects. If the comparison is set with the best structures that
operate in a similar range of activities (“best in class” in Anglo-Saxon literature)
it is possible to evoke the term “benchmarking”. The benchmark to which we
report, therefore, is the optimal standard of performance.
Definition and classification
The
number of definitions of “benchmarking” in literature is uncountable. It is
necessary, first of all, to remember the definition given by the man to whom
the paternity of the benchmarking belongs to, Camp. He defined such tool of
management as a continuous process of product measuring, services and business
routine, through the comparison with stronger competitors, or with leading
enterprises of a sector. Within the plentiful literature on the matter, even
Italian contributions are numerous and authoritative, some of which derive
from, or however find position in the specific field of Public Health. In
particular, Zanetti reminded that in business management, benchmarking < means
a management technique, that consists in activities of structured and permanent
evaluation, with the purpose of comparing the products / processes of one’s
own health services firm with the market leaders >, in order to underline
also that < the differences found considering the standard of the health services
firms of reference are analysed to define strategies, aims, improvement plans
and courses for tending to excellence >. In any case, if on one hand, mainly
in business management, the benchmarking is the process of analysing the indicators
of success to be used to make the health services firm grow and to improve
the products’ quality, on the other hand, more properly in health areas, the
benchmarking represents the critical point of view, or an “instant photo”
(Roediger) of the practices adopted, that lets us determine which are the
preferable and the “best in class” to which we should refer. Yet, because
of the market peculiarity, the benchmarking can assume different connotations
in Public Administration, in comparison with the private, where it was born
and improved with the passing of years.
Besides, we also believe that < in Public Administration the benchmarking
should become of collaborative type, not only for the scarce competitive mechanism
that links public health services firms, but also for the necessity to raise
the performance standard of all health services firms that offer public services
> (Compagno and Cagnina).
Nevertheless, from the impelling necessity to align with the best Public Services
in the European Union, it turned out the demand that the Italian Public Administrations
recover in efficiency terms, effectiveness and management inexpensiveness,
also in compliance to the norms emanated by the government in the last ten
years (Marchitto). Even about classification criteria, the literature about
benchmarking is wide. The division offered by Masoni is certainly original.
He distinguishes a normality benchmark (where the results of others are assumed
as a norm for the project), from the excellence benchmark (set – in relation
to the results obtained from others – as a sight to overcome). Azzone, instead,
distinguishes a performance oriented benchmarking from a processes oriented
benchmarking: in the performance oriented benchmarking the “best practice”
is a < point of reference for other organizations > and it constitutes, therefore,
a < stimulating aim (opposite to the historical data) >; in the processes
oriented benchmarking, on the contrary, the comparison concerns (if the routines
can pacifically coexist) < management modalities of the different organizations
>. Agreeing with Compagno and Cagnina, our opinion is that it is needed to
distinguish first of all: I. the environment of the comparison (internal or
external); II. the object of the comparison (institutional finality).
Classification in accordance with the environment of the comparison
We
identify: the internal benchmarking, when performed in the context of the
organization, usually among services that have the same institutional duties,
but are located in different geographical areas of the same firm. The internal
benchmarking is easier to realise, even because it evokes less resistances,
it is feasible in acceptable times and it is a useful tool for a first knowledge
and verification of the business processes. In public administration, the
reasons that can make the difference in the services performance, in different
territorial realities, have been stated by Marchitto; we remind the most useful:
the socio-cultural model and, most of all, behavioural, possibilities of interpretation
of the norms, modalities of distribution of the service, attitude toward the
consumers, sense of duty, of affiliation and of the hierarchy, ability to
assume decisions. For what was quoted previously, he notices that such routine,
often, is not a real benchmarking, but a mere comparison with a structure
that offers good quality, even if rarely meaningful for a best practice. The
external benchmarking, when carried out with external firms, sometimes competing.
The external benchmarking implies the comparison of some business processes
with those used by accredited organizations in the field. It may be (but not
necessarily) competitive; in this case, some difficulties may arise to establish
a constructive dialogue with the partners, with the risk to compare only production
factors, rather than processes (Trivellini and Caliendo). The functional benchmarking
or process oriented, when performed with firms that act in other sectors.
The compared object is a definite function, or a determined process, independently
from the fact that the products / services of the two compared firms are different.
Usually these are support activities, such as personnel management, economic
service, computer service, etc. The generic or pure benchmarking, when performed
with firms that are considered absolute leaders in the sector (the best in
class), that < contains the basic philosophy of the benchmarking > (still
Trivellini and Caliendo). Such form of benchmarking implies a remarkable and
consolidate experience, since as a rule absolutely different procedures are
analysed.
Classification in accordance with the object
We
distinguish: an operational benchmarking, that involves the comparison among
the services offered; the processes that bring directly to supply / distribution
of products / performances are verified; a managerial benchmarking, or the
comparison among the procedures of support to the production line; a strategic
benchmarking, with which the determinants in the competitive advantage are
analysed, through the observation of the strategies that have brought other
organizations to success.
About the choice of the benchmarking partner, we recall the main elements,
often underlined in literature; in order of importance (still Compagno and
Cagnina):
I. localization;
II. affiliation to the same branch of activity;
III. potential credibility or reputation;
IV. will to participate;
V. performance or supposed leadership.
About this, Marchitto underlines that < hardly a single firm or a single administration
realizes, contemporarily, excellence situations in all the process phases
>, for which it is necessary to figure out the partner in relation to the
process to be submit to benchmarking, after you have verified that such partner
is the “best in class” in that process and not in general.
Azzone, after reminding that the organizations have to be separate, first
of all, on the performances level, or in the quality of the managerial solutions,
he thinks that the organizations can be identified in relation to four typologies:
I. ”star”, leader organizations with very high performance levels;
II. emerging organizations, in course of improvement;
III. declining organizations, characterized by obsolete processes and obsolete
technologies;
IV. “dog” … whose term (no offence to the man’s best friend) appears clearly
and explains sufficiently.
The Benchmarking path
The
planning of an effective system of benchmarking involves a correct methodological
articulation, that is a really effective path in order to obtain a concrete
process of improvement in the quality of performances: that also is the subject
of many scientific contributions (see table
2). A careful analysis of the operational models, mainly of the courses
used to carry out the benchmarking, allows to notice that, in fact, they can
be referred to the known cycle of Deming (see table
3). Marchitto, considering the need of having, before the benchmarking
planning, a sort of auto-diagnosis about < the weak points and the errors
of the preceding cycle >, he suggests a chronological change, precisely from
PDCA to CAPD, in accordance with the sequence shown in
table 4. In the path sequence, independently from the clearness of
the ideas on what is to be compared, the greatest difficulty stays however
in the choice of the partner to compare to, even for the often unsolved matter
of the identification of the best in class in Public Health.
About the process to be verified, on the contrary, it is good rule to focus
the attention at first on the “critical factors of success” (Cevolani), which
means to perform first of all a mapping of the processes (for example by a
flow-table) and then rearrange these in accordance with priority, to be submit
to benchmarking.
Problems of the Benchmarking in Public Health
The
benchmarking derives from the experience of great industrial firms, even if
some principles can be transferred in the field of Public Health.
Yet, certainly, the results in sanitary field are difficult to be identified
and interpreted, for rather a lot of reasons: first of all it is difficult
to test the results directly; whereas the measures are somehow identified,
they are not always homogeneous and/or the same used among benchmarking partners;
the results can be measured in different dimension contexts, so that a good
result, obtained in accordance with a specific term of evaluation, could compromise
the result in accordance with another parameter (this can be observed more
clearly in the perspective of the trade-off between costs and effectiveness);
the results of a benchmarking are not necessarily to be referred to one of
the treatment components; rather, they are a consequence of different phases
in the search and in the evaluation; not all these phases are verified during
the permanence in the hospital, so that the deductions about the hospital
performance and the results could be even absolutely fictitious; it is possible
to observe still the persisting of political, psychological and sociological
factors for which an organization doesn’t succeed in reaching the performance
level of the predetermined benchmark; in the end, the limits previously remembered,
often, in sanitary field, make the identification of the “best practices”
problematic.
Nevertheless, though it is difficult to identify proper indicators of outcome
for medical treatments, it is yet possible to test the processes and the output
of treatments, that contribute to produce outcome.
Again, even if a series of proper indicators are towards the performance of
the wholly considered system, it is opportune that the benchmarking turns
(and in this sense has remarkable utility) also and above all to the different
levels of the organization, that is to say where the strategic decisions express,
or should express, a real change of the managerial behaviour. Yet, since the
benchmarking needs information about the performance of a certain moment of
the organization, the exchange of information with those who held the “best
practice” and the following implementation of changes happen generally at
the individual level, or in the context of a specific service giving performances.
From it comes that, the information on the performance and on the necessary
changes, are available also for high management, but they revert only in a
limited field of activity. The exchange of information is in fact certainly
of main importance to build an effective program of benchmarking. The distributors
of performances are required to communicate with the organizations that hold
the best practice, sharing information about the processes and the routines
that lead to a higher performance; but this doesn’t always happen.
Another occasion of “communication” are congresses, where it could be possible
to exchange information effectively and directly, though it is common to observe
that such information are often spoil with a certain opportunism, marked from
the fact that not infrequently only good results are underlined, while specific
gaps are disguised, by an interpretation of the data which is opportunely
manipulated.
Actually, inaccurate or incomplete information are not necessarily always
to be linked to the need to confer a positive image to one’s own activity,
if not, more expressly, to hide criticality cues; sometimes that happens even
for a kind of jealousy of one’s own work method, out of envy, etc.
The advantages of benchmarking
The
advantages of benchmarking, when effected in accordance with correct methodological
applications, are various (Cocconi): advantages in terms of costs (therefore,
efficiency, effectiveness, economics); advantages in terms of quality (continuous
improvement); advantages in terms of business culture (growth of the organization).
Benchmarking experiences in the sphere of competence of Health are reported
in rather a lot of scientific contributions.
About it, we remind the cost analysis of hospital management in the program
Medicare (Medicare Cost Report: see Magnus and Smith); or the benchmarking
for evaluation and the support of programs about pharmaceutical costs of the
“MeritCare Medical Center”, outfit from the Millard Fillmore Hospital (see
Murphy; Nelson). Jones underlined the importance and the contribution of the
activity of benchmarking aiming to a correct redistribution of the resources
in the British Health Service.
Daniels and coll. remind the “Benchmark of fairness” (impartiality) in the
context of the evaluation of the Health System reform in the United States;
mainly, it is underlined that < Fairness is a wide term that includes exposure
to risk factors, access to all forms of care, and to financing. It also includes
efficiency of management and resource allocation, accountability, and patient
and provider autonomy >; Daniels and coll. remind also that < The benchmarks
standardize the criteria for fairness >, with the possibility to be used at
all the levels, either national or local. Yurk and coll. recalled instead
the applications of the benchmarking, among which, beside the improvement
of the quality and the consequent satisfaction of the customer, there is also
a full participation in the business strategic planning. Dove and Greene,
let alone Homa-Lowry, underlined the role of the benchmarking in the context
of quality evaluation of the treatments, mainly its links to the increase
of costs and to the reduction of resources. John considers extremely relevant
that the benchmarking is used in Public Health in the evaluation of the treatments’
outcome (“Therapy Outcome Measure”), but also in the diffusion and sharing
of information. Ellis look out upon a punctual and wider application of the
benchmarking process to “Evidence Based Medicine”. Weissman and coll. imported
the concept of ABC (“Achievable Benchmark of Care”), or the feasible benchmarking
in health treatments, characterised by the following three main connotations:
the benchmarking expresses a measurable level of efficiency; the benchmarking
can be achieved in accordance with provable procedures; the benchmarking is
the result of objective data, reproducible and predetermined. Nevertheless,
Weissman and coll. underline that suppliers of high performance are selected
in a way to define beforehand a level of excellence; yet, the same suppliers
– even offering an elevated performance – are not able to influence appreciably
the levels of a specific benchmark, if they have modest case histories available.
In the wide bibliography on the theme, it finally deserves to be mentioned
the “2001 Benchmarking Guide” (Hoppszallern), manual that summarizes evaluation
methods of financial, technical-professional and marketing order for any hospital
structure. In particular, it highlights positive and negative aspects, in
the comparison among hospitals with absolutely different management level.
We finally recall our attention (see Beretta) on ethically wrong behaviours,
that is when the comparison of the data is actually used to try to hide the
aims of costs reduction (“downsizing”), rather than to regain the efficiency,
effectiveness and economics of the health care performances.
Conclusions
In spite of the negative aspects recalled, it is our belief that a progressive introduction of the benchmarking in Public Health, though demanding an initial and intense effort to implement the methodology, can give very interesting results, with remarkable advantages aimed to a real process of marketing change (see table 5). Mainly, even to overcome concrete problems, previous to the benchmarking procedure and usually tightly related to a certain resistance to reveal the real productivity data and the habitually adopted procedures, it is desirable the introduction of forms of anonymous association, as, for example, the “benchmarking network”, managed by an organization (benchmarking club), that shares information and, if necessary, also the elaboration and the general interpretation of the data, without yet revealing the origin, suggesting indicators, standards and quality improvement procedures. Perhaps, to some the anonymity appears a little weird … but probably the effectiveness may broadly justify it. (traduzione Interpres- Giussano)
Mauro Martini
Direttore del Nucleo di Valutazione
dell’Azienda Sanitaria
di Ferrara (Italia)
|
ABSTRACT
|