i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 79(2010)e 43–e
57
j o u r n a l h o m e p a g e :w w w.i n t l.e l s e v i e r h e a l t h.c o m /j o u r n a l s /i j m
i
Example of a Human Factors Engineering approach to a medication administration work system:Potential impact on patient safety
Marie-Catherine Beuscart-Zéphir a ,?,Sylvia Pelayo a ,b ,Stéphanie Bernonville a ,b
a EVALAB,CIC-IT Lille et EA 2694,Facultéde médecine,1place de Verdun,59045Lille,France
b
LAMIH-PERCOTEC et RAIHM,UMR CNRS 8530,UVHC,Le Mont Houy,59313Valenciennes Cedex 9,France
a r t i c l e
i n f o Article history:
Received 4April 2008Received in revised form 25June 2009
Accepted 18July 2009
Keywords:Human Factors
Medication administration Patient safety Usability CPOE
a b s t r a c t
Objective:The objectives of this paper are:
1.To describe a Human Factors Engineering (HFE)approach to a medication administration
work system,in the context of a hospital medication Computerized Provider Order Entry (CPOE)project.
2.To identify the determinants of this work system potentially impacting both the ef?-ciency and the safety of the medication use process.
In this approach,the implementation of such a complex IT solution is considered a major redesign of the work system.The paper describes the Human Factor (HF)tasks embedded in the project lifecycle:(1)analysis and modelling of the current work system and usability assessment of the medication CPOE solution;(2)HF recommendations for work re-design and usability recommendations for IT system re-engineering both aiming at a safer and more ef?cient work situation.
Methods:Standard ethnographic methods were used to support the analysis of the current work system and work situations,coupled with cognitive task analysis methods and docu-ments https://www.doczj.com/doc/2011850153.html,ability inspection (heuristic evaluation)and both in-lab (simulated tasks)and on-site (real tasks)usability tests were performed for the evaluation of the CPOE can-didate.Adapted software engineering models were used in combination with usual textual descriptions,tasks models and mock-ups to support the recommendations for work and product re-design.
Results:The analysis of the work situations identi?ed different work organisations and pro-cedures across the hospital’s departments.The most important differences concerned the doctor–nurse communications and cooperation modes and the procedures for preparing and administering the medications.The assessment of the medication CPOE functions uncov-ered a number of usability problems including severe ones leading to impossible to detect or to catch errors.
Models of the actual and possible distribution of tasks and roles were used to support decision making in the work design process.The results of the usability assessment were translated into requirements to support the necessary re-engineering of the IT application.Conclusion:The HFE approach to medication CPOE ef?ciently identi?es and distinguishes currently unsafe or uncomfortable work situations that could obviously bene?t from an
?
Corresponding author .
E-mail address:mcbeuscart@univ-lille2.fr (M.-C.Beuscart-Zéphir).
1386-5056/$–see front matter ?2009Elsevier Ireland Ltd.All rights reserved.doi:10.1016/j.ijmedinf.2009.07.002
e44i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s79(2010)e43–e57
IT solution from other work situations incorporating ef?cient work procedures that might be
impaired by the implementation of the CPOE.In this context,a careful redesign of the work
situation and of the entire work system is necessary to actually bene?t from the instal-
lation of the product in terms of patient safety and human performances.In parallel,a
usability assessment of the product to be implemented is mandatory to identify potentially
dangerous usability?aws and to?x them before the installation.
?2009Elsevier Ireland Ltd.All rights reserved.
1.Introduction
In the healthcare domain,the last decade has been charac-terized by a growing concern over patient safety[1,2].In an era where investigative techniques and treatments are becom-ing ever more powerful,adverse events and medical errors are increasingly threatening to the patients’welfare.Unfor-tunately,it is also clear that a signi?cant number of adverse effects actually reach patients,with possible deadly conse-quences[3].We also know that most of these adverse events are preventable[4,5].This has led to considerable effort to ren-der the health care process more secure[6]and to improve its quality and ef?ciency.
In this context,the Human Factors Engineering(HFE) approach to analysis,evaluation and redesign of the health-care work systems has proven ef?cient[7].This approach allows identifying the determinants of the work system that make the situation potentially dangerous.It also provides recommendations to optimize this work system,on organi-sational,cognitive and technical levels.
This paper presents a case study of a Human Factors Engi-neering approach to a medication administration process in a large academic hospital,in the context of a medication Computerized Provider Order Entry(CPOE)project.The study focuses on the nurses’tasks of preparing and administering oral route drugs to the patients,with a particular attention to the nurses’needs in terms of information necessary to ef?-ciently and safely support their tasks.
2.Rationale and theoretical background
2.1.Rationale:the HFE approach
The Human Factors Engineering framework(Fig.1)provides structured methods and tasks to achieve the optimisation of the work system and to inform its re-design:human well-being,usability of the products or work devices,overall work performance and safety of the care process[8].
The?rst task of the HFE approach is the analysis of the work system.It requires the understanding,description,analysis and if possible modelling of the work situation.The descrip-tion of the work situation identi?es who(people,user)does what(tasks),how(technology),in which context(environment, care process)and under which constraints(regulations).This analysis issues a description of the work situations along with a list of diagnosed problems from the Human Factors point of view,and proposes recommendations to?x these problems or at least mitigate their potential negative impact.
When a new IT system is envisioned,the second step of the HFE approach is the usability evaluation of this system.This evaluation identi?es general usability problems as well as speci?c ones,i.e.problems that would affect the work sys-tem under consideration.The usability evaluation results in recommendations for?xing the problems.
The recommendations must be reconciled with the insti-tution/designers/developers capabilities,leading to the third step,the cooperative design or re-design of the expected,re-engineered work situation featuring the new product/IT application.This design phase should generate a model of the re-engineered work situation,incorporating organisational and usability goals that can be translated,as far as possi-ble,into detailed requirements for the future product/work situation.
As soon as early prototypes or advanced mock-ups are available,or as soon as pilot sites start functioning,the last phase of iterative evaluation starts,that aims at identifying dis-crepancies between the expected work system or product and the observed ones.Human Factors or usability problems are identi?ed and reported,along with suggestions for?xing the problems.When the new work system meets all the HF and usability requirements,the product may be released and/or the new organisation implemented throughout the institu-tion.
2.2.Theoretical background:cognitive models
Within the HFE framework,the theoretical background sup-porting the analysis and modelling of the work system is a key point.Where patient safety is concerned,the analysis of the work situation also aims at identifying error prone factors and should therefore incorporate cognitive models of informa-tion processing,decision making and human errors as well as taxonomies of errors[9].
In the healthcare domain,the work situation presents some remarkable features,the most important ones being the importance of cognitive factors and the continuous evolution of this situation.
?Importance of cognitive factors.A great deal of healthcare professionals’activities comes down to{medical,clinical, physiological,biological,psycho-sociological}information management,https://www.doczj.com/doc/2011850153.html,rmation gathering,processing,selec-tion,interpretation,documentation,and transmission.This information management in turn is the foundation for deci-sion making and for the planning and execution of the care process.Those cognitive activities are therefore of partic-ular importance when tracking error prone factors in the healthcare work situation.Mishaps,?aws and problems in the information accessibility and display may lead to med-ical errors both in the form of slips(application of correct procedures or reasoning on incomplete or false informa-
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s79(2010)e43–e57
e45
Fig.1–The Human Factors Engineering framework for healthcare situations.
tion)and mistakes(inability to search for obviously missing information or to rectify incorrect/incomplete information)
[9].Moreover,the necessary information is often distributed
across different documents,different colleagues,different locations,etc.,therefore calling for a distributed cognition approach[10–12].
?Continuous evolution or dynamic aspect of the work situation.
From a cognitive point of view,the patient represents a dynamic system or process whose variables are functions of time and whose physiological status inevitably evolves, partly in reaction to the treatment applied by the healthcare team[13].Therefore,the care process and the correspond-ing work situation are dynamic.The cognitive analysis of healthcare professionals’activities should therefore focus on the cognitive dimensions of the management and con-trol of dynamic environments[13,14].In such a“Process Control”approach,the physician is considered as a sys-tem supervisor:he is in charge of making the decisions and elaborating the plan to maintain the values of the patient’s variables within an acceptable range.The nurse is charac-terized as a system controller:he/she is the one who acts on the patient when he/she implements the physician’s plan and translates it into operation and procedures.For exam-ple,in the medication process,the physician makes the therapeutic decision and the nurse is in charge of admin-istering the medication to the patient.In this dynamic environment where the process evolves continuously,both on its own and under the impact of the healthcare worker’s actions,at each encounter with the patient,the physicians and nurses need a display of relevant information in order to support and update their current representation of the situation.3.Methods
This section lists the methods used in each step of the HFE framework,i.e.(i)analysis of the work system,(ii)usability assessment of the candidate application,and(iii)coopera-tive re-design and re-engineering of the work system and of the application.Then the context of the case-study used to illustrate the HFE approach is presented.
3.1.Methods for the analysis of the work system
In the context of the project described here,a systematic qualitative analysis of the medication ordering–dispensing–administration process has been per-formed in several departments of(i)the academic hospital handling the CPOE project,i.e.Neurosurgery,Nephrology and Cardiology(ii)two hospitals already equipped with a CPOE system,i.e.Cardiology(one department in each hospital), Infectious disease,Nephrology and Immunology.In total,8 different departments were investigated.In this paper,the analysis focuses on the nurses’medication administration tasks.An analysis of nurses’tasks and activities,focusing on their cognitive and collective(communication)aspects has been carried https://www.doczj.com/doc/2011850153.html,anisational contexts and work routines were also recorded.To perform the observation and analysis of nurses’activities,we used standard methods from cognitive psychology and ergonomics:
?Semi-structured and structured interviews of target users.?Naturalistic observations supported by handwritten time-stamped detailed?eld notes focused on nurses’tasks in the
e46i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s79(2010)e43–e57
medication process,on physician–nurse and nurse–nurse dialogues about medication and on nurses’interactions with patient records.
?Document review and chart review:all the paper?les of the patients present on a given day were selected and copies of the physicians’medication order sheets and of the nurses’medication administration records were made.Over7000 orders were analyzed,including3883oral routes orders.?Debrie?ng interviews:nurses were presented with the results of observations and documents review performed in their departments and were asked to comment on and mentally replay the processes involved in the reading and interpretation of orders for the preparation of pill dispensers and for the administration itself.
?Software engineering models were elaborated to model the distribution of tasks observed in the different departments.
3.2.Methods for the usability evaluation of the
candidate medication CPOE system
Usual methods of usability assessment were used:?Usability inspection(heuristic evaluation).Usability inspec-
tion[15]aims at?nding usability problems in a given application(HCI)and then using these problems to make recommendations for?xing the problems.Five independent evaluators inspected the application’s Graphic User Inter-face(GUI)according to a set of ergonomic criteria[16]and draw up a list of usability problems.These problems were rated for their severity when considering the characteristics of the target users’activity on a four point scale.?Usability tests.We performed in-lab usability tests focused on the preparation of the pill dispensers and the valida-tion and documentation of administration.The nurses were asked to think aloud and the session was recorded using the Evalab usability lab.The experimental room was equipped to reproduce the usual working environment of the nurses.
The scenarios for the tests were elaborated from the results of the analysis of the work situation and in cooperation with the project managers,the users’representatives and the Company representatives.Eight Nurses went through the tests,after a short training on the system(1h)delivered individually the week before.
3.3.Methods for the cooperative re-design and
re-engineering of the work system/IT application
The results of the analysis of the work situation and of the usability assessment of the candidate application were pre-sented to the project group in order to prioritize the problems and to reach a consensus concerning the requirements for the expected work system.
Starting form this list of problems,the?rst step of re-design of the IT application listed all possible solutions for the identi?ed usability problems along with the assessment of cost–bene?t ratio of these solutions.Once a solution was selected,the designers of the CPOE system could propose mock-ups or early prototypes of the future re-engineered system.These mock-ups and prototypes underwent iterative usability evaluations issuing adapted recommendations sup-ported by software engineering models such as UML and Petri Nets[17].These iterative evaluations were run until all usabil-ity goals were met.
3.4.Context of the case study
The Centre Hospitalier Universitaire of Lille(CHU Lille)in the North of France is a3000-bed-capacity hospital.This academic hospital is equipped with a Hospital Information System(HIS)integrating over80different applications,but no medication CPOE facility,nor speci?c nurses’documen-tation or other support functions such as nursing plan or medication administration record(MAR),which are still paper-based.Moreover,the current HIS is aging and needs to be replaced.The CHU Lille has a cooperation agreement with a Company(Syndicat Interhospitalier de Bretagne,SIB)imple-menting a complete Hospital/Clinical Information System (Sillage?)including a CPOE system able to support all medica-tion ordering–dispensing–administration tasks.The CHU Lille has been incorporating the Human Factor tasks in its projects management for seven years.In the present project,the CHU Lille and the SIB Company concurrently ordered and partici-pated in a HF preliminary study to:
?assess the state of preparedness of the CHU departments before the medication CPOE installation;
?assess the usability of the medication CPOE functions of the system;
?optimize both the IT system and the organisation of the medical departments in order to secure and?nancially opti-mize the medication process.
4.Results:example of the application of
the HFE approach to a medication
administration work system
4.1.Results of the analysis of the work system
For illustration purposes,we present here some excerpts of the description of the work system that was delivered to the project group.The work system is described successively on three hierarchical levels:
1.At the organisational level:the highest level,to identify and
model the distribution of tasks across the different health-care professionals along with the work?ows.
2.At the collective level:the middle one,to describe the main
verbal transmissions and communications supporting the cooperative management of the medication administra-tion process.
3.At the individual level:the lower level,to describe how each
nurse interacts with the technical environment to perform the administration tasks.
https://www.doczj.com/doc/2011850153.html,anisational level
According to national and local regulations,the tasks neces-sary to carry out the medication ordering and administration procedures are distributed as follows.The physician is in charge of the therapeutic decision making and of ordering the
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s79(2010)e43–e57
e47
Fig.2–Pill dispensers are composed of four compartments, one for each of the four times of administration usually scheduled in the ward:morning,noon,evening,and bedtime.A tag with the Patient’s ID and corresponding bar code is af?xed to the pill dispenser.
meds.He is supposed to write the prescription and to date and sign it.The nurse has no medication ordering rights except for a small number of usual drugs.She/he is not supposed to copy the physician’s orders on any support except to validate the administration.The nurse has to check the meds and the related prescription before administering them to the patient, and she must validate(date and signature)the administra-tion.
The organisation of the administration of oral route medi-cations is structured by the preparation of24h pill dispensers (PDs).PDs are used in most of the departments of French and European hospitals;they are described in Fig.2.
At some point in the24-h period a nurse prepares the pill dispensers for all the patients of the ward.To perform this preparation,the nurse relies on the information contained in each patient’s medication orders list as prescribed by the physician.This preparation takes place in the room where the ward medication cabinet is located,usually the nursing room. Each dispenser is identi?ed by a room number and some-times also with the patient’s name.At administration times, the nurse veri?es the prepared medications by checking them against the order list or the MAR,takes the pill dispenser or the meds to the patient’s room,administers the meds to the patient,validates the administration on the MAR,and even-tually documents any abnormalities.
During the24-h period covered by the pill dispensers,the physicians visit the patients and place new orders or modify the existing patients’treatments.These modi?cations require an update of the corresponding pill dispensers by the nurse. This update is executed as soon as the nurse gets a modi?ed medication orders list.Over this24h time period,different nurses are in charge of administering the meds to the patients. This makes the whole procedure or work?ow more complex, as described in Fig.3
As indicated in Fig.3,the pill dispensers are a source of information for the nurses.When a compartment is empty, this means that the corresponding administration has been done.Unfortunately,there is no easy way for the nurse to know for sure whether a pill dispenser has been updated or not.This kind of information is generally transmitted verbally between nurses.
4.1.2.Collective level(verbal communications)
Given the organisation described above,the doctor–nurse and nurse–nurse written and verbal communications appear of critical importance[18–21].The section below describes the main types of organisation of the doctor–nurse verbal com-munications.
Three main types of organisation could be identi?ed:
https://www.doczj.com/doc/2011850153.html,mon rounds:Nurses participate in the medical rounds
with the physicians.The therapeutic plan may be nego-tiated between the physician,the nurses and eventually the patient,thus anticipating on possible administration dif?culties[20].
2.Brie?ngs:Doctors and nurses participate in short daily
meetings where they systematically review the patients’cases and the corresponding therapeutic plans.
3.Opportunistic:There is no time slot dedicated to
doctor–nurse verbal exchanges.As a consequence, verbal communications are rare and opportunist,leading to negative consequences for nurses.In particular,writ-ten orders contain a lot of implicit information some of them being dif?cult to interpret even by expert nurses.
The nurses’control task is more dif?cult and dangerous when they do not understand the objectives of unusual or modi?ed orders.
4.1.3.Individual level:interactions with the paper-based work environment
The main document used to prepare and administer the med-ications is the list of orders documented by the physicians.
The physicians document the medication order sheets as follow:on the?rst day they document the entire list of orders, but on the following days,they document only the modi?-cations to this initial list,i.e.changes in dosing:“increase Previscan?to1tablet a day”,suspension of a drug,introduc-tion of a new drug,etc.As a consequence,after a few days, the list of orders becomes in fact a list of modi?cations of orders.
This list of orders/modi?cation of orders partially?ts the nurse’s needs for the task of updating the pill dispensers because it displays primarily the last modi?cations.But it is a weak support for other tasks of preparation of PD and administration because of the lack of overview of the cur-rent treatment.After a few days,the nurse has to track back the information in previous pages to make sure he/she has the entire list of the meds the patient is currently on.In the departments where no other document is available,nurses frequently have to ask the physicians to write a recapitulation of the current treatment.From the cognitive point of view,this situation constrains the nurse in charge of preparation of PD to perform a double task:to mentally reconstruct the full list of active orders and to simultaneously?ll the PD therefore augmenting the risk of forgetting a medication.Nurses also point out that when a speci?c medication is no longer listed in a recapitulation,they never know for sure whether this drug was voluntarily suspended or if the physician himself forgot to write it.
e48
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 79(2010)e 43–e
57
Fig.3–T ypical organisation of the medication preparation and administration work?ow distributed over several nurses and several shift.In this observed example,the night shift nurse (N1)starts her shift by administering the “bedtime”meds:
after this last administration of the day,the pill dispensers are empty.She is then in charge of preparing the pill dispensers,which include four compartments for morning,noon,evening and bedtime administrations.The ?rst nurse of the morning shift (N2)takes care of the ?rst administration round of the day,after which the physician performs his medical round issuing modi?cations of existing orders or new orders.The same morning nurse accordingly updates the pill dispensers.Then the second morning nurse (N3)takes care of the second administration round around noon.In the afternoon,a physician performs a rapid follow-up of the patients,again issuing some modi?cations and new medication orders.The afternoon nurse (N4)takes care of the update of the pill dispensers,and then performs the evening administration round.
4.2.Interpretation of the results of the analysis of the work situation
The HFE analysis allows identifying the main determinants of the work system that impact both the ef?ciency and the safety of the medication administration process.
https://www.doczj.com/doc/2011850153.html,anisation of the dispensing
The organisation of the preparation and administration of drugs by the nurses is conditioned by the current hospital organisation of the pharmacy dispensing where about 70%of the drugs are “globally”dispensed for all the patients in the ward.In this organisation,the control of the administration task is distributed between
?several steps:preparation of PDs,update of PD,administra-tion itself;
?several actors,i.e.different nurses and different physicians;?several shifts (3–4shifts per 24-h periods).
4.2.2.Cooperation and communication procedures
The procedures of cooperation and communication between the nurses (horizontal,functional cooperation)and with the doctors (vertical,hierarchical cooperation)appear to be of critical importance.There is a strong interaction between the verbal transmission and the information documented on technical supports (paper sheets,CPOE systems).Verbal transmissions tend to complete the documented information and provide clues for its proper interpretation.In the depart-ments organised with common rounds and brie?ngs,the doctor–nurse communication is ef?cient:these organisations ef?ciently support the adjustment of each participant’s repre-sentation of the situation and reinforce their shared mental model of the patient’s case and of the therapeutic plan.Con-versely,the lack of systematic organisation of doctor–nurse transmission of information in the “opportunistic”organi-sation must be considered inef?cient and dangerous.When verbal communications are de?cient,even more detailed doc-umented information proves unable to catch up with the lack of verbal exchanges,particularly when it comes to the under-
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s79(2010)e43–e57e49 Table1–Nurses information needs for preparation and update of PD and administration tasks.For each task,it is important that the nurse be sure that she/he is dealing with the right patient.Therefore the?rst fundamental information need is the name of the patient,clearly highlighted on each page of each document.Given that the pill dispensers are generally identi?ed by room/bed number,this information is useful besides the patient’s name for control purposes. Preparation of PD Update of PD Administration to each patient
Highlighting of each patient’s identity (and eventually room/bed number)Highlighting of each patient’s identity
(and eventually room/bed number)
Highlighting of patient’s identity(and
eventually room/bed number)
Overview of each patient’s orders(current treatment)Overview of each patient’s orders(current
treatment)with highlighting of recent
changes and modi?cations
Overview of the patient’s treatment(at
bedside)
Detailed view of each order
Physician’s intention/therapeutic objective(s)motivating the order, especially if this order is:Physician’s intention/therapeutic
objective(s)motivating the order,
especially if this order is:
Physician’s intention/therapeutic
objective(s)motivating the order,
especially if this order is:
?New?New?New ?Unusual?Unusual?Unusual ?Recently modi?ed?Recently modi?ed?Recently modi?ed
Detailed view of each order Display of orders zoomed on the time
period starting at the last
modi?cation/update to the next
preparation of PD Display by administration moments or by precise time(h)
Easy shift from one patient’s list of orders to the next one’s(following room numbering order)Easy shift from one patient’s list of orders to the next one’s(following room numbering order)
At each step of the process and for each subtask,the nurse has to verify the accuracy of the relationship between the patient’s identity,the physician’s orders and the actual medications prepared or to be administered.This control is made much easier and more ef?cient if the nurse has access,on an operational level,to the physician’s intentions or therapeutic objectives underlying new or recently modi?ed orders or unusual prescriptions.
The updates of pill dispensers(PD)are also easier if the medication orders can be focused on the relevant time period,i.e.starting at the time of the last update to the end of the24-h period covered by the PD.As for the administration task,the same information need requires a display of orders focused on the administration time currently under execution,allowing the nurse to know for example what the patient’s meds are at noon or at bedtime administration.
standing of physicians’intentions or objectives underlying unusual prescriptions.
4.2.3.Dynamic aspects of the care process
The continuous evolution or dynamic characteristics of the situation is another important determinant of the work sys-tem.The patients’clinical and physiological status ineluctably evolves.In accordance with this the physicians adapt,mod-ify and update their medication orders.The frequency and magnitude of these modi?cations depend on the speed of this clinical and physiological process,i.e.of the type of pathology and its severity.In order to ef?ciently and safely handle the changes engendered by the continuous changes of prescrip-tions,the nurses need to continuously update their current representation of the patients’care plan and therapeutic plan. The risks of error are more likely to occur in departments char-acterized by a high speed of the care process issuing frequent changes in prescriptions.
4.2.4.Distributed cognition
The medication preparation and administration work?ow is characterized by a distributed cognition model.The main sources of relevant information are the technical artifacts (paper sheets or computer screens),the verbal communica-tions with doctors and other nurses,and the status of some objects such as the PDs.The work system tends toward a kind of equilibrium between documented and verbal infor-mation[20].Ef?cient verbal communications can partially mitigate the negative impact of inadequate documents.On the contrary,the departments suffering from both poor doc-umentation and inef?cient or inexistent verbal exchanges are dangerous work systems generating a high risk of errors. In these situations,the nurses rapidly become unsecured and their level of stress increases,along with the probability of errors in PD preparation or in medication administra-tion.
Indeed the observed paper sheets are far from perfect from the perspectives of both ef?ciency and safety dimensions. While it is supposed to be the only authorized document, the list of orders is probably the worst possible support for the preparation of the pill dispensers.In this situation,when the nurses comply with local/national regulations forbidding them to copy the orders on unauthorized supports,the prob-ability of errors increases.The main risks of errors are:(1) to forget a drug,(2)to misread or misinterpret the name or the dose of a drug,and(3)to overlook a modi?cation of the patient’s treatment.
4.3.Recommendations for the future work system [excerpts]
(1)The hospital is characterized by a great variety of organ-
isations of the medication ordering,dispensing and administration tasks.Some of these organisations are
e50i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s79(2010)e43–e57
neither optimally ef?cient nor safe.It is important to char-acterize the current departments’organisations in order to identify for each department the requirements for the optimisation of its work system and to anticipate the impact of the installation of the CPOE system.
(2)The current distribution of tasks between physicians and
nurses depends heavily on the organisation of the dispens-ing process.The project group should check and anticipate on any major planned evolution of this organisation for the entire hospital.
(3)Given that this organisation remains as is,the future sys-
tem should provide the nurses with speci?c information at each step of the preparation and administration of drugs.
Nurses’information needs are summarized in Table1. (4)The system should provide different functions for the
preparation–documentation–validation of the PD and for the documentation–validation of the administration itself.
(5)All departments should secure a minimal organisation of
doctor–nurse communications about the patients’treat-ment and its changes,in the form of a daily brie?ng just before and/or after the physician’s medical round.
(6)The departments characterized by the“common rounds”
organisation require a speci?c CPOE con?guration and work system organisation supporting the doctor–nurse synchronous communications in the patient room.
4.4.Results of the usability evaluation of the administration functions of the CPOE candidate
The CPOE system offers a number of functions to support the nurses’tasks of medication preparation and administration. Most of the relevant information can be displayed on the com-puter screen,but a lot of printing facilities are also offered to make up with the lack of mobiles devices(https://www.doczj.com/doc/2011850153.html,puters on wheels or tablet PCs)in most of the hospitals.
4.4.1.List of functions of the CPOE system supporting the nurses’tasks
4.4.1.1.Display of the list of patients.This list includes two interesting indicators:
?P/NP(Print/No Print)tells the nurse whether an adminis-tration plan has been printed or not since the last time the physician modi?ed the patient’s medication orders.
?AV(Afternoon Visit)indicates whether the physician has modi?ed the patient’s orders list during his afternoon (evening,night)follow-up visit.
4.4.1.2.Display of the medication orders(Fig.4).The nurse can access this physician screen which indicates whether orders have been modi?ed and when.
4.4.1.3.Preparation plan.The CPOE system provides a speci?c preparation plan to support the pill dispensers’preparation. However the system provides this facility only in a“print”version.The nurse can select the patients,the time period cov-ered by the preparation,the type of drugs(all drugs,oral route only,IV route only,etc.).The preparation plan allows a display by administration times.For each patient,recently modi?ed drugs are?agged.If several patients have been selected,they are displayed in alphabetic order.
4.4.1.4.Administration plan acting as medication administra-tion record(Fig.5).The administration plan displays the drugs according to the time for administration.The nurse can choose to display only those patients for which orders have been placed or modi?ed in the last24-h period.
The CPOE functions apparently?t most of the nurses’infor-mation needs.One of the good points is the separation of the administration plan(i.e.the electronic equivalent to the MAR) from the preparation plan.Recent modi?cations are high-lighted in the preparation plan(but not in the administration plan and MAR),and the nurse can display the patient’s drugs according to relevant criteria(time for administration,route).
We performed an evaluation to check the actual usability of these functions.
4.4.2.Results of the usability inspection
The?ve evaluators identi?ed35problems with the nurses’user interface(see Fig.6).
T wo problems were rated at the maximum severity level (Severity4)and eight problems were rated level3.The most important problems were due to the lack of“compatibility”of the system when considering the nurses activities,partic-ularly the documentation of the administration plan(i.e.the MAR).For example,the system offered the possibility for the nurse to enter notes to document administration problems such as“the patient could not swallow his oral route meds”but documenting such a note would automatically validate the administration.Similarly,there were problems when doc-umenting a change of dose or a change of administration time (the initial prescribed dose or time would disappear).Prob-lems were also identi?ed with the display of information:the route for administration was not mentioned in the adminis-tration plan,the display of the list of orders would change haphazardly after some administration validation and the list of patients in the preparation plan was organized following alphabetic order only,and so on.
Additional observation of the work situation in a hospital using the system con?rmed the results of the usability inspec-tion.For example the nurses would not use the system’s notes to document administration problems(“It does not work”)but rather rely on verbal communications or paper notes.Such usability problems signi?cantly diminish the“ease of use”of the system.
All the problems were carefully documented,along with screen shots when necessary,and recommendations were provided to?x the problems,sometimes supported by mock-ups.
4.4.3.Results of the usability tests
The nurses experienced28usability problems during the tests. Table2presents the most important ones,i.e.the problems encountered by at least50%of the users.
Most of these problems were due to the lack of intuitivity or visibility of some buttons and icons or to an improper display of the information.Such usability problems would generate important dif?culties for the nurses starting to work with the system and signi?cantly deteriorate its“ease of learning”.
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s79(2010)e43–e57e51
Fig.4–Screenshot of the CPOE current display of the patient’s medication orders(partially translated).
Fig.5–Screenshot of the CPOE current administration plan.
e52
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 79(2010)e 43–e 57
Table 2–Results of the usability tests:list of problems encountered by 50%of the users and over.
Nurse 1
N2
N3
N4
N5
N6
N7
N8
Percentage of users verbalizing and/or encountering a problem
Scenario 1:visualisation of therapeutic modi?cations Dif?culties in accessing the prescriptions
X
X
X
X
50%
Dif?culties in noticing and visualizing the therapeutic modi?cations
X
X X X X 63%
Dif?culties in noticing and visualizing the termination of an order
O O X O X O 63%
Dif?culties in
closing the list of orders screen
X X X O X X X 75%
Scenario 2:validation of the administration Dif?culties in reading the care and administration plan
X O
X O
X O
X
50%
Dif?culties in ?nding the validation button
X O
X O X X X 63%
Dif?culties in modifying a dose X X X X X 63%Dif?culties in validating a perfusion
X O
X
X
X
X
X
75%
Scenario 3:preparation of the pills dispensers Cannot ?nd the icon to access the preparation plan
X
X
O
X
X O
X O
X
X O
100%
Dif?culties in printing the preparation plan
X O X O X O X 63%
Usability problems:the user fails to perform the required action (O)and/or verbalizes a comment on this problem (X).
Some problems were also potentially dangerous,for exam-ple when they would prevent the nurse from noticing recent changes or terminations of orders.
All the problems were documented and sometimes illus-trated with video excerpts.Recommendations were provided to the Company,along with mock-ups when necessary.
4.5.Re-engineering of the CPOE system
The persons in charge of the CPOE re-engineering in the Company participated in all the project meetings and were presented with the detailed results and interpretation of the analysis of the work system.This gave them the opportunity
Fig.6–Results of the usability inspection.
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 79(2010)e 43–e 57
e53
Fig.7–Display of the patient’s medication orders in the re-engineered system (mock-up,December 2007).
to better understand the healthcare professionals tasks and activities and the challenges of the CPOE implementation.
Following the ?nal report on usability problems identi?ed with the system,a cooperative re-engineering process was undertaken.Meetings were necessary to:
?give the Human Factor specialists the opportunity to explain in detail the nature of the problems from the usage and safety point of view;
?prioritize the re-engineering tasks depending on the sever-ity of the problems and the cost of the corresponding developments;
?agree on solution sketches.
During the re-engineering phase,an iterative “quick-and-dirty”evaluation was performed on mock-ups to check the usability of the solutions proposed and their ef?ciency in ?x-ing the usability problems they were addressing.At times,it was necessary to provide the developers excerpts of video from the usability tests to help them understand the exact nature of users’dif?culties with the system.The iterative evaluation proved time consuming (hence the choice of “quick &dirty”methods),and sometimes confusing for the ergonomists.The re-engineering process covered the whole system and not only the CPOE functions therefore leading to upper level design and technical choices having unex-
pected consequences on the CPOE functions and forcing the ergonomists to adjust the usability goals to the new context.
The re-engineering process went on for two years.In the end,all the usability problems have been ?xed,but no com-plete usability counter evaluation has been performed yet on the re-engineered system.
Figs.7and 8give examples of re-engineered GUI for the display of the list of orders (6)and the administration plan (7).
5.Discussion:bene?ts and limits of the HFE approach
In the Human Factors Engineering approach,the description of the existing work situations along with the results of the usability assessment of the candidate IT system are impor-tant and fruitful documents.They provide all the partners with a systematic HF based analysis of the work situations and of any candidate IT application that ef?ciently support a cooperative (re-)design of the future work system incorpo-rating the new technology.These documents provide a good basis for a common understanding of the current work system by all the partners (hospital and editor)and help them reach a consensus on the desired characteristics of the future work system including organisational aspects as well as usability features of the IT application.The gap between the current
e54
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 79(2010)e 43–e
57
Fig.8–New administration plan (MAR)in the re-engineered CPOE system.
organisation and the existing application on the one hand and the desired work system on the other hand are made explicit along with the possible strategies and necessary resources to reach these objectives.
The section below provides some examples of the potential bene?ts of the HFE approach to a healthcare IT project and of their corresponding limits or constraints.
5.1.Bene?ts for the healthcare institution and the software editor
The healthcare institution bene?ts from a structured descrip-tion of its existing work situations.It is possible to elaborate a framework identifying key variables and indicators to sup-port the characterisation of the departments’organisation throughout the hospital.Excerpts of such a framework elabo-rated in the context of the CPOE case study illustrated above are presented in Table 3.We can identify in this framework variables or combination of variables that might be error-prone,and therefore have to be dealt with during the re-design process of the work system.The HFE approach also makes it clear that the installation of a new IT system is a key opportunity to re-design the work system,and to purposefully evolve towards a more ef?cient and safer work system clearly described and modelled,and agreed upon by all the project partners and the hospital managers.
The involvement of users’representatives and end users in the project is also a key success factor in terms of users’acceptance.We also know that on the long term,the hospital bene?ciates of a usable product,which signi?cantly dimin-ishes the cost of training and the risk of users’rejection [8,22].
The Company developing and commercializing the soft-ware bene?ts from objective,qualitative and quantitative data issued from the usability assessment of its https://www.doczj.com/doc/2011850153.html,bined with the models of the work situations and the identi?cation of key factors for a good integration of the IT application in the healthcare professionals’activities,the usability results provide a very ef?cient basis for the re-engineering of the product aiming at its usability optimisation.Past experiences of system re-engineering have proven that in the long run,the Company can get important commercial bene?ts based on the acknowledgement of the good usability of its product.
5.2.Limits and constraints of the HFE approach
We can identify some limits pertaining to the HFE method itself.First,the observation,description analysis and mod-elling of the work system requires a double expertise,in ergonomics and in the healthcare domain.To be ef?cient,the HFE experts must also know the main features of the work system they are going to analyze,i.e.the healthcare institu-tions work systems.For the time being,this double expertise is still scarce resource.Secondly,the analysis of the work system
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s79(2010)e43–e57e55 Table3–Excerpts of the framework identifying key variables and descriptors to support the characterisation of the
organisation of the medication use process in the various departments of the hospital.
Speed of the care process
?T ype of department(ICU,surgery,medical specialty)
?Number of prescribing physicians(senior/junior),number of nurses,number of patients
?Daily frequency of physicians rounds/visits/follow-ups per patient
?Estimated number of times a physician places new orders or modi?es existing orders per day and per[category of]patient
?Etc.
Organisation of the dispensing
?Estimated proportion of drugs delivered in the ward on the basis of
?A unit dose dispensing
?A nominative dispensing
?A global dispensing
?Estimated proportion of injectable/oral route/other drugs
?Location(s)of drugs storage in the ward
?Nurses room
?Dedicated room
?Patients rooms
?Etc.
Organisation of administration
?Pills dispensers or not,or proportion of patients on PD
?Number of planned medication rounds;estimated proportion of drugs administered during these planned medication rounds
?Number of nurses shifts(see Fig.4)
?Estimated proportion of drugs prepared and administered with PD
?Etc.
Communications and cooperation(doctor–nurse)
?Common rounds(participants,time,duration,supports)
?Brie?ngs(participants,time,duration,supports)
?Opportunistic(estimated frequency,supports)
?Etc.
issues general descriptions identifying potentially dangerous organisational or cognitive variables that impact the entire hospital organisation.This high level,hospital wide analy-sis should be completed by more focused studies of actual risks or errors using either prospective analysis of poten-tial errors or risks such as HFMEA(Healthcare Failure Mode and Effect analysis)[23–25],or retrospective methods such as root cause analysis of past errors[26],or any method aiming at identifying actual Adverse Events in the hospital setting.
There may also be some limits to the redesign of the work system.When dealing with large organisations such as the3000bed Lille hospital and dif?cult projects such as medication CPOE systems that impact the entire organ-isation,the complexity of the work organisations and the diversity of the habits of work require matching complex and adapted solutions.The evolution of such large institu-tions is costly and dif?cult to achieve.In this context,the risk of the HFE approach is that the severity of the HF diag-nosis and the proper anticipation of the solutions required drive the managers to freeze and postpone the project. Indeed the re-design of the work system proves easier and faster in smaller institutions[27]or with more dedicated systems.
Finally there are also limits to the re-engineering of the IT applications.In the case study presented here,the results of the usability evaluation were well accepted by the Com-pany and the cooperative redesign and re-engineering of the CPOE system have been successfully completed.However the scope and available resources of the re-engineering project did not allow incorporating into the system really innovative functions such as:
?functions supporting a doctor–nurse or nurse–nurse collab-orative planning of the therapeutic plan[28];?functions supporting an ef?cient shared representation of the process care and its continuous update;
?functions able to capture the physician’s intentions when prescribing a drug.
Indeed,no commercially available system is currently equipped with such advanced functions that are still in the research domain.Therefore and for the time being,it is impor-tant to secure at the organisational level the doctor–nurse and nurse–nurse verbal communications,for they are the only way allowing the transmission of these important elements of knowledge.
6.Conclusion and perspectives
In the past?ve years we have completed a number of success-ful HFE-based healthcare IT projects,and the HFE approach is now routinely integrated in the IT project management
e56i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s79(2010)e43–e57
Summary points
What was known before the study?
?The Human Factors Engineering(HFE)approach is ef?-
cient to:
?Support the usability evaluation and re-engineering process of healthcare IT applications.
?Support the characterisation of work organisa-tions in healthcare allowing the identi?cation of
the determinants of this situation that potentially
impact patient safety.
?However there are few studies describing a complete HFE approach combining both dimensions.
?A number of Adverse Drug Events are due to errors at the administration stage.Most of the time,these
errors are attributed to faulty individual procedures,
thus underestimating the role of collective factors in
the generation of errors.
What has the study added to the body of knowledge??Our study describes a complete HFE approach to a
medication work system combining the usability opti-
misation of the IT solution and the identi?cation of key
organisational and cognitive factors for the re-design
of the work situation.
?The study identi?es important determinants of the medication administration work situation at the indi-
vidual,collective and organisational levels.Some
combinations of those determinants can make the sit-
uation either safe or error prone.
of the CHU Lille hospital.Other hospitals and other coun-tries have implemented similar methods or are trying to do so.It is now time to harmonise and standardise,as far as possible,the methods and models of the HFE approach to healthcare IT projects.This is precisely the objective of a group of researchers of this domain,supported by the IMIA WG on ‘Human Factors Engineering for Health Informatics’and the EFMI WG on‘Human and Organizational Factors for Medical Informatics’,who have started organising annual workshops on the subject.
The HFE-based re-design of healthcare work systems also seems promising to improve patient safety.But to achieve this goal,it is necessary to link the analysis of the existing system and its potential redesign to the actual identi?ca-tion of Adverse Events.We are currently trying to do so in the European project PSIP(Patient Safety through Intelligent Procedures in medication)that aims at semi automatically identifying Adverse Drug Events by searching large medical repositories using statistical data mining methods.The HFE module of this project links the analysis and modelling of the work systems to the identi?cation of ADE by statistical methods.
In this context,we also aim at elaborating and validating a cross-cultural framework to describe the work systems of the medication use process in the Danish and French hospitals.Such a framework would ef?ciently support the HF analysis and re-design of a large variety of work systems and would have the capacity to inform the design of usable medication functions of the CPOE system.
Acknowledgements
The research leading to these results has received funding from the French Ministry of Research within the RNTS projects (the Presc’Info project).We thank the healthcare professionals (physicians,nurses and head nurses)of the Denain General Hospital and the University Hospital of Lille for their close cooperation and clever insights.We also thank our industrial partner in the Presc’Info project,the Syndicat Interhospitalier de Bretagne for their openness and constructive participation.
r e f e r e n c e s
[1]R.Kaushal,K.G.Shojania,D.W.Bates,Effects of
computerized physician order entry and clinical decision
support systems on medication safety:a systematic review, Arch.Intern.Med.163(2003)1409–1416.
[2] B.Chaudhry,J.Wang,S.Wu,M.Maglione,W.Mojica,E.Roth,
S.C.Morton,P.G.Shekelle,Systematic review:impact of
health information technology on quality,ef?ciency,and
costs of medical care,Ann.Intern.Med.144(2006)742–
752.
[3]L.T.Kohn,J.M.Corrigan,M.S.Donaldson,To Err is Human,
National academy Press,Washington,DC,1999.
[4] D.W.Bates,L.L.Leape,D.J.Cullen,https://www.doczj.com/doc/2011850153.html,ird,L.A.Petersen,J.M.
Teich,E.Burdick,M.Hickey,S.Klee?eld,B.Shea,M.Vander
Vliet,D.Seger,Effect of Computerized Physician Order Entry and a team intervention on prevention of serious
medication errors,J.Am.Med.Assoc.280(1998)1311–
1316.
[5] D.W.Bates,J.M.Teich,M.Lee,D.Seger,G.J.Kuperman,N.
Ma’Luf,D.Boyle,L.Leape,The impact of Computerized
Physician Order Entry on medication error prevention,J.Am.
https://www.doczj.com/doc/2011850153.html,rm.Assoc.6(1999)313–321.
[6] D.W.Bates,R.S.Evans,H.Murff,P.D.Stetson,L.Pizziferri,G.
Hripcsak,Detecting Adverse Drug Events using information technology,https://www.doczj.com/doc/2011850153.html,rm.Assoc.10(2003)115–128. [7]P.Carayon,Human factors and ergonomics in health care
and patient safety,in:P.Carayon(Ed.),Handbook of Human Factors and Ergonomics in Health Care and Patient Safety,
Lawrence Erlbaum Associates,New Jersey,2007,pp.3–21. [8]M.C.Beuscart-Zephir,P.Elkin,S.Pelayo,R.Beuscart,The
human factors engineering approach to biomedical
informatics projects:state of the art,results,bene?ts and
challenges,https://www.doczj.com/doc/2011850153.html,rm.(2007)109–127.
[9]J.Zhang,V.L.Patel,T.R.Johnson,E.H.Shortliffe,A cognitive
taxonomy of medical errors,https://www.doczj.com/doc/2011850153.html,rm.37(2004)
193–204.
[10]J.Zhang,D.A.Norman,Representation in distributed
cognitive tasks,Cogn.Sci.18(1994)87–122.
[11] B.Hazlehurst,P.N.Gorman,C.K.McMullen,Distributed
cognition:an alternative model of cognition for medical
informatics,https://www.doczj.com/doc/2011850153.html,rm.77(2008)226–234.
[12] E.Hutchins,Cognition the Wild,MIT Press,Cambridge,MA,
1995.
[13]J.M.Hoc,R.Amalberti,Diagnosis:some theoretical
questions raised by applied research,Curr.Psychol.Cogn.1 (1995)73–101.
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s79(2010)e43–e57e57
[14]Y.Xiao,Interacting with Complex Work Environments:A
Field Study and a Planning Model,University of Toronto,
Toronto,1994.
[15]J.Nielsen,R.L.Mack,Usabilty Inspection Methods,John
Wiley&Sons,New York,1994.
[16] C.Bastien,D.L.Scapin,Ergonomic Criteria for the Evaluation
of Human Computer Interfaces,Technical report156,1993
(INRIA Rocquencourt).
[17]S.Bernonville,C.Kolski,N.Leroy,M.C.Beuscart-Zephir,
Integrating the SE and HCI models in the human factors
engineering cycle for re-engineering Computerized
Physician Order Entry systems for medications:basic
principles illustrated by a case study,https://www.doczj.com/doc/2011850153.html,rm.79 (2010)e147–e154.
[18]S.Pelayo,D’une coopération verticaleàune plani?cation
coopérative des actions:le cas de la gestion des
prescriptions thérapeutiques hospitalières,University Lille2, 2007.
[19]M.C.Beuscart-Zephir,S.Pelayo,P.Degoulet,F.Anceaux,S.
Guerlinger,J.J.Meaux,A usability study of CPOE’s
medication administration functions:impact on
physician–nurse cooperation,Medinfo11(2004)1018–1022.
[20]M.C.Beuscart-Zephir,S.Pelayo,F.Anceaux,D.Maxwell,S.
Guerlinger,Cognitive analysis of physicians and nurses
cooperation in the medication ordering and administration process,https://www.doczj.com/doc/2011850153.html,rm.1(Suppl.76)(2007)S65–
S77.
[21]M.C.Beuscart-Zephir,S.Pelayo,F.Anceaux,J.J.Meaux,M.
Degroisse,P.Degoulet,Impact of CPOE on doctor–nurse
cooperation for the medication ordering and administration process,https://www.doczj.com/doc/2011850153.html,rm.74(2005)629–641.[22]M.C.Beuscart-Zephir,F.Anceaux,H.Menu,S.Guerlinger,L.
Watbled,F.Evrard,User-centred,multidimensional
assessment method of Clinical Information Systems:a
case-study in anaesthesiology,https://www.doczj.com/doc/2011850153.html,rm.74(2005) 179–189.
[23]R.Koppel,T.Wetterneck,J.L.Telles,B.T.Karsh,Workarounds
to barcode medication administration systems:their
occurrences,causes,and threats to patient safety,J.Am.
https://www.doczj.com/doc/2011850153.html,rm.Assoc.15(2008)408–423.
[24]P.Bonnabry,C.spont-Gros,D.Grauser,P.Casez,M.Despond,
D.Pugin,C.Rivara-Mangeat,M.Koch,M.Vial,A.Iten,C.
Lovis,A risk analysis method to evaluate the impact of a
computerized provider order entry system on patient safety, https://www.doczj.com/doc/2011850153.html,rm.Assoc.15(2008)453–460.
[25] E.W.Israelski,W.H.Muto,Human Factors risk management
in medical products,in:P.Carayon(Ed.),Handbook of
Human Factors and Ergonomics in Health Care and Patient Safety,Lawrence Erlbaum Associates,New Jersey,2007,pp.
615–649.
[26]K.B.Percarpio,B.V.Watts,W.B.Weeks,The effectiveness of
root cause analysis:what does the literature tell us?Jt.
Comm.J.Qual.Patient Saf.34(2008)391–398.
[27]M.C.Beuscart-Zephir,L.Watbled,A.M.Carpentier,M.
Degroisse,O.Alao,A rapid usability assessment
methodology to support the choice of clinical information
systems:a case study,in:Proc.AMIA Symp.,2002,pp.46–50.
[28] D.Ho,Y.Xiao,V.Vinay,P.Hu,Communication and
sense-making in intensive care:an observation study of
multi-disciplinary rounds to design computerized
supporting tools,AMIA(2007)329–333.