Volume 52, Issue 1 p. 113-123
Original Manuscript
Free Access

The Contribution of Staffing to Medication Administration Errors: A Text Mining Analysis of Incident Report Data

Marja Härkänen PhD, RN

Corresponding Author

Marja Härkänen PhD, RN

Post-doctoral researcher, Department of Nursing Science, University of Eastern Finland, Kuopio, Finland


Dr. Marja Härkänen, University of Eastern Finland: Department of Nursing Science; Academy of Finland, Yliopistoranta 1c, Kuopio, Finland.

E-mail: [email protected]

Search for more papers by this author
Katri Vehviläinen-Julkunen PhD, RN, RM

Katri Vehviläinen-Julkunen PhD, RN, RM

Professor, Department of Nursing Science, University of Eastern Finland, Kuopio University Hospital, Finland

Search for more papers by this author
Trevor Murrells BSc, MSc

Trevor Murrells BSc, MSc

Statistician (Nursing & Midwifery), King’s College London, Florence Nightingale Faculty of Nursing, Midwifery & Palliative Care, London, UK

Search for more papers by this author
Jussi Paananen PhD

Jussi Paananen PhD

Research manager, University of Eastern Finland, Institute of Biomedicine, Kuopio, Finland

Search for more papers by this author
Bryony D. Franklin PhD

Bryony D. Franklin PhD

Professor, Pharmacist, Imperial College Healthcare NHS Trust, UCL School of Pharmacy, London, UK

Search for more papers by this author
Anne M. Rafferty PhD, RN

Anne M. Rafferty PhD, RN

Professor, King’s College London, Florence Nightingale Faculty of Nursing, Midwifery & Palliative Care, London, UK

Search for more papers by this author
First published: 25 November 2019
Citations: 17



(a) To describe trigger terms that can be used to identify reports of inadequate staffing contributing to medication administration errors, (b) to identify such reports, (c) to compare the degree of harm within incidents with and without those triggers, and (d) to examine the association between the most commonly reported inadequate staffing trigger terms and the incidence of omission errors and “no harm” terms.

Design and Setting

This was a retrospective study using descriptive statistical analysis, text mining, and manual analysis of free text descriptions of medication administration–related incident reports (N = 72,390) reported to the National Reporting and Learning System for England and Wales in 2016.


Analysis included identifying terms indicating inadequate staffing (manual analysis), followed by text parsing, filtering, and concept linking (SAS Text Miner tool). IBM SPSS was used to describe the data, compare degree of harm for incidents with and without triggers, and to compare incidence of “omission errors” and “no harm” among the inadequate staffing trigger terms.


The most effective trigger terms for identifying inadequate staffing were “short staffing” (n = 81), “workload” (n = 80), and “extremely busy” (n = 51). There was significant variation in omission errors across inadequate staffing trigger terms (Fisher’s exact test = 44.11, p < .001), with those related to “workload” most likely to accompany a report of an omission, followed by terms that mention “staffing” and being “busy.” Prevalence of “no harm” did not vary statistically between the trigger terms (Fisher’s exact test = 11.45, p = 0.49), but the triggers “workload,” “staffing level,” “busy night,” and “busy unit” identified incidents with lower levels of “no harm” than for incidents overall.


Inadequate staffing levels, workload, and working in haste may increase the risk for omissions and other types of error, as well as for patient harm.

Clinical Relevance

This work lays the groundwork for creating automated text-analytical systems that could analyze incident reports in real time and flag or monitor staffing levels and related medication administration errors.

Medication administration is a routine nursing practice. However, medication administration errors (MAEs) are common (Keers, Williams, Cooke, & Ashcroft, 2013; McLeod, Barber, & Franklin, 2013). In this study, we use the term “error” as a synonym for the term “incident” to represent near misses as well as errors that reach the patient, whether they cause harm or not. Based on previous knowledge, dose omissions are amongst the most common MAE subtypes (Härkänen, Vehviläinen-Julkunen, Murrells, Rafferty, & Franklin, 2018; Keers et al., 2013). Staffing adequacy is one of the factors that may contribute to such missed care and adverse patient outcomes (Kalisch & Lee, 2010), since important nursing tasks are often left undone because of lack of time (Ausserhofer et al., 2014; Ball, Murrells, Rafferty, Morrow, & Griffiths, 2014; Ball et al., 2018). Inadequate staffing levels can arise for multiple reasons (e.g., cost reductions, sickness absences, and inability to fill vacant posts). These factors increase the workload elsewhere in the system, for example, when establishment staff have to supervise agency staff, resulting in reduced productivity.

Research on the relationship between nurse staffing and patient outcomes has demonstrated the important role that nurses play in the provision of safe and high-quality care (e.g., Aiken et al., 2014; Brennan, Daly, & Jones, 2013; Coster, Watkins, & Norman, 2018; Royal College of Nursing [RCN], 2017). Important nursing tasks were more likely to be left undone as the burdens upon staff and workload increased (Aiken et al., 2013, Ball et al., 2014; Ball et al., 2018). This can lead to adverse outcomes and hospital readmissions (Aiken et al., 2014; Ball et al., 2018; Brooks-Carthon, Lasater, Rearden, Holland, & Sloane, 2016; West et al., 2014), as well as reductions in productivity that place additional burdens on the organization both operationally and financially. Inadequate resources and staffing that contribute to heavy workload have also been shown to contribute to medication errors (e.g., Berdot et al., 2012; Lawton, Carruthers, Gardner, Wright, & McEachan, 2012; Seynaeve et al., 2011). Nursing work is often demanding and stressful (Kvist et al., 2013). Increased stress lowers job satisfaction, has consequences for healthcare workers’ mental and physical health, and precipitates burnout and job turnover (Kelly, Runge, & Spencer, 2015). The burden on the remaining staff increases and the cycle continues, with many posts left unfilled, particularly amongst registered nurses (RCN, 2017). This is all set against a backdrop of rising demand due to the increasing and aging population.

According to the report To Err Is Human (Kohn, Corrigan, & Donaldson, 2000), one of the key recommendations for learning and decreasing errors was for greater attention to be paid to incident reporting, with the primary purpose of facilitating learning, avoiding the same incidents recurring, and monitoring progress in the prevention of errors (Leape, 2002). Nowadays, many healthcare organizations worldwide gather information on incidents and aggregate it into so-called incident reports. For example, in England and Wales, the National Reporting & Learning System (NRLS) database on patient safety incidents has captured over 16 million reports (NHS Improvement, 2017) since 2003. The information in incident reports is both structured and unstructured (e.g., free text descriptions of the incidents). Free text information includes valuable information about contextual factors that may contribute to incidents that may remain hidden if solely relying on structured information (Verma & Maiti, 2018). However, manual analysis of free text found in the incident reports is challenging using traditional qualitative text-based analysis methods, as it can include extraneous information. These datasets are also much larger than those normally analyzed using qualitative software, and novel analytic methods are therefore required.

Text mining brings together multiple techniques from different fields: information retrieval deals with indexing and searching unstructured texts, data mining attempts to discover patterns in structured data, and natural language processing analyzes and synthesizes language and speech (Wachsmuth, 2015). By using text mining, it is possible to analyze words, clusters of words, or whole documents to find associations and similarities and also to explore how these entities are related to other variables (Statsoft, 2018). Examples of using text mining in health care include the study of adverse drug reactions using electronic patient records (Warrer, Holme Hansen, Juhl-Jensen, & Aagaard, 2012), automated detection of follow-up appointments (Ruud, Johnson, Liesinger, Grafft, & Naessens, 2010), and extracting detailed structured medication information from free-text prescriptions (Karystianis, Sheppard, Dixon, & Nenadic, 2016). Text mining has been used to study incident report data from other disciplines, for example, steel plant incidents (Verma & Maiti, 2018), and for the automated analysis of medical critical incident reports (Denecke, 2016). To our knowledge, apart from our previous pilot study (Härkänen, Vehviläinen-Julkunen, Murrells, Rafferty, & Paananen, 2019), this is the first time text mining has been used to analyze MAEs and more specifically to investigate how MAE reports relate to staffing factors.

The purpose of this study is to describe trigger terms that can be used to identify reports of inadequate staffing contributing to MAEs, to identify such reports, to compare the degree of harm within incidents with and without those triggers, and to examine the association between the most commonly reported inadequate staffing trigger terms and the incidence of omission errors and “no harm” terms. Trigger terms are key words that act as clues to identify specific predefined themes derived from the text-based data.


This is a retrospective study using descriptive statistical analysis, text mining, and manual analysis of free text descriptions of medication administration–related incident reports reported to the NRLS for England and Wales in 2016.

Data Collection

The data comprise MAEs reported to the NRLS that occurred between January 1 and December 31, 2016. These data were obtained from NHS Improvement. To be included, incidents needed to involve (a) medication and (b) administration or supply of a medicine from a clinical area, and need to have occurred (c) in an acute NHS trust (a healthcare organization that provides specialist or nonspecialist secondary healthcare services within England or Wales; these exclude mental health and long-term care settings). For this study, only free text descriptions of the incidents (descriptions of what happened) were used.

Data Analysis

The data analysis process included multiple phases (Figure 1). First, staffing-related keywords contributing to MAEs were identified. We used the classification structure described in our pilot study (Härkänen et al., 2019) that manually analyzed 1,012 MAE reports from Finland. In that study, key words (translated here into English) were divided into eight themes: (a) staffing inadequacy, (b) work overload, (c) number of patients, (d) time pressure, (e) skill level or mix, (f) nurse ability to conduct the task, (g) distractions and interruptions, and (h) patient-related issues increasing workload. For verification of this classification, a small sample of the NRLS’s medication administration incidents was collected. All incidents that included “work environment” contributing factors (n = 211 of 72,390 reports) were analyzed manually to identify key words that described staffing factors. For this study, only key words based on both classifications that might be linked to inadequate staffing were chosen as the starting point of the analysis: these were “staff,” “staffing,” “lack,” “pressure,” “stress,” “workload,” and “busy” (Table S1).

Details are in the caption following the image
Data analysis process. [Colour figure can be viewed at wileyonlinelibrary.com]

The SAS Enterprise Miner 13.2 Text Miner tool (SAS Institute Inc., Cary, NC, USA) with a “bag-of-words” method was used to count words in the text and to understand how these words relate to each other (summarizing and classifying text), rather than using a semantic (meaning of words) method. The purpose was to discover themes and concepts within the free text descriptions. The method is described in more detail in a previous pilot study (Härkänen et al., 2019). Data in the form of text (Excel file) was first converted into the SAS format for importing into Text Miner, where the algorithms would then be applied (Verma & Maiti, 2018). SAS Text Miner automatically processes the data using text parsing, which includes tokenization (breaking text into words or terms); stemming, which reduces words to their stem or root forms; and part-of text tagging (for each word, the algorithm decides whether it is a noun, verb, adjective, adverb, preposition, and so on). Text filtering is then used to reduce the total number of parsed terms and check the spellings. The English language was chosen for parsing and filtering the text. An SAS Text Miner stop list was used (a list of all of the possibly irrelevant words), so some parts of the text that included auxiliary verbs, conjunctions, possessive pronouns, interjections, numbers, participles, prepositions, and pronouns were ignored. Using an interactive filter viewer, synonyms were combined manually. Unwanted terms were excluded (such as most abbreviations), as well as terms occurring in fewer than 10 reports.

Concept linking is a way to find and display other terms that are highly associated with a selected term. The selected term is shown at the center of a link diagram, and the terms that circle this are those that occur together most often with that central term (SAS Institute Inc., 2012.) The strength of association between two terms is based on the principle of conditional probability, which is the probability that term B exists given that term A already exists in the document. Initially, automated concept linking by the SAS Text Miner was verified by comparing the results with a manual analysis. The term “workload” was chosen for this comparison. Concept linking was used to identify highly correlated terms and compared manually against all descriptions that included the term “workload.” Further, concept linking was conducted on other terms that described inadequate staffing level.

The search field of the SPSS database including all (N = 72,390) incident reports was used to review the free text descriptions and identify trigger terms in the data. Characteristics of the data (trigger, error category, degree of harm) were described using frequencies and percentages (Table 1). Degree of harm was designated by reporters as no harm, low harm (patients required extra observation or minor treatment), moderate harm (short-term harm patients required further treatment or procedures), severe harm (permanent or long-term harm), or death (caused by the patient safety incident; see Table 1). Statistical testing of associations, using Fisher’s exact tests, was confined to the triggers that most commonly reported “omissions” and “no harm.” The “no harm” category was chosen for statistical testing since it was the only category that existed within all triggers. Incidents with any staffing trigger were then compared with all other incidents to find out if staffing triggers as a group were pointing to higher, lower, or similar risk overall.

Table 1. Characteristics of Medication Administration Incidents (N = 72,390)
Incidents n %
Degree of harm
No harm 62,461 86.3
Low 9,147 12.6
Moderate 708 1.0
Death 46 0.1
Error type
Omitted medicine/ingredient 19,815 27.4
Other 12,528 17.3
Wrong frequency 6,975 9.6
Wrong drug 5,494 7.6
Wrong/unclear dose or strength 5,187 7.2
Wrong quantity 4,415 6.1
Mismatching between patient and medicine 3,437 4.7
Wrong storage 2,091 2.9
Wrong method of preparation/supply 2,001 2.8
Wrong route 1,715 2.4
Adverse drug reaction (when used as intended) 1,584 2.2
Wrong/omitted/passed expiry date 1,458 2.0
Wrong formulation 1,256 1.7
Patient allergic to treatment 1,248 1.7
Unknown 1,131 1.6
Contraindication to the use of the medicine in relation to drugs or conditions 1,079 1.5
Wrong/transposed/omitted medicine label 505 0.7
Wrong/omitted verbal patient directions 406 0.6
Wrong/omitted patient information leaflet 65 0.1
Total 72,390 100


The Research Ethics Office of King’s College London gave ethical approval for this study (LRS-17/18-5150) in October 2017.


Characteristics of the Data

Data included 72,390 descriptions of medication administration incidents, which comprised 1,257,570 nouns, 781,418 verbs, 226,746 adverbs, 172,527 adjectives, 131,295 noun groups, and 111,156 pronouns. The majority (86.3%, n = 62,461 incidents) were reported as not causing patient harm. The most common error types were omitted medicine or ingredient (27.4%, n = 19,815), other (17.3%, n = 12,528), wrong frequency (9.6%, n = 6,975), or wrong drug (7.6%, n = 5,494; see Table 1).

Identifying Triggers Describing Inadequate Staffing Level

Workload (including the terms “work load,” “busy workload,” and “high workload”) was recorded 114 times in 99 MAE reports. Terms that were most often related to workload based on automated concept linking were “staffing,” “high,” “unable,” “busy,” “heavy,” “work,” and “load.” Concept linking of the term “workload” and expanded links is shown in Figure S1. When findings were compared with the manual analysis of terms closely related to the term “workload,” it was found that most terms were identical (Table S2).

Further, concept linking of the terms chosen from the preliminary classification and additional terms based on “workload” revealed connections between those describing staffing inadequacies (Table 2). Utilizing this information, triggers (by combining connected words) were chosen for testing. Those triggers were “poor staffing,” “short staff(ing),” “lack of staff,” “staffing level,” “staffing workload/workload of staffing,” “workload/work load/busy load/heavy load,” “busy ward/busy unit/busy staff/busy nurse/busy colleague/busy time/busy shift/busy night,” “extremely busy,” “workload stress,” and “stress load” (see Table 2).

Table 2. Concept Linking and Identification of Staffing Triggers for Testing
Terms Frequency Documents Related terms based on concept linking
Concept linking by preliminary classification
Staff 19,033 13,281 Know, pharmacist, aware, medical, drug, nurse, patient, ward
Staffing 134 125 Short, duty, unable, ward, skill, poor, train, level
Lack 227 218 Administration, issue, Datix [incident reporting software], supply, case, care, delay, communication
Pressure 1,056 833 Area, high, infusion, immediately, scan, CT, contrast, cannula
Stress (verb) 47 46 Midnight, labetalol, oral medication, load, feel, duplicate, workload, importance
Workload 114 99 Staffing, high, unable, busy, heavy, load, work, due
Busy (hurry, rush) 1,115 892 Colleague, unit, time, night, shift, check, staff, nurse, ward
Terms added based on concept linking of the term “workload”
Heavy (adjective) 42 42 Workload, load, feel, marcaine, extremely, block, spinal, epidural
High (adjective) 1,525 1,401 Risk, high risk, hour, team, unit, dose, level, insulin
Unable 3,311 2,913 Medication, care, home, visit, request, staff, medical, nurse, ward
Load (noun) 64 59 Case, stress, busy, caseload, heavy, viral, workload, work
Due 5,101 4,449 Time, night, dose, level, shift, morning, staff, drug, nurse
Terms added based on above findings
Poor 297 284 Staffing, output, poor documentation, poor renal function, intake, renal, access
Short (adverb) 78 78 Work, workload, porter, help, busy, feel, staff, breath
Extremely 252 231 Time, explain, anxious, high, agitated, upset, busy, angry
Identified triggers for testing

Poor staffing

Short staff(ing), lack of staff, staffing level

Staffing workload/workload of staffing

Workload/work load/busy load/heavy load

Busy ward, busy unit, busy staff, busy nurse, busy colleague, busy time, busy shift, busy night

Extremely busy

Workload stress, stress load

Testing the Triggers for Identifying Inadequate Staffing Level

The most common trigger was “short staffing,” found in 81 incidents. Of these incidents, 28% were dose omission and most (89%) did not cause patient harm. Other common triggers were “workload” (n = 80), “extremely busy” (n = 59), “staffing level” (n = 35), and “busy shift” (n = 34). For the triggers “workload” (n = 80), “busy night” (n = 8), “busy unit” (n = 6), the percentage of no harm was lower than that found for all incident’s (75.0–85.0% vs. 86.3%). Omission errors were more common amongst incidents with documented inadequate staffing triggers than for all incidents combined (with staffing triggers and without); for example, for the trigger “poor staffing” the incidence of omission errors was 89.0% (8/9) vs. 27.4% (19,815/72,390; Table 3).

Table 3. Testing Identified Triggers by Searching Medication Administration Incidents (N = 72,390) and Analyzing Frequencies of Triggers, Error Category, and Degree of Harm of the Identified Incidents

Error category

f (%)

Degree of harm

f (%)

Short staff(ing) n = 81

Omitted, n = 23 (28.4)

Other, n = 12 (14.9)

Mismatching, n = 10 (12.3)

wrong drug, n = 7 (8.6)

Wrong frequency, n = 6 (7.4)

Wrong dose, n = 4 (4.9)

Wrong quantity, n = 4 (4.9)

Wrong storage, n = 3 (3.7)

Wrong expiry date, n = 3 (3.7)

Wrong verbal patient directions, n = 3 (3.7)

Unknown, n = 2 (2.5)

Wrong formulation, n = 1 (1.2)

Adverse drug reaction, n = 1 (1.2)

Contraindication, n = 1 (1.2)

Wrong method of preparation, n = 1 (1.2)

No harm

n = 72



n = 8



n = 1 (1.2%)

“Two patients missed doses of IV medication due to MEA being too short staffed to be able to come to the ward and re-canullate [canulate] for 6 hr on night shift.”
Workload n = 80

Omitted, n = 40 (50.0)

Other, n = 15 (18.8)

Wrong frequency, n = 6 (7.5)

Mismatching, n = 5 (6.3)

Wrong quantity, n = 3 (3.8)

Wrong drug, n = 3 (3.8)

Wrong dose, n = 2 (2.5)

Unknown, n = 2 (2.5)

Wrong storage, n = 1 (1.3)

Wrong route, n = 1 (1.3)

Wrong method of preparation, n = 1 (1.3)

Wrong expiry date, n = 1 (1.3)

No harm

n = 68



n = 11



n = 1


“08:00 Medications omitted in error. Workload on ward very busy and staff nurses mis-communicated with each other during this busy period, which resulted in the omission. The number and acuity of patients on the ward exceeded the number of nursing staff required on shift; this was highlighted before the shift began by the night nurses on duty, as well as being highlighted to the clinical site team at the start of the day, as well as in the 08:15 hospital operational meeting. There were no spare nurses in the hospital to help nor available bed spaces elsewhere to move patients off the ward to ease workload.”
Extremely busy n = 59

Other, n = 14 (23.7)

Omitted, n = 13 (22.0)

Wrong frequency, n = 8 (13.6)

Wrong drug, n = 7 (11.9)

Wrong dose, n = 5 (8.5)

Wrong quantity, n = 4 (6.8)

Mismatching, n = 2 (3.4)

Wrong method of preparation, n = 2 (3.4)

Wrong verbal patient directions, n = 1 (1.7)

Contraindication, n = 1 (1.7)

Adverse drug reaction, n = 1 (1.7)

Wrong expiry date, n = 1 (1.7)

No harm

n = 57



n = 2


“At 5.30 am while updating handover I realized one of my 14 patients was on IV antibiotics and I had not given them last night. Ward extremely busy last night.”
Staffing level (n = 35)

Omitted, n = 19 (54.3)

Other, n = 5 (14.3)

Wrong quantity, n = 2 (5.7)

Unknown, n = 2 (5.7)

Other, n = 2 (5.7)

Mismatching, n = 1 (2.9)

Wrong frequency, n = 1 (2.9)

Wrong dose, n = 1 (2.9)

Wrong drug, n = 1 (2.9)

Wrong patient directions, n = 1 (2.9)

No harm

n = 30



n = 5


“Due to decreased staffing levels (only one trained pool staff) some medications from the 8 am medication round were not given whilst after 11:30, thus affecting the 12:00 medication round.”
Busy shift n = 34

Omitted, n = 8 (23.5)

Wrong drug, n = 5 (14.7)

Mismatching, n = 5 (14.7)

Wrong dose, n = 4 (11.8)

Other, n = 4 (11.8)

Wrong quantity, n = 3 (8.8)

Wrong frequency, n = 3 (8.8)

Wrong formulation, n = 1 (2.9)

Wrong method of preparation, n = 1 (2.9)

No harm

n = 30



n = 3



n = 1


“When drawing up a morphine bolus on a very busy shift, the dose was incorrectly calculated, despite being singularly checked by three nurses. As a result, 1 mg was given rather than 100 µg.”
Work load n = 26

Omitted, n = 14 (53.8)

Other, n = 5 (19.2)

Mismatching, n = 2 (7.7)

Wrong formulation, n = 1 (3.8)

Wrong frequency, n = 1 (3.8)

Mismatching, n = 1 (3.8)

Wrong quantity, n = 1 (3.8)

Wrong dose, n = 1 (3.8)

No harm

n = 23



n = 3


“Dose of levemir insulin was omitted by mistake as medication due at 08:00 in the morning, acuity in the area was high due to both work load, poorly patients, skill mix, and staffing.”
Busy ward n = 16

Omitted, n = 6 (37.5)

Other, n = 3 (18.8)

Wrong drug, n = 2 (12.5)

Wrong dose, n = 1 (6.3)

Wrong frequency, n = 1 (6.3)

Wrong expiry date, n = 1 (6.3)

Wrong formulation, n = 1 (6.3)

Patient allergic to treatment, n = 1 (6.3)

No harm

n = 15



n = 1


“Due to the busy ward environment and nurses involved in drugs round being assigned to 1:1 observations throughout the morning, the medication was not administered when the patient woke and was a missed dose.”
Lack of staff n = 15

Omitted, n = 10 (66.7)

Mismatching, n = 1 (6.7)

Wrong dose, n = 1 (6.7)

Wrong quantity, n = 1 (6.7)

Wrong frequency, n = 1 (6.7)

Unknown, n = 1 (6.7)

No harm

n = 13



n = 2


“IV PIP/TAZ [piperacillin/tazobactam] administered to wrong patient in error. . . . Rearranging of patients and staff across two pods to enable us to take an admission whilst facilitating/supporting junior skill mix in one pod. I had informed the bleep holder [senior manager] that the shift was busy and I was concerned about lack of staff to cover patients on the day shift if we were to take the admission.”
Poor staffing n = 9

Omitted, n = 8 (88.9)

Wrong drug, n = 1 (11.1)

No harm

n = 9


“Due to poor staffing levels, as a result of sickness and bank shift cancellation, there was no capacity to visit some patients on eye drops [to administer the medication].”
Busy time n = 8

Wrong drug, n = 2 (25.0)

Other, n = 1 (12.5)

Omitted, n = 1 (12.5)

Wrong dose, n = 1 (12.5)

Wrong method of preparation, n = 1 (12.5)

Patient allergic to treatment, n = 1 (12.5)

Mismatching, n = 1 (12.5)

No harm

n = 8


“Wrong medication given to patient by nurse. Nurse gave 5 mg warfarin and 2.5 mg apixaban, which was for another patient (and given) to this patient by mistake…. It was a busy time, and not having a local drugs trolley and having to hunt down keys to get meds from drugs room, I decided to dispense two persons medication at once—which I will now never do again.”
Busy night n = 8

Omitted, n = 3 (37.5)

Wrong dose, n = 2 (25)

Wrong frequency, n = 1 (12.5)

Wrong quantity, n = 1 (12.5)

Mismatching, n = 1 (12.5)

No harm

n = 6



n = 2


“During a busy night shift the patient requested oramoph [morphine sulphate elixir] as she was in quite a lot of pain. I checked the drug chart and it appeared to have 5–10 mL of oramorph prescribed. I administered 10 mL = 20 mg. It wasn't until I went to document how much I had given that I saw the previous midwife had written 10 mg, not mL. I then realized my mistake.”
Busy unit n = 6

Omitted, n = 2 (33.3)

Wrong frequency, n = 2 (33.3)

Other, n = 1 (16.7)

Wrong quantity, n = 1 (16.7)

No harm

n = 5



n = 1


“Very busy unit, IV pantoprazole medication was running at 25 mL/hr instead of what was prescribed as 8 mL/hr. Medication stopped and nurse in charge informed.”
Heavy load n = 1 Other, n = 1

No harm

n = 1


“Patient was assessed by phone the previous day to alleviate heavy load of chemotherapy clinic.”
No findings

Workload stress, n = 0

Stress of workload, n = 0

Busy load, n = 0

Staff(ing) workload, n = 0

Workload of staffing, n = 0

Busy staff, n = 0

Busy nurse, n = 0

Busy colleague, n = 0

Stress load, n = 0


  • f = frequency; IV = intravenous; MEA = Medical Emergency Assistant.

Associations Between Triggers and the Incidence of Omission Errors and “No Harm”

There was significant variation in omission errors across inadequate staffing level trigger terms (Fisher’s exact test = 44.11, p < .001), with those related to workload having the highest percentage of omissions, followed by terms that mention staffing and being busy. The prevalence of “no harm” did not vary statistically between the inadequate staffing trigger terms (Fisher’s exact test = 11.45, p = .49; Table 4).

Table 4. Omission Errors and No Harm Situations for Incidents With Different Trigger Terms
Trigger terms Omission errors No harm situations
Short staff(ing), n = 81 Omitted, n = 23 (28.4%) No harm, n = 72 (88.9%)
Workload, n = 80 Omitted, n = 40 (50.0%) No harm, n = 68 (85.0%)
Extremely busy, n = 59 Omitted, n = 13 (22.0%) No harm, n = 57 (96.7%)
Staffing level, n = 35 Omitted, n = 19 (54.3%) No harm, n = 30 (85.7%)
Busy shift, n = 34 Omitted, n = 8 (23.5%) No harm, n = 30 (88.2%)
Work load, n = 26 Omitted, n = 14 (53.8%) No harm, n = 23 (88.5%)
Busy ward, n = 16 Omitted, n = 6 (37.5%) No harm, n = 15 (93.8%)
Lack of staff, n = 15 Omitted, n = 10 (66.7%) No harm, n = 13 (86.7%)
Poor staffing, n = 9 Omitted, n = 8 (88.9%) No harm, n = 9 (100.0%)
Busy time, n = 8 Omitted, n = 1 (12.5%) No harm, n = 8 (100.0%)
Busy night, n = 8 Omitted, n = 3 (37.5%) No harm, n = 6 (75.0%)
Busy unit, n = 6 Omitted, n = 2 (33.3%) No harm, n = 5 (83.3%)
Stressed due, n = 5 Omitted, n = 0 (0%) No harm, n = 4 (80.0%)
Heavy load, n = 1 Omitted, n = 0 (0%) No harm, n = 1 (100.0%)


We identified trigger terms associated with reports of inadequate staffing levels that contributed to MAEs. These triggers could be used to study incidents and gain further understanding of inadequate staffing levels and their repercussions. The data used in this study (free text descriptions of 72,390 incidents) contained millions of words and large amounts of extraneous information. Only a limited number of terms describing inadequate staffing levels were found. Data that are classified or structured, however, would have been limited in identifying in-depth conditioning factors associated with the occurrence of incidents. Manual categorization of narrative reports can sometimes result in a classification based on a restricted list of categories with a high degree of inconsistency. Furthermore, multiple analysts, and single analysts categorizing over time, may classify or group descriptions differently. Additionally, forcing the narrative description into a predetermined and limited number of categories may carry the risk for losing meaningful and relevant factors (Verma & Maiti, 2018).

The free text descriptions of incident reports provided by the staff are a rich data source (Verma & Maiti, 2018). We found that omission errors were more common in incident reports that included terms linked to inadequate staffing levels; therefore, reports of lack of staff or high workload were associated with medications not being administered at the correct time or not at all. In previous studies, it was found that essential nursing care may be left undone due to a lack of time (e.g., Ball et al., 2018; Brooks-Carthon et al., 2016), thus contributing to adverse patient and staff outcomes (Kalisch & Lee, 2010). Administering the wrong drug or at the wrong frequency were amongst the more common types of incident. One possible interpretation is that high workload leads to verification practices as part of medication administration not being carried out. When compared with all incidents, some inadequate staffing level triggers were linked to lower levels of no harm. This link (i.e., more staff and safer care) has been previously found in other studies (e.g., Aiken et al., 2014; Brooks-Carthon et al., 2016; West et al., 2014), which demonstrated that insufficient numbers of registered nurses can have life-threatening consequences for patients. When care is left “undone,” this places even greater risk on the patient (Ball et al., 2018; RCN, 2017).

Future analysis should consider all other factors that increase nurses’ workloads, such as inadequate skill mix, and patient-related factors. Our data were confined to medication administration incidents in acute care, but the triggers identified could be tested in other healthcare settings and home care facilities, especially those providing for elderly patients (over 75 years of age) who are particularly at risk for serious medication administration incidents (Härkänen et al., 2018). This work lays the groundwork for creating automated text-analytical systems that analyze incident reports in real time and warn managers and staff that staffing levels need adjusting upwards to reduce the occurrence of MAEs. Data from these systems will also identify the circumstances that lead to MAEs, to help inform future intervention studies.

Strengths and Limitations

The text mining application was useful for identifying triggers. Its ability to transform qualitative into quantitative and schematic data was effective, and the algorithms were helpful for identifying the concept links between terms. In addition, there are now so many records that it is no longer practical, in terms of time and cost, to manually review all the reports (Ruud et al., 2010; Verma & Maiti, 2018). Thus, novel text mining methods need to be employed. The credibility of text mining has already been recognized and tested (Verma & Maiti, 2018). Its accuracy, sensitivity, and specificity have proven to be high when compared with manual analysis (Ruud et al., 2010), which was confirmed in our study comparing automated concept linking and manual analysis. Manual analysis was unlikely to be a practical solution for finding triggers because the free text descriptions lack structure and typically have not undergone a process of classification prior to analysis.

The analyses demonstrated in this study required the researchers to make some subjective decisions, such as identifying and selecting the triggers for testing. It is possible that some have been missed. Studies similar to ours could not be found, so there was no clear guidance to follow. Nevertheless, all analysis phases were conducted methodically utilizing knowledge, processes, and procedures from an earlier pilot study that can be applied and adapted for future studies. Combining synonyms were challenging without understanding the original meaning of the word. Many words (e.g., “alert”) could be either a verb, adjective, or noun, or could have multiple meanings, such as the term “back.” We found that certain words were written multiple ways, including some with typographical errors, such as “ampoule,” “ampule,” “ampuole,” and “ampulla.” These differences are due to the variations in language usage, so the same meaning can be expressed in different ways (Denecke, 2016).

Incident report data also have some limitations; in particular, it suffers from under-reporting, which may introduce bias. The quality of the reports may also vary in terms of detail and accuracy (NHS England, 2014.) In addition, the data may have been biased because staff reporting an incident might have attributed the incident to low staffing whilst under-reporting contributing factors that relate to other aspects of inadequate staffing. Finally, SAS Text Miner uses the "bag-of-words" approach, which means that documents are represented with a vector that contains the frequency with which each term occurs in each document. Additionally, word order of sentences is ignored. This approach is very effective for short, paragraph-sized documents, but it can lead to a loss of information with longer documents (SAS Institute Inc., 2012).


A number of triggers related to inadequate staffing were found in incident reports of MAEs. Omission errors were common in incidents that included inadequate staffing trigger terms, especially with the terms “workload” and being “busy.” In addition, the potential risk for patient harm was increased with incident reports that included the terms “workload,” “staffing level,” “busy night,” and “busy unit.” Although this study did not intend to establish causality, it does provide evidence of potential harm. The degree to which MAEs are associated with verification practices that were omitted or not followed to a high enough standard due to staffing-related factors (e.g., heavy demands on a person’s time, high patient acuity) would need to be verified and determined through further research.

The application of text mining to free text reports of MAEs is very much in its infancy. This study will hopefully encourage others to pursue similar research that will ultimately lead to safer care and better patient outcomes.


This work was financially supported by the post-doctoral research funding for the first author (M.H.) of this article by the Academy of Finland. The fifth and the sixth authors (B.D.F., A.M.R.) are supported by the National Institute for Health Research (NIHR) Imperial Patient Safety Translational Research Centre, and the fifth author (B.D.F.) by the NIHR Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance at Imperial College London, in partnership with Public Health England (PHE). The views expressed are those of the authors and not necessarily those of the NHS, NIHR, PHE, or Department of Health and Care.

Clinical Resources