ARTIGO - Assessment of SOAP note evaluation tools in colleges and schools of PHARMACY

Prévia do material em texto

Contents lists available at ScienceDirect
Currents in Pharmacy Teaching and Learning
journal homepage: www.elsevier.com/locate/cptl
Research Note
Assessment of SOAP note evaluation tools in colleges and schools of
pharmacy
Karen R. Sandoa,⁎, Elizabeth Skoyb, Courtney Bradleyc, Jeanne Frenzelb,
Jennifer Kirwind, Elizabeth Urteagae
a Dept. of Pharmacotherapy & Translational Research, University of Florida College of Pharmacy, Gainesville, FL, United States
b Dept. of Pharmacy Practice, North Dakota State University School of Pharmacy, Fargo, ND, United States
c High Point University, Fred Wilson School of Pharmacy, High Point, NC, United States
d Dept. of Pharmacy and Health Systems Sciences, School of Pharmacy, Northeastern University, Boston, MA, United States
e Dept. of Pharmacy Practice, Feik School of Pharmacy, University of the Incarnate Word, San Antonio, TX, United States
A R T I C L E I N F O
Keywords:
SOAP notes
Documentation
Assessment
A B S T R A C T
Introduction: To describe current methods used to assess SOAP notes in colleges and schools of
pharmacy.
Methods: Members of the American Association of Colleges of Pharmacy Laboratory Instructors
Special Interest Group were invited to share assessment tools for SOAP notes. Content of
submissions was evaluated to characterize overall qualities and how the tools assessed subjective,
objective, assessment, and plan information.
Results: Thirty-nine assessment tools from 25 schools were evaluated. Twenty-nine (74%) of the
tools were rubrics and ten (26%) were checklists. All rubrics included analytic scoring elements,
while two (7%) were mixed with holistic and analytic scoring elements. A majority of the rubrics
(35%) used a four-item rating scale. Substantial variability existed in how tools evaluated
subjective and objective sections. All tools included problem identification in the assessment
section. Other assessment items included goals (82%) and rationale (69%). Seventy-seven
percent assessed drug therapy; however, only 33% assessed non-drug therapy. Other plan items
included education (59%) and follow-up (90%).
Discussion and conclusions: There is a great deal of variation in the specific elements used to
evaluate SOAP notes in colleges and schools of pharmacy. Improved consistency in assessment
methods to evaluate SOAP notes may better prepare students to produce standardized
documentation when entering practice.
Introduction
Pharmacists are considered essential members of the healthcare team because of their expertise in delivering comprehensive
medication management.1 Consequently, pharmacist documentation of their services is vital to the patient's continuity of care and
demonstrates both the value and accountability of the pharmacist's service to the team.2 Additional purposes of pharmacist
documentation include ensuring compliance with laws and regulations for maintenance of patient records and creating a record of
http://dx.doi.org/10.1016/j.cptl.2017.03.010
⁎ Corresponding author.
E-mail addresses: ksando@cop.ufl.edu (K.R. Sando), Elizabeth.skoy@ndsu.edu (E. Skoy), cbradley@highpoint.edu (C. Bradley),
Jeanne.frenzel@ndsu.edu (J. Frenzel), j.kirwin@northeastern.edu (J. Kirwin), montfort@uiwtx.edu (E. Urteaga).
Abbreviations:SOAP, subjective, assessment, objective, plan
Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
1877-1297/ © 2017 Elsevier Inc. All rights reserved.
Please cite this article as: Sando, K.R., Currents in Pharmacy Teaching and Learning (2017), 
http://dx.doi.org/10.1016/j.cptl.2017.03.010
http://www.sciencedirect.com/science/journal/18771297
http://www.elsevier.com/locate/cptl
http://dx.doi.org/10.1016/j.cptl.2017.03.010
http://dx.doi.org/10.1016/j.cptl.2017.03.010
mailto:ksando@cop.ufl.edu
mailto:Elizabeth.skoy@ndsu.edu
mailto:cbradley@highpoint.edu
mailto:Jeanne.frenzel@ndsu.edu
mailto:j.kirwin@northeastern.edu
mailto:montfort@uiwtx.edu
http://dx.doi.org/10.1016/j.cptl.2017.03.010
services provided for billing and reimbursement.3 Standardized documentation of pharmacists’ clinical services will be critical to
receive reimbursement should national provider status efforts prove successful.
Several forms of pharmacist documentation can be used to record clinical interventions, including unstructured notes, semi-
structured notes, and systematic records.4 Systematic documentation styles include SOAP (subjective, objective, assessment, plan),
TITRS (title, introduction, text, recommendation, signature), and FARM (findings, assessment, recommendations or resolutions/
management).4 SOAP documentation is the primary documentation method utilized by pharmacists and other healthcare providers in
both inpatient and outpatient settings.
The American Society of Health System-Pharmacists (ASHP), the American College of Clinical Pharmacy (ACCP), and the
American Pharmacists Association (APhA) each provide guidelines for the documentation of pharmaceutical care services in the
permanent medical record (Table 1).2,3,5 Each guideline underscores the importance of consistency in the care delivery process as
well as standardization in documentation of comprehensive medication management services. The Joint Commission of Pharmacy
Practitioners (JCPP) Pharmacists’ Patient Care Process6 also emphasizes the importance of documentation to communicate with
physicians and other healthcare providers in the provision of safe, effective and coordinated care (Table 2).
The Accreditation Council for Pharmaceutical Education (ACPE) Standards 20167 list documentation of clinical activities as
essential to the delivery of patient-centered care (Key Element 2.1 – Patient-centered care). Assessment of a student's ability to
accurately and concisely document patient care activities is required throughout the Pharm.D. curriculum to ensure students are
ready to progress to advanced pharmacy practice experiences (“APPE-ready”) and at graduation can provide direct patient care in a
variety of healthcare settings (“practice-ready”) within an interprofessional collaborative team (“team-ready”). Instruction and
assessment of SOAP note writing is frequently delivered in skills laboratory courses, as clinical documentation is a fundamental
patient care skill. With increased emphasis on standardizing the patient care process and documentation of pharmacist services
within the medical record,5,6 it is important to understand if our evaluation tools for evaluating student SOAP notes (such as rubrics
or checklists) include the recommended components to document a comprehensive medication management encounter. However,
there has been no published evaluation of how SOAP notes are assessed within pharmacy education. The purpose of this study is to
describe current practices in assessing SOAP notes within skills laboratory courses at colleges and schools of pharmacy.
Table 1
Comparison of national pharmacy organizations recommendations for documenting pharmaceutical care services.
Category ASHP2 APhA3 ACCP5
Subjective • Summary of medication history including
compliance
• Medication allergies and manifestations
• Demographics
• PMH
• FH
• SH
• Allergies
• Medication history including patient
concerns, reports of adverse events
Medication history
• Patient's past medication use
and related health problems
• Current medications and actual
use, adherence, attitudes
towards therapy
• Medication-related allergies and
ADE
Objective • Drugs administered (including investigational
drugs)
• Clinical and PK data pertinent to drug regimen
• Physical signs and symptoms relevant to drug
therapy
• Vital signs
• Medication lists
• Laboratory data
Not specifically stated
Assessment Actual and potential DRPs that warrant
surveillance
• List of identified DRP
• Goals of therapy for each DRP
• Degree of control (eg above goal,
uncontrolled) for each disease state or
DRP
Active problem list with an assessment
of each problem
• Therapeutic appropriateness (including
route and method of administration)
• Drug-drug, drug-food,drug-laboratory test,
and drug-disease interactions
• Drug toxicity and ADE
• List of current health conditions
and supporting data for status of
each condition
• Emphasize DRPs that impact on
desired goals
• List of DRPs that may be unrelated
to current health conditions
Plan • Adjustments made to drug dosage, dosage
frequency, dosage form, or route of
administration
• Drug-related patient education and counseling
provided
• Oral and written consultations to other health
care providers
• Specific changes to drug therapy (eg drug,
dose, route, and frequency)
• Monitoring parameters
• Follow-up care
• Patient education
• Specific medication therapy plan
including drug, dose, route,
frequency and relevant
monitoring parameters
• Collaborative plan for follow-up
evaluation and monitoring and
future visits
Overall
Qualities
Legibility, clarity, lack of judgmental language,
completeness, use of standard format (eg SOAP),
how to contact the pharmacist (eg pager or
telephone number)
Legibility, use objective language, avoid
judgmental language, correct spelling and
grammar, date/time and patient's name listed,
avoid making a diagnosis, completed soon
after patient encounter
Compliant with accepted standards for
documentation in setting (including
billing, where applicable)
Abbreviations: APhA=American Pharmacists Association; ASHP=American Society of Health-System Pharmacists; ACCP=American College of Clinical Pharmacy;
PMH=past medical history; FH = family history; SH=social history; ADE=adverse drug event; PK=pharmacokinetic; DRPs=drug-related problems;
SOAP=subjective, objective, assessment, plan.
K.R. Sando et al. Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
2
Methods
In 2013, the American Association of Colleges of Pharmacy (AACP) Laboratory Instructors Special Interest Group (Lab SIG)
formed an assessment committee in order to collect, share, and standardize assessment tools used across multiple schools. In 2014, an
11-item survey was sent to members of the AACP Lab SIG listserv (n=275 members) to evaluate the types of assessments delivered
within skills laboratory courses. The voluntary survey collected demographic information including institution type (public vs.
private) (n=1), respondent's role in their institutions’ skills laboratory course (n=1), the size and structure of the lab (n=2), types of
personnel assisting in the course (e.g., residents, students, teaching assistants; n=2), credit hours assigned to the course (n=1), and
four questions on the type of skills (n=2) and assessments (n=2) delivered in the skills laboratory course. Of the 41 respondents to
the survey (response rate=15%), 80% (n=32) indicated that SOAP note writing was assessed during their laboratory course. Other
common assessments included physical assessment (e.g., blood pressure measurement) (90%; n=36), patient counseling (90%;
n=36), patient interviewing (85%; n=34), drug information skills (83%; n=33), and professionalism (80%; n=32).
Following review of these survey data, the AACP Lab SIG assessment committee decided to focus committee efforts on analyzing
assessment tools for SOAP note writing within colleges and schools of pharmacy. The AACP Lab SIG assessment committee via the
Lab SIG listserv requested colleges and schools of pharmacy to submit their SOAP note assessment tool for evaluation. All tools were
submitted voluntarily and placed in a shared cloud storage service that was accessible to members of the AACP Lab SIG assessment
committee. Demographic information (e.g., geographic location, public vs. private, duration of establishment) was collected for each
school that submitted an assessment tool.
Five Lab SIG assessment committee members individually evaluated all submitted SOAP note assessment tools and identified
items evaluated by each tool. Each committee member recorded identified items evaluated by each tool. Examples of identified items
included rating of spelling and grammar, inclusion of vital signs in the objective section, or whether the evaluation tool was a rubric
or checklist. All committee members then met face-to-face to discuss items identified and determined common themes to structure a
content analysis. Members also examined the PPCP6 for additional content analysis items. The identified items were separated into
five content areas: overall qualities, subjective, objective, assessment, and plan information.
After content areas were established, three members of the committee read through the assessment tools a second time to perform
content analysis using the established categorical items. If disagreement occurred for classifying an item between members, the
majority classification was included in the final content analysis. The content analysis and descriptive statistics were performed
within Microsoft Excel™. This study received exempt approval by the University of Florida Institutional Review Board.
Results
A total of 39 SOAP note assessment tools were collected from 25 different colleges and schools of pharmacy in the United States
Table 2
Documentation elements included in the pharmacists' patient care process.6
Collect
• Current medication list and medication use history for prescription and non-prescription medications, herbals, and dietary supplements
• Medical history, health and wellness information, biometric test data, physical assessment findings
• Patient lifestyle habits, preferences and beliefs, health and functional goals
• Socioeconomic factors that affect access to medications and other aspects of care
Assess
• Assess and analyze collected information in context of patient's overall health goals to identify and prioritize problems and achieve optimal care
• Assessment should include:
• Each medication for appropriateness, effectiveness, safety, and patient adherence
• Health and functional status, risk factors, health data, cultural factors, health literacy, and access to medications
• Immunization status and need for preventative care or other health care services
Plan
• Develop an individualized patient-centered plan, in collaboration with other health care professionals, the patient, and caregiver that is evidence-based and
cost-effective
• Elements of the plan should include:
• Medication-related problems and plans to optimize medication therapy
• Goals of therapy
• Patient education, empowerment, and self-management
• Follow-up and transitions of care as appropriate
Implement
• Address medication- and health-related problems and engage in preventive care strategies including vaccine administration
• Initiate, modify, discontinue, or administer medication therapy as authorized
• Provide education and self-management training to the patient or caregiver
• Contribute to coordination of care (e.g., referral or transition to another health care professional)
• Schedule follow-up care as needed to achieve goals of therapy
Follow-Up: Monitor and Evaluate
• Continuous monitoring and evaluation of:
• Medication appropriateness, effectiveness, safety, and patient adherence
• Clinical endpoints that contribute to patient's overall health
• Outcomes of care, including progress toward goals of therapy
K.R. Sando et al. Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
3
and Canada (approximately 20% of AACP member institutions). Some programs (n=9) submitted multiple assessment tools tailored
to specific types of SOAP note activities (e.g., IPPE, APPE, disease-specific case). Most geographic areas of the United States were
represented, and one tool was submitted from a Canadian institution. In addition, representation from both public and private
institutions was approximately equal (Table 3). Most schools that submitted assessment tools have been established for greater than
10 years. Seventy-four percent of the tools were rubrics, while the remaining 26% were in a checklist format (Table 4). Over 50% of
the SOAP note tools assessed grammar and spelling (59%), organization (79%), conciseness(62%), and relevance of the information
provided (77%). Relevance was defined as lack of excessive or extraneous information. A smaller number of tools assessed for the
presence of references, unapproved abbreviations, quality, and professional tone (Table 4).
The content areas were divided into subjective, objective, assessment, and plan sections. The most variability existed in the
subjective and objective sections of tools. In the subjective section, 46% of the tools assessed subjective information globally versus
individual areas of subjective information, such as history of present illness (HPI) and social history (SH). Only 8% of assessment tools
specifically evaluated medication history in the subjective section. Vital signs, physical exam, laboratory data, current medications
and other diagnostic tests were items that were assessed in the objective portion of the tools. None of these items were consistently
assessed by most schools. Forty-four percent of the tools included medications in the objective portion and 10% included medications
in the subjective portion (Table 4). The remaining 46% of tools did not specify where the medication list should reside in the SOAP
note.
More consistency was seen in the assessment and plan sections of the tools. All the tools included drug-related problem
identification in the assessment section. Priority order of drug-related problems was evaluated in 49% of the tools. Most tools
evaluated goals of therapy (82%) and rationale (69%) for drug therapy selection. Items noted in the plan sections of the tools related
to specific drug therapy, specific non-drug therapy, monitoring for efficacy and toxicity, patient education and follow-up. Seventy-
seven percent assessed specific drug therapy; however only 33% assessed non-drug therapy. Fifty-five percent assessed for monitoring
for efficacy, but only 36% assessed monitoring for toxicity. Most schools assessed for patient education (59%) and follow-up (90%),
as shown in Table 4.
All of the rubrics evaluated items of the SOAP note in an analytic manner (e.g., scoring specific elements in S, O, A, P). Few rubrics
(7%, n=2) were mixed, combining both holistic and analytic scoring elements. Point allocation and weighting of the subjective,
objective, assessment, and plan sections of the note also varied between schools. Nearly half the evaluation tools (46%; n=18) had
equal weighting between sections while 44% (n=17) weighted the assessment and plan sections of the note with a higher point
allocation. Four evaluation tools (10%) did not contain a weighting or point system. Of the rubric evaluation tools (n=29), rating
scales for the evaluation tools ranged from six-item rating scales (7%, n=2) to two-item rating scales (3%, n=1). The majority of
schools (35%, n=10) used a four-item rating scale.
Discussion
There is a wide degree of variation in the SOAP note assessment tools used across colleges and schools of pharmacy. A majority of
the tools evaluated some component of note structure (e.g., organization, readability, and conciseness), but this varied on whether it
was scored within each section or holistically for the entire note. There was also considerable variation in what specific elements
were evaluated within each section of the SOAP note. For example, only 8% of evaluation tools assessed for documentation of a
patient's medication history in the subjective section. Collection and documentation of a patient's relevant medication history is an
important part of the Pharmacists’ Patient Care Process and its inclusion in SOAP note evaluation tools may communicate its
importance to students as they learn how to effectively document care.
While it may be likely that students would be instructed to include specific elements within designated sections of a SOAP note,
Table 3
Demographic information of schools submitting SOAP note
assessment tools (N=25).
Category Number (%)
Type of Institution
Private 11 (44%)
Public 14 (56%)
Geographic Distribution
New England 4 (16%)
Mid-Atlantic 0 (0%)
South 6 (24%)
Southwest 3 (12%)
Midwest 9 (36%)
West 2 (8%)
Canada 1 (4%)
Duration of Establishment
Less than 5 years 0 (0%)
5–10 years 5 (20%)
> 10 years 20 (80%)
K.R. Sando et al. Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
4
the lack of specificity within the evaluation tool may make it challenging for different raters to score student work consistently. The
lack of consistency in SOAP note assessment tools may lead to variability in documentation when students enter practice. As the
profession moves towards improved consistency in the way pharmacist care is delivered, uniformity in pharmacists’ documentation
will also be needed to communicate the role of the pharmacist to other healthcare providers.
Each of the major pharmacy organizations in the United States provides guidance on recommended elements of pharmacist
documentation (Table 1). While there is some variability between recommended items to include, each organization suggests that a
thorough evaluation of a patient's medication history should be included in the subjective section. The medication history may
include: history of current, and past medication use; adherence; medication-related allergies and adverse drug events (ADE); attitudes
towards current and past drug therapy; and a review of past medical history (PMH), family history (FH), and SH (social history).
Table 4
Content analysis of SOAP note evaluation tools (n=39).
Criteria N (%)
Overall Items
Rubric 29 (74%)
• Analytic rubric elements (n=29) 29 (100%)
• Holistic rubric elements (n=29) (mixed rubrics) 2 (7%)
Checklist 10 (26%)
Rating Scale for Rubrics (n=29)
6 – items 2 (7%)
5 – items 7 (24%)
4 – items 10 (35%)
3 – items 8 (28%)
2 – Items 1 (3%)
No scale specified 1 (3%)
Content and Structure
References evaluated 7 (18%)
Grammar/spelling evaluated 23 (59%)
Assesses for presence/absence of unapproved abbreviations 15 (38%)
Global assessment of note quality 5 (13%)
Tool is specific to a case or disease state 2 (7%)
Overall organization 31 (79%)
Conciseness 24 (62%)
Relevance of information or marks against excessive/extraneous information 30 (77%)
Professional tone 8 (21%)
Subjective Information
Chief complaint 17 (44%)
History of present illness 13 (33%)
Past medical history 15 (38%)
Medication history 3 (8%)
Allergies 13 (33%)
Social history 13 (33%)
Family history 11 (28%)
Subjective information is globally assessed (specific items above not listed as criteria) 19 (46%)
Medications are assessed in ‘S’ vs. ‘O’ 4 (10%)
Objective Information
Vital signs 16 (41%)
Physical exam information 12 (31%)
Laboratory data 16 (41%)
Current medications 17 (44%)
Other diagnostic tests 11 (28%)
Assessment Information
Identify DRPs 39 (100%)
Priority order of DRPs 19 (49%)
Goals of therapy 32 (82%)
Rationale 27 (69%)
Plan Information
Specific drug therapy recommendation(s) 30 (77%)
Specific non-drug therapy recommendation(s) 13 (33%)
Monitoring parameters for efficacy 16 (55%)
Monitoring parameters for toxicity 14 (36%)
Patient education 23 (59%)
Follow-up 35 (90%)
Global monitoring is assessed vs. specific efficacy/toxicity parameters 22 (56%)
Abbreviations: ‘S’=subjective; ‘O’=objective; DRPs=drug-related problems.
K.R. Sando et al. Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
5
There is little agreement in the objective data required; however, defining objective information is usually straightforward as it is
clinically observable and typically includes the medication list, vital signs, and relevant laboratory or diagnostic data. Each
organization agrees on the components of the assessment to include listing actual and potential drug-related problems, as well as
goals of therapy for each problem. For the plan, specific adjustments to medication regimens should be listed including drug dosage,
route, and frequency. Monitoring plans and follow-up care should be included. There is less agreement with the inclusion of patient
education in the plan.Overall, pharmacist documentation should be legible, avoid judgmental or opinionated language, adhere to a
standard format of the organization, and contain no spelling or grammatical errors.
In 2003, Currie et al.8 developed guidelines for the documentation elements that need to be included in any record of pharmacist-
provided care. This was the first paper to develop consensus recommendations for what components pharmacist documentation
should include. Through review of a list of 85 potential documentation elements drawn from literature followed by a consensus
Delphi process, a list of 27 documentation data elements was selected.8 These data elements were incorporated into the Tool for
Evaluation of Documentation (TED), shown in Fig. 1. The TED was developed for use in a broad range of patient care settings to
evaluate the quality of pharmacist documentation. The first section of the TED can be used to evaluate a single patient encounter note
and the second section is used to evaluate the entirety of a patient record for essential documentation elements. The tool was
validated by a team of reviewers in a quality assurance environment with reviewers reaching consensus on presence or absence of the
elements included in 94% of the 100 encounter notes reviewed.9 Authors recommended the use of the TED to improve documentation
abilities of new pharmacists entering the workforce through including its use in pharmacy school curricula.8 Pharmacy educators
may consider following a similar process within their institutions to develop a standardized assessment tool for evaluating student
SOAP notes. Faculty utilizing SOAP note evaluation tools in their courses may assess their tools for the presence of essential elements
of documentation found in the TED and alter as necessary to include these elements.
The more recently published PPCP6 describes the steps pharmacists should use to provide patient-centered care in collaboration
with other health care team members. Within the process (shown in Table 2), five specific steps are described including: collect,
assess, plan, implement, and follow up (including monitor and evaluate). While the process does not specify which of these elements
should be documented in the patient's permanent medical record, it advocates for consistency in delivery of the patient care process
and how it is taught and practiced. To promote consistency in how the PPCP is taught across institutions, uniformity in the
documentation elements for a comprehensive medication management encounter should also be taught and evaluated with
consistency. It is generally accepted that “assessments drive students’ learning,” therefore, standardizing how student SOAP notes are
evaluated may better prepare our students for documenting in practice.10
The majority of SOAP note assessment tools included in this study were rubrics. Of the rubrics included, there was variation in the
number of items included in the rating scale, with four items as the most common number. The items within the rating scales were
categorized differently between SOAP evaluation tools; for example, one tool included four items ranging from “unacceptable” to
“proficient” for documentation of subjective, objective, assessment and plan information. The number of items in a rating scale can
influence rater reliability in making valid conclusions from a specific learning assessment. Based on cognitive psychology research,
rating scales with four (± 1) options are recommended due to limitations in short-term memory of human raters and the
subconscious tendency to “chunk” ratings with a greater number of categories.10 Use of an odd number of rating scales (e.g., three,
five, or seven) can lead to reduced interrater reliability with removal of the midpoint improving reliability measures.11 Several
pharmacy education applications have shown success using a four-item rating scale when conducting performance-based
assessments.12–14
Of the rubrics included in the analysis (n=29), all were analytical rubrics with few (7%; n=2) including holistic judgments of
note quality. With analytic rubrics, the evaluator scores separate, individual parts of the product or performance and sums individual
scores to obtain a total score.15 A holistic rubric requires the evaluator to score the overall product or work without judging the
component parts separately. The score reported using a holistic rubric is on the overall quality, proficiency, or understanding of the
specific content and skills included in the assessment.15 When considering use of analytic versus holistic rubrics for evaluation of
student SOAP notes, faculty time, interrater reliability, and purpose of the assessment (e.g., formative versus summative) should be
considered.15 Use of analytic rubrics is typically more time-consuming than holistic rubrics due to the need to grade several
individual components. However, the degree of feedback provided to learners can be greater in that they receive information of what
specific aspects of their performance should be improved.15
The evaluation of analytic and holistic rubrics to assess student written communication has been published in the pharmacy and
medical education literature.16–19 In evaluation of pharmacy program admission essays, a holistic rubric was found to be more precise
than the more complicated analytic rubric.16 In the discussion, authors state that additions of analytic subsections for evaluating an
essay may not adequately score the whole body of the applicant's essay, and therefore, validity of the assessment suffers.16 Holistic
rubrics may remind raters of the characteristics for evaluation, but does not constrain their scoring, leading to improved inter-rater
consistency and agreement.16 When rating scales were collapsed to four-items in both the holistic and analytic rubrics, the standard
error of measurement (SEM) decreased while inter-rater agreement improved.16 Applying these findings to SOAP note evaluation
tools may be an area of future research to investigate the impact of analytic or holistic scoring elements on assessing student SOAP
notes.
Within medical education, use of analytic scoring of medical student written communication skills may not adequately evaluate
organization or clarity. Analytic scores are typically based solely on the content of the note and may ignore the inclusion of
information that is irrelevant or erroneous.17 However, use of holistic rubrics requires that the scoring categories are well-defined and
raters have sufficient knowledge of the content domain.18 This may prevent use of inexperienced raters such as senior pharmacy
students or pharmacy residents who are often employed in laboratory settings to assist with grading of student assignments. Use of a
K.R. Sando et al. Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
6
“mixed” rubric combining both analytic and holistic scoring elements may be beneficial in overcoming the limitations of each rubric
type.19
While effectively documenting patient encounters is considered a fundamental competency within the health professions, there
are few studies describing valid and reliable methods for assessing student written documentation. In medical training, use of
Fig. 1. Tool for evaluation of documentation.8 OTC = over-the-counter; Rx = prescription.
K.R. Sando et al. Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
7
workplace-based assessments such as reviewing patient charts and engaging in case-based discussions have been used to evaluate
clinical competence.20 Authors of a systematic review evaluating literature on the use of chart-recalls in post-graduate medical
trainees concluded that further research was needed to standardize scoring tools and calibrate faculty members for successful
application of assessment tools.20 Following this systematic review, Baker et al.21 published the IDEA (interpretative summary,
differential diagnosis, explanation of reasoning, and alternatives) assessment toolto evaluate medical students’ clinical skills
following review of a new patient history and physical (H & P) note. The authors provided validity evidence for their tool, but
indicated that it should be used for formative or lower stakes summative uses due to its moderate inter-rater reliability.21 This
suggestion highlights the challenges of producing a valid and reliable tool to evaluate student documentation of patient encounters.
Limitations of this study include the low number of representative SOAP note assessment tools included in the analysis. Thirty-
nine SOAP note assessment tools were submitted from 25 unique schools, which represents approximately 20% (25 of 132
institutional members of AACP) of colleges and schools of pharmacy. While the response rate to our email solicitation was low, the
evaluation tools included did have overlap in structure and information assessed, so it is unknown if including additional evaluation
tools would decrease the variation in criteria assessed. Tools were collected in early 2015, so it is unknown if the tools evaluated in
this study are the most recent versions employed by the institution. It is possible that colleges and schools of pharmacy have adapted
their SOAP note evaluation tools to include items emphasized in the PPCP since it was published in May 2014. The SOAP note
assessment tools were submitted from laboratory faculty through solicitation of the Lab SIG listserv, so generalizability of these
results may be limited to the laboratory setting.
Conclusions
The current investigation provided information of how SOAP notes are assessed in pharmacy education. To the authors’
knowledge, this is the first study to evaluate assessment methods for SOAP notes within pharmacy education. Based on the number of
SOAP note assessment tools included in the study, there appears to be a great deal of variation in the specific elements that are
evaluated in student SOAP notes. While there was agreement with some elements, improved consistency in our assessment methods is
recommended to prepare students to produce standardized documentation when entering practice.
When designing a SOAP note assessment tool, inclusion of specific elements considered essential for pharmacist documentation
should be drawn from literature as well as recommendations of national pharmacy organizations. Educators should consider
inclusion of elements described in the PPCP to promote consistency in pharmacist documentation. For example, collection of a
patient's medication history and factors affecting medication use (e.g., socioeconomic factors, preferences, beliefs, and health goals)
should be included in the subjective portion of pharmacist documentation to communicate this unique aspect of the pharmacists' care
process to the healthcare team. The use of four-item rating scales and combined elements of analytic and holistic rubrics may improve
inter-rater reliability of SOAP note assessment and provide feedback to learners to improve their clinical documentation. Areas of
future research include creation of a standardized SOAP note rubric using recommended elements from this paper with subsequent
reliability and validity testing in a variety of institutions. Evaluation of whether standardizing assessment of student SOAP notes
within pharmacy schools promotes improved consistency in pharmacist documentation upon entering practice should also be
pursued.
Conflict of interest and financial disclosure statements
Authors report no conflicts of interest or relevant financial relationships in conjunction with the preparation of this manuscript.
This research was not supported by any funding, therefore we have no financial relationships to disclose.
References
1. Patient-Centered Primary Care Collaborative. The patient-centered medical home: integrating comprehensive. Medication Management to. Optimize Patient
Outcomes, 2e. Washington, DC: Patient-Centered Primary Care Collaborative; 2012.
2. American Society of Health-System Pharmacists . ASHP guidelines on documenting pharmaceutical care in patient medical records. Am J Health Syst Pharm.
2003;60(7):705–707.
3. American Pharmacists Association. Medication Therapy Management services: documenting pharmacy-based patient care services. In: Documenting Patient Care
Services. American Pharmacists Association; 2007. Available at: 〈https://www.pharmacist.com/sites/default/files/files/mtm_documentating_pc_services.pdf〉.
Accessed 27 May 2017.
4. Zierler-Brown S, Brown TR, Chen D, Blackburn RW. Clinical documentation for patient care: models, concepts, and liability considerations for pharmacists. Am J
Health Syst Pharm. 2007;64(17) 1851‐18.
5. American College of Clinical Pharmacy . Standards of practice for clinical pharmacists. Pharmacotherapy. 2014;34(8):794–797.
6. Joint Commission of Pharmacy Practitioners. Pharmacists' patient care process; Published 29 May 2014. Available at: 〈https://www.pharmacist.com/sites/
default/files/files/PatientCareProcess.pdf〉. Accessed 27 May 2017.
7. Accreditation Council for Pharmacy Education. Accreditation standards and key elements for the professional program in pharmacy leading to the doctor of
pharmacy degree (“Standards 2016”); Published February 2015. Available at: 〈https://www.acpe-accredit.org/pdf/Standards2016FINAL.pdf〉. Accessed 27 May
2017.
8. Currie JD, Doucette WR, Kuhle J, et al. Identification of essential elements in the documentation of pharmacist-provided care. J Am Pharm Assoc.
2003;43(1):41–49.
9. Kuhle J, Doucette WR. Quality assurance tool for documentation of pharmaceutical care: tool validation. J Am Pharm Assoc. 2001;41:297 Abstract 43.
10. Peeters MJ. Measuring rater judgment within learning assessments—Part 1: why the number of categories matters in a rating scale. Curr Pharm Teach Learn.
2015;7(5):656–661.
11. Weems GH, Onwuegbuzie AJ. The impact of midpoint responses and reverse coding on survey data. Meas Eval Couns Dev. 2011;34(3):166–176.
12. Peeters MJ, Sahloff EG, Stone GE. A standardized rubric to evaluate student presentations. Am J Pharm Educ. 2010;74(9):Article 171.
K.R. Sando et al. Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
8
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref1
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref1
http://https://www.pharmacist.com/sites/default/files/files/mtm_documentating_pc_services.pdf
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref2
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref2
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref3
http://https://www.pharmacist.com/sites/default/files/files/PatientCareProcess.pdf
http://https://www.pharmacist.com/sites/default/files/files/PatientCareProcess.pdf
http://https://www.acpe-accredit.org/pdf/Standards2016FINAL.pdf
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref4
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref4
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref5
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref6
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref6
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref7
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref8
13. Peeters MJ, Stone GE. An instrument to objectively measure pharmacist professionalism as an outcome: a pilot study. Can J Hosp Pharm. 2009;62(3):209–216.
14. Peeters MJ, Serres ML, Gundrum TE. Improving reliability of a residency interview process. Am J Pharm Educ. 2013;77(8):Article 168.
15. Mertler CA. Designing scoring rubrics for your classroom. Pract Assess Res Eval. 2001;7(25):1–10.
16. Peeters MJ, Schmude KA, Steinmiller CL. Inter-rater reliability and false confidence in precision: using standard error of measurement with PharmD admission
essay rubric development. Curr Pharm Teach Learn. 2014;6(2):298–303.
17. Boulet JR, Rebbecchi TA, Denton EC, McKinley DW, Whelan GP. Assessing the written communication skills of medical school graduates. Adv Health Sci Educ
Theory Pract. 2004;9(1):47–60.
18. Regehr G, MacRae H, ReznickRK, Szalay D. Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-
format examination. Acad Med. 1998;73(9):993–997.
19. Peeters MJ. Measuring rater judgments within learning assessments—Part 2: a mixed approach to creating rubrics. Curr Pharm Teach Learn. 2015;7(5):662–668.
20. Al-Wassia H, Al-Wassia R, Shihata S, Park YS, Tekian A. Using patients' charts to assess medical trainees in the workplace: a systematic review. Med Teach.
2015;37(Suppl 1):S82–S87.
21. Baker EA, Ledford CH, Fogg L, Way DP, Park YS. The IDEA assessment tool: assessing the reporting, diagnostic reasoning, and decision-making skills demonstrated
in medical students’ hospital admission notes. Teach Learn Med. 2015;27(2):163–173.
K.R. Sando et al. Currents in Pharmacy Teaching and Learning xxx (xxxx) xxx–xxx
9
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref9
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref10
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref11
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref12
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref12
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref13
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref13
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref14
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref14
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref15
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref16
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref16
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref17
http://refhub.elsevier.com/S1877-1297(16)30218-0/sbref17
	Assessment of SOAP note evaluation tools in colleges and schools of pharmacy
	Introduction
	Methods
	Results
	Discussion
	Conclusions
	Conflict of interest and financial disclosure statements
	References