# I. Health Informatics Integrated System Post-Implementation Evaluation hen one speaks of a successful health information technology(HIT) system implementation, there are several dimensions that go into determining that success. While the satisfaction of the workforce is very important, it is only one dependent factor tied to how well the healthcare process has succeeded. How well current practices are redesigned to take advantage of the technologyis a factor. Quality of the data is another influence. Confidence in the documentation and the information it contains us an important aspect. How a system will work through barriers and enable facilitators are other dimensionsof a success implementation. Measurement of improvement to patient care is another facet. So many characteristics go into determining a successful implementation. Finding the right instruments to put into position before, during and after an HIT system implementation is an ongoing task that continually needs to be evaluated. As with the integrated systems, implementation standards need to be studied and enhanced to strive for even better success. A well designed process should allow for success that is on par or surpasses the importance of the former. # II. Background A little over a decade ago, the Institute of Medicine put forward that improved patient safety, efficiency of health care delivery competences and quality of care would be realized by make use of an effective integrated HIT(Crossing the Quality Chasm: A New Health System for the 21st Century, 2001). More recently, government incentives and mandates have been placed on healthcare institutions advocating for their adoption of HITsystems (DHS, 2010). While there are legislative whys and wherefores that go into the need for an HIT system, the drive to have a system that helps the patient and staff needs to be the driving force in the desire to find mechanisms which encourage a positiveand effectiveapplication. The purpose of the research topic of interest is to identify elements necessary for a successful HIT system implementation at acute care hospital sites. The research study will help determine what critical elements are necessary to have in place in order for healthcare facilities to have a successful transition from an older medical record system to a new electronic medical record (EMR) system. The research study will evaluate what operations should be set in place by healthcare facilities before transitioning to an HIT system. Moreover, the research will focus on possible ways to prevent issues that may develop during and from the implementation of the new electronic system. The study will survey employees of healthcare facilities who have already transitioned to an HITsystem and examine how they believe the implementation process could be been improved.Furthermore, barriers to a successful HIT system implementation will be attempted to be identified. As a final point, information found in the study will be used to synthesize material and identification of possible gaps in research. Information found in the literature review was employed to integrate data and identify gaps in present research such as the need for greater variety of positions giving feedback. While several of the recommendations for successful implementation were similar, some studies had opposing views of nurses' attitudes after implementation. The type of support by the healthcare facility before and after implementation may have been a factor in these findings. Moreover, in the majority of studies, nurses were the population studied and findings were based on these responses. Although in all five articles the implementation of a comprehensive HITsystem was being evaluated, rarely was health care personnel who work outside of direct patient care evaluated. No staff within areas such as admissions or billing was interviewed (Table 1). # III. Methodology The methodology of the research study is partedinto its research design, population, and data collection procedures. Additionally, the suitable data collection instrument is determined based on the research design and population. Applied to the study will be the appropriate data analysis. A prospective post-implementation survey was used as the research method on the comprehensive HIT system within the facility healthcare system. The intent of the design was to help describe the current views of the healthcare staff in relation to the quality of the system, the implementation and its current operation. Research was conducted at two acute care hospitals that recently rolled out the EMR system within the last year. The study population was end usersof the integrated system within the Continuum of Care departments ofacute care hospital sites in Temple, Texas. The first facility is a 64-bed pediatric specialty care and teaching hospital. The second is a 636-bed specialty care and teaching hospital. The health information technology employed was the commercial software system, Epic. The execution of the research study used the direction laid out in Health Informatics Research Methods: Principles and Practice (Layman, 2009). # a) Data Collection Procedures Data collection was performed by anonymous submission online via REDCap(REDCap, 2009). Notification was given through the employer email system with permission from management. A cover letter was included stating participation was voluntary and not part of an institutional initiative (Figure 1). After one week, a reminder email was provided to the same staff. At the end of fourteen days, the link to survey was ended. # IV. Results The following results describe the response rate and break down the demographics of the respondents. # a) Response Rate of Population The response rate was determined to be 37.78%. One hundred seven possible respondents were emailed a cover letter and link to the autonomous website. Again, one week later the same cover letter and link were emailed to the same one hundred and seven staff members. The link was terminated one week later. In total, thirty-four valid surveys were completed. # b) Representativeness of Population The staff ranged in age from younger than twenty-five to greater than sixty-six. The largest number of respondents was present in the fifty-six to sixty-five year age range (32.4%). The majority stated their computer proficiency as average (61.8%) and had prior EMR experience (55.9%). (Figure 5, 6 & 7; Table 3). # c) Research Questions In developing an understanding of the attitude of the staff, the quality of the system, its information and the service provided regarding the HIT system were measured. Additionally, the particular aspects of the clinical data were analyzed. A five-level Likert scale was utilized to measure the employee'sstance on the quality of the HIT system, the information within the HIT system, the service provided to support the HIT system and particular aspects related to the clinical information of the HIT system. In regard to the quality of the system, a majority of the staff strongly agree that the system is consistently available (47.1%) and has acceptable security (50%). As for the system appropriatelyintegrating with previous workflows, the employees were mostly divided between mildly agree (26.5%), moderately agree (29.4%) and strongly agree (29.4%). None of workers disagreed in a majority to any of the aspects measured related the quality of the system. The remainder moderately agreed that the system was easy to use (70%), its performance was reliable (44.1%), had acceptable response time (47.1%), provided effective communication between team members (41.2%), had acceptable exchange of information with other systems (38.2%) and enabled staff to perform work well (38.2%). (Figure 8; Table 3) The criteria measured related to the system's information was mostly seen as moderately agreeable. A majority of the staff moderately agree that the information is accurate (52.9%), relevant (47.1%), complete (47.1%) and has an acceptable layout (41.2%). An even number moderately agrees (41.2%) as strongly agree (41.2%) that the information is available when needed. (Figure 9;Table4) In the three characteristics of service measured, a majority of staff moderately agreed that the implementation process (55.9%), level of training (47.1%) and on-going support (47.1%) is acceptable. (Figure 10; Table 5) Because most of the respondents do not work directly with the patients, the majority answered that they were not sure of the patient's satisfaction with clinicians' use of system (35.3%) or patient's concerns with system security and confidentiality (41.2%). A majority strongly believe that the clinical data has improved patient outcomes (41.2%), improved patient safety (41.2%), improved patient's knowledge of their health (38.2%) and improved clinical documentation (38.2%). A majority moderately believe the clinical data of the patient is accurate and valid (44.1%), the timely manner of the patient care services has increased (35.3%) and that there is an appropriate selection of patient care orders (35.3%). (Figure 11; Table 6) The standard deviation of the criteria within the four quality themes were calculated and presented within Table 7. Within Table 8, cross tabulations are provided based on prior EMR experience vs. each of the acceptability of quality of the system, information, service and clinical data. The number of staff with prior EMR experience (N=19) slightly outnumbered the staff with no prior experience (N=15). Having experience with an EMR system or having no experience did not appear to affect the acceptability. In the four measures, the respondents in both groups found the quality of the system, its information, its service and specifically the clinical area all moderately acceptable. From the four core categories, each quality set's comment section was reviewed for common themes applicable to productive transition of HIT systems. Within the system qualityfocus, interoperability between modules within the system and to other systems is a noted concern of staff. As one respondent stated, "communication in the system is available but isn't utilized as well as possible." Another staffer mentions that the system "doesn't consistently interface properly with Midas." (Figure 12)For the information quality, an issue raised was the inability to access information. The view of a Case Manager is different than that of a nurse which brings concern that information is not being interpreted in the same manner (Figure 13). Figure 9Figure 12 Figure 13 The largest numbers of concern are in relation to the service quality. One of the concerns is the training was not specific enough for particular job titles. An example given was a class attended by a Case Manager but included staff from the Admissions department. The class was taught using a task list for the Admissions department which was a "different view and way to enter" thesystem's authorization module. The Case Management felt the "class was not tailored enough" for their department. The same concern was noted by a staff member who not employed during the implementation but came after. She felt the training was inappropriate for her job description. Along the lines of training, it was mentioned for "more training services on over all process of Epic flow of documentation of a patient." (Figure 14) The staff to be unsure of how the system's modules are interconnected. Lastly, concerns were stated in regard to the timeliness of resolving issues. "IT is slow to respond and resolve issues when they arise" was the comment of one employee. For the final quality measure, the statements The survey concluded with more general questions related to the implementation process. The topics mentioned by the staff tended to reflect the appropriate training of staff with statements such as "training should have been more specific to my job" and "educate staff thoroughly to obtain the best results. Benefits stated my respondents were more in relation to the system such as "work flow is improved" and "f aster easier access to information." (Figure 16) # V. Discussion The significance of the results continues to help develop critical elements necessary for a successful transition to a new comprehensive system. The study focused on the end users' beliefs regarding the quality of the system and particularly, its information and service. Areas of enhancement were revealed included improving training specific to job and supplying more fitting integration of processes and workflows. Likewise, confirmatory aspects of current procedures were observed throughout the study. After the implementation, a greater part of the respondents appreciated many of the aspects of having the new technology such as the ease of use, the ability to access to documents within one system and timeliness of information. to make progress, staff will need motivation. Motivation is provided by allowing choices, knowledge and connection to the progress. As an illustration, the barrier of providing jobspecific training could be tackled. Addressing the goal of job-specific training would acknowledge the staff concerns. Providing acknowledgement and recognizing the concerns will engage the personnel. Respond quickly with a plan of action will continue the commitment. Finally, provided feedback will continue the support of a positive transition. In close, understand the critical elements to support positive HIT transitions are essential but the continued engagement of end users is also vital. Before, during and after implementation, healthcare personnel need to feel competent and related to the transition. Two future studies are recommended. First, a study could be developed to correlate staff engagement to positive HIT changeovers. The second would still be covering the gap in present research which continues to be the need for greater variety of positions giving feedback. # VI. Conclusions The subsequent conclusions and recommendations will provide a summary of findings. Along with the findings, conclusions related to the implications to a positive implementation process related to the study and previous studiesare provided. The participants were employed within the Continuum of Care departments of two acute care inpatient facilities. The majority of respondents declared themselves to be Case Management staff. This group includes RN Case Managers and Social Workers. The remaining staff was administrative support staff or management staff of the Continuum of Care departments. The quality of the four areas of focus all was seen in a largely positive light. Over eighty percent of the respondents moderately or strongly agreed that the system was easy to use, had reliable performance, was consistently available and had an acceptable response time. While acceptable response time did have a ninetyseven percent positive response, one staffer did moderately disagree. Two other areas did contain responses that ranged from strongly agree to moderately or strongly disagree which were the acceptability of information exchange with other systems and the appropriate integration of previous workflows. As the system information as a whole and the clinical information surveyed individually, the workers replied a mostly affirmative response or stated that they were unsure. Most felt the information was relevant, accurate and had an acceptable layout. A small minority mildly disagreed the information was complete (2.9%) or available when needed (6.9%). Within the clinical quality survey questions, the response of "Not Sure" was selected than any of the four quality specific areas. From the comments given by the respondents, this was due to the staff not working directly with the patients. Still, a majority strongly believed that the system had provided improved patient outcomes, patient safety, patient knowledge of their health and improved clinical documentation. While the quality of service still received mostly agreeable responses, it provided the large number of comments of concern by the respondents. Although the majority of survey takers moderately agree the implementation process, level of training and on-going supports were acceptable, the three questions also had responses that included mild, moderate and strong disagreement. The primary issue noted appeared to be centered on job-specific training. Whereas the remarks did convey a desire to better understand the overall process of Epic, the many staff members mentioned the need for training related to "addressing case management." One employee mentioned that there were "many questions and frustrations expressed in classes and for a few months after" because "when (the staff) first took Epic training, it did not relate to what they did." (Figures 5 and 6). Similar to previous studies, some of the same topics were observed in this study. As with other studies, the implementation process appeared to provide a mostly encouraging transition witha small number of components noted of concern to the staff. Similar to the study in "Transitioning from a computerized provider order entry and paper documentation system to an electronic health record: Expectations and experiences of hospital staff", positive characteristics observed included the quality and safety of patient care. Readily available allinclusive clinical documentation and the ability to locate patient demographic information quickly were additional benefits of transitioning. Moreover, conceivable enhancements for future implementations were illustrated with the recent study. One feature of greater apprehension was highlighted by staff with two other concerns of smaller notation. As mentioned in the article "Learning from Within to Ensure a Successful Implementation of an Electronic Health Record", the few of the staff within the current studyexpressed the similar need for further attention to processes and workflows within the new HIT system. Another minor concern was improving the exchange of information with other systems. More than an ability that can be imparted to the staff during the transition process, the implementation of this the element may be a requirement on the quality of the system itself. The greatest concern appears to be appropriate staff training. While an understanding of the overall structure of Epic is wanted, a focus on more job-specific training was repeatedly articulated. In summary, the critical elements essential for a successful transitionemerging from the study appear to include appropriate training, attention to incorporating processes and workflows, swift feedback to questions and concerns and attention to the staff impression and opinion regarding the HIT system and its implementation. ![Sajeesh Kumar PhD ? , Anne L. Pedigo ? RN MSN ? Medical Research © 2015 Global Journals Inc. (US)](image-2.png "W") 6![XV Issue III Version IYear 2015](image-3.png "6 Volume") 1283![Figure 1 : Cover Letter introducing Epic System Post-Implementation and Use Assessment Survey](image-4.png "Figure 1 :Figure 2 : 8 VolumeFigure 3 :") 5![Figure 5 : Pie Chart of Age Range](image-5.png "Figure 5 :") 6![Figure 6 : Computer Proficiency Frequency](image-6.png "Figure 6 :") 7![Figure 7 : Prior EMR Experience Frequency](image-7.png "Figure 7 :") ![Figure 8](image-8.png "") ![Figure 10](image-9.png "") ![Figure 11 ](image-10.png "") ![III Version IYear 2015 indicated that the staff was unsure because they did not deal with patients directly (Figure15).](image-11.png "") 1415![Figure 14](image-12.png "Figure 14 Figure 15") ![Figure 16 Survey General Comments](image-13.png "") ![associated healthcare facilities located in one city in central Texas. Moreover, the questionnaire was limited to responses from same type of department within the two hospitals. The responses were limited to staff that do not have access to patient care as a routine part of their job responsibilities. Lastly, the fear of participating in survey may have limited the response. Disbelief in true anonymity may have limited or swayed respondents in their scoring or comments. Key limitations of the study should to beunderscored. The study was conducted at two The resulting recommendations are focused on fostering staff engagement Taking guidance from a lecture presented by Rod Brace (2014), "The Science of Engagement", engagement is correlated to making progress. As part of progress, there needs to be clarity of goals, a feasible challenge and feedback on actions. But Volume XV Issue III Version I © 2015 Global Journals Inc. (US)](image-14.png "") ![](image-15.png "") ![](image-16.png "") ![](image-17.png "") ![](image-18.png "") ![](image-19.png "") ![](image-20.png "") ![](image-21.png "") ![](image-22.png "") ![](image-23.png "") ![](image-24.png "") ![](image-25.png "") ![](image-26.png "") ![](image-27.png "") ![](image-28.png "") ![](image-29.png "") 1 3ProfessionCumulativeFrequencyPercentValid PercentPercentValidAdministrative Support Staff25.95.95.9Case Management Staff2985.385.391.2Other38.88.8100.0Total34100.0100.0Age RangeCumulativeFrequencyPercentValid PercentPercentValid25 or younger38.88.88.826 to 35411.811.820.636 to 45926.526.547.146 to 55411.811.858.856 to 651132.432.491.266 or older38.88.8100.0Total34100.0100.0Computer ProficiencyCumulativeFrequencyPercentValid PercentPercentValidBasic411.811.811.8Average2161.861.873.5Advanced823.523.597.1Expert12.92.9100.0Total34100.0100.0Prior EMR ExperienceCumulativeFrequencyPercentValid PercentPercentValidYes1955.955.955.9No1544.144.1100.0Total34100.0100.0© 2015 Global Journals Inc. (US) 4System -Easy to UseCumulativeFrequency PercentValid PercentPercentValidStrongly Agree617.617.617.6Moderately Agree2470.670.688.2Mildly Agree25.95.994.1Mildly Disagree25.95.9100.0Total34100.0100.0System -Reliable PerformanceCumulativeFrequency PercentValid PercentPercentValidStrongly Agree1338.238.238.2Moderately1544.182.4Mildly Agree514.714.797.1Mildly Disagree12.92.9100.0Total34100.0100.0© 2015 Global Journals Inc. (US) K Volume XV Issue III Version I 4Information -RelevantCumulativeFrequencyPercentValid PercentPercentValidStrongly Agree1441.241.241.2Moderately Agree1647.147.188.2Mildly Agree411.811.8100.0Total34100.0100.0© 2015 Global Journals Inc. (US) 5Service -Acceptable Implementation ProcessCumulativeFrequencyPercentValid PercentPercentValidStrongly Agree720.620.620.6Moderately Agree1955.955.976.5Mildly Agree38.88.885.3Mildly Disagree25.95.991.2Moderately Disagree12.92.994.1Strongly Disagree12.92.997.1Not Sure12.92.9100.0Total34100.0100.0Service -Acceptable Level of TrainingCumulativeFrequencyPercentValid PercentPercentValidStrongly Agree617.617.617.6Moderately Agree1647.147.164.7Mildly Agree823.523.588.2Mildly Disagree25.95.994.1Moderately Disagree12.92.997.1Strongly Disagree12.92.9100.0Total34100.0100.0© 2015 Global Journals Inc. (US) 6Improved Patient OutcomesCumulativeFrequencyPercentValid PercentPercentValidStrongly Agree1441.241.241.2Moderately Agree1132.432.473.5Mildly Agree617.617.691.2Not Sure38.88.8100.0Total34100.0100.0Improved Safety of PatientFrequencyPercentValid PercentValidStrongly Agree1441.241.241.2Moderately Agree1235.335.376.5Mildly Agree617.617.694.1Not Sure25.95.9100.0Total34100.0100.0Improved Knowledge of Health by PatientsCumulativeFrequencyPercentValid PercentPercentValidStrongly Agree1338.238.238.2Moderately Agree1029.429.467.6Mildly Agree411.811.879.4Mildly Disagree25.95.985.3Not Sure514.714.7100.0Total34100.0100.0Patient Satisfied w/ Clinicians Use of SystemCumulativeFrequencyPercentValid PercentPercentValidStrongly Agree720.620.620.6Moderately Agree823.523.544.1Mildly Agree617.617.661.8Mildly Disagree12.92.964.7Not Sure1235.335.3100.0Total34100.0100.0Concerns w/ Security & Confidentiality by PatientsCumulativeFrequencyPercentValid PercentPercentValidStrongly Agree411.811.811.8Moderately Agree25.95.917.6Mildly Agree38.88.826.5Mildly Disagree617.617.644.1Moderately Disagree38.88.852.9Strongly Disagree25.95.958.8Not Sure1441.241.2100.0Total34100.0100.0© 2015 Global Journals Inc. (US) 7Descriptive StatisticsNMinimum MaximumMeanStd. DeviationSystem -Easy to Use34142.00.696System -Reliable Performance34141.82.797System -Consistently Available34141.68.768System -Acceptable Response Time34151.76.855System -EffectiveCommunication b/t Team34142.00.888MembersSystem -Acceptable Exchange Information w/ Other Systems34172.651.668Volume XV Issue III Version I 30 Year 2015System -Integrated Appropriately w/ Previous Workflows Security Information -Relevant Information -Accurate Information -Complete Information -Acceptable Layout Information -Available When Needed Service -Acceptable Implementation Process Service -Acceptable Level of Training Service -Acceptable On-Going Support System -Acceptable System Perform Work Well System -Enables Staff to34 34 34 34 34 34 34 34 34 34 341 1 1 1 1 1 1 1 1 1 17 7 3 7 7 3 4 7 6 3 72.50 2.06 1.71 1.88 2.03 1.82 1.82 2.35 2.38 1.94 2.031.581 .676 1.094 1.167 .758 .869 1.390 1.129 .736 1.705 1.167( D D D D ) KNDescriptive Statistics Minimum MaximumMeanStd. DeviationMedical ResearchClinical -Improved Knowledge of Health by Patients Clinical -Patient Satisfied w/ Clinicians Use of System Clinical -Concerns w/ Security & Confidentiality by Patients Clinical -Patient Care Date is Accurate and Valids34 34 34 341 1 1 17 7 7 72.59 3.79 4.88 2.502.047 2.508 2.185 1.796Global Journal ofClinical -Timely Manner of Patient Care Services Increased Clinical -Appropriate Selection of Patient Care Orders Clinical -Improved Clinical Documentation Valid N (listwise)34 34 34 341 1 17 7 72.97 2.82 2.382.209 2.081 1.875© 2015 Global Journals Inc. (US) 8CountPrior EMR ExperienceYesNoTotalAcceptability of the Quality ofHighly Acceptable7714the Epic SystemModerately Acceptable9716Neither Acceptable nor Unacceptable213Moderately Unacceptable101Total191534Acceptability of the Quality of the Information Provided in Epic * Prior EMR ExperienceCountPrior EMR ExperienceYesNoTotalAcceptability of the Quality ofHighly Acceptable8816the Information Provided inModerately Acceptable10616EpicNeither Acceptable nor Unacceptable112Total191534Acceptability of the Quality of the Services Provided for Epic * Prior EMR ExperienceCountPrior EMR ExperienceYesNoTotalAcceptability of the Quality ofHighly Acceptable7411the Services Provided for EpicModerately Acceptable7714Neither Acceptable nor Unacceptable325Moderately Unacceptable224Total191534Acceptability of the Clinical Data within Epic * Prior EMR ExperienceCountPrior EMR ExperienceYesNoTotalAcceptability of the ClinicalHighly Acceptable7512Data within EpicModerately Acceptable8816Neither Acceptable nor Unacceptable426Total191534 © 2015 Global Journals Inc. (US) * The Science of Engagement. Present at the Memorial Hermann Case Management Retreat RodBrace 2014. April 11. 2014 * Canada Health Infoway System And Use Assessment Survey CanadaHealth Infoway 2007 * Crossing the Quality Chasm: A New Health System for the 21st Century 2001 The National Academies Press * HumanDepartment Of Health Services Health Information Technology: Initial Set 75 144 2010 Federal Register * Ibm Spss Statistics Statistics Coach. Retrieved from Help section of IBM SPss Statistics 2013 * ESKirkendall LMGoldenhar JLSimon DSWheeler SSpooner 2013 * Transitioning from a computerized provider order entry and paper documentation system to an electronic health record: Expectations and experiences of hospital staff International Journal of Medical Informatics * Learning From Within to Ensure a Successful Implementation of an Electronic Health Record ASLaramee MBosek CAKasprisin TPowers-Phaneuf CIN: Computers, Informatics, Nursing 30 2012 * A Comparison of Nurse Attitudes Before Implementation and 6 and 18 Months After Implementation of an Electronic Health Record AS MLaramee A.-BBosek MD R NShaner-Mcrae HD R FPowers-Phaneuf TM R CIN: Computers, Informatics, Nursing 30 10 2012 * Health informatics research methods: Principles and practice Layman, E., & Watzlaf, V. 2009 American Health Information Management Association Chicago, IL * PaulAHarris RobertTaylor RobertThielke JonathonPayne NathanielGonzalez GJose * Research electronic data capture (REDCap) -A metadata-driven methodology and workflow process for providing translational research informatics support Conde J Biomed Inform 42 2 2009 Apr * A new instrument for measuring clinician satisfaction with electronic health records PSSockolow JPWeiner KHBowles HPLehmann 10.1097/NCN.0b013e31821a1568 Comput Inform Nurs 29 10 2011 * What determines successful implementation of inpatient information technology systems? JSpetz JFBurgessJr CSPhibbs American Journal of Managed Care 18 3 2012 * Nurses' Perceptions of How Clinical Information System Implementation Affects Workflow and Patient Care MMWard SVartak TSchwichtenberg DSWakefield 510.1097/NCN Computers Informatics Nursing 29 9 2011. 1090b101 3e31822b38798