Alpha testing new technologies within the pharmacy is a concept that is simultaneously alluring and daunting. Access to the newest technological advances may tantalize with promises of advanced system improvements in such areas as workflow, inventory management, and charge capture; however, alpha products, almost by definition, may also threaten or delay the very processes that one aims to improve. Thus, careful consideration is prudent prior to partnering with outside vendors in the evaluation of new technologies. Creating a successful partnership between the pharmacy department and the vendor to develop an alpha testing process requires clear communication and thoughtful preparation.

Making the Case for Alpha Testing
Most technological improvements involve sizable capital expense and/or expensive recurring contractual requirements, as an alternative, alpha testing can offer a more cost effective approach to achieving these same goals. Because in-house testing of a fully developed product prior to acquisition is generally limited or, as is the case with large devices such as carousels or robots, simply impossible, pharmacy is often forced to rely on vendor demonstrations—which may not accurately reflect real world conditions and requirements—or the experiences of colleagues in other hospitals that may not have equivalent operational models or departmental demands. As a result, pharmacy generally purchases (or leases) new products as is, with little opportunity for design improvement or modification when unforeseen challenges arise. This arrangement can pose unnecessary financial risk to already cash-strapped and closely scrutinized pharmacy department budgets. Conversely, alpha testing a product within the hospital, either in a simulated or carefully controlled live setting, provides pharmacy with the opportunity to understand how the product will perform in a real world setting and, perhaps even more important, to impact the design improvement process so that the final product delivers the features that are most key to the institution.

At the same time, such testing provides invaluable data to the product developer. Modern pharmacy departments exhibit great variability in practice models, available resources, staffing limitations, order volumes, and general workflow. When faced with such variety, developers, particularly those with backgrounds outside of pharmacy, may find it exceptionally challenging to accurately predict the real world demands that will influence the product’s performance and impact the users’ experience. Alpha testing within a hospital pharmacy, therefore, can be of great value to the product developers also.

Designing the Testing Process
At the University of North Carolina Hospitals (UNC), we chose to alpha test a new handheld medication scanning device with order tracking software, as our existing handheld devices were unable to provide real time order location information and would not allow us to transition to a fully electronic documentation system. After completion of the necessary pre-testing documentation, including non-disclosure agreements and patient privacy protections, hospital staff worked with the order tracking technology vendor, PlusDelta Technologies, to design a multi-phase testing process. Crucial to this design phase was the involvement of key stakeholders, including representatives from the hospital’s IT department, as well as pharmacy department managers, pharmacists, and technicians responsible for the testing areas. Involvement of each of these groups allowed for an effective evaluation of needs prior to the initiation of testing, as well as a thorough analysis of the potential impact of the testing process on pharmacy workflow. Included in this analysis was identification of the optimal initial site, department personnel, and time of day for testing to ensure that any impact on workflow was minimized. As a wireless device was to be the focus of the testing, we conducted a compatibility comparison of our facility’s wireless capabilities to those of the product. In addition, we had detailed discussions to determine the device’s basic compatibility with the pharmacy department’s labels and bar codes.

After receiving input from all relevant parties, we designed the initial testing process to occur in two phases. The first phase consisted of a three day period of testing on an extremely limited basis, with day one dedicated to an initial evaluation of the devices, training of select staff and amelioration of test-limiting bugs, such as discrepancies between bar code standards and workflow limitations that precluded scanning at certain pre-specified scanning points. The second and third day of testing were split into multiple testing sessions, each devoted to different aspects of the pharmacy workflow such as the preparation of IV medications, patient-specific or bulk packaged medications and remote medication storage device restocking, oral liquid dose syringe filling, and stat doses. During this phase, basic functionality of the device was evaluated, including bar code scanning, and bar code tracking of order preparation, verification, and delivery status.

The second phase of testing was held one week after the initial phase to allow the vendor an opportunity to respond to concerns that arose during the initial testing period. During this second phase, we increased the number of doses involved in the testing and decided to incorporate a satellite specialty pharmacy (one with slight differences in workflow) into the testing process. A third phase of testing, involving full implementation and use of the handheld device over the course of several weeks in a single location, was designed to occur several weeks later, again allowing time for product improvements to be addressed.



Averting Negative Impacts on Workflow and Patient Safety
Of foremost concern throughout the testing process was the potential impact the testing could have on patient safety and workflow within the pharmacy. Significant delays in patient access to care were not acceptable, nor could patient safety be compromised. Thus, we instituted several safeguards, including the assignment of an additional pharmacy administration resident familiar with the testing area’s staffing practices and requirements. The resident coordinated the testing process and served as the main point of contact for the vendor. By assigning this resident we enjoyed the additional benefit of having an extra staff member on hand to offset any workflow impact from a device still under development.

Areas where testing could potentially effect workflow were carefully isolated to prevent larger operational interruptions. For example, orders that were assigned to be run through the testing process were physically separated from the main order volume and a dedicated pharmacist and pharmacy technician were assigned solely to the testing process of managing these orders. In addition, a maximum acceptable delay (or queue length) was established prior to the initiation of testing and was communicated to staff within the effected work areas. When the established queue length was reached, testing was temporarily halted to allow staff to return to standard operational status, thus minimizing delays in patient care.

Vital to the success of the testing was early and frequent communication with the hospital’s IT services department. As the handheld devices utilized our existing wireless networks for communication with vendor servers, naturally there were concerns regarding the implications of untested hardware and software accessing the hospital network. Given the increasingly vital role hospital networks play in patient care, external threats to stability and functionality should be avoided when possible to preclude endangering patient care. It would be reasonable to assume that alpha stage software may contain a multitude of coding errors or omissions, any of which could potentially create disruptions for a hospital network. Of course, issues of patient confidentiality and security of proprietary information are of foremost concern when network access is provided to vendors. Due to transparent and open communication between IT, pharmacy, and the vendor, the vendor was able to make minor modifications to their software that allowed for a combination of off-line simulation of a live environment and the use of a publicly available wireless network for secure transfer of de-identified data. This approach precluded the need for a vendor-developed interface with the internal hospital network during the testing process. While this method created a somewhat greater workload during the manipulation of testing data on the front end, vendor exposure and access to sensitive patient data was minimized and use of the hospital’s critical internal network was avoided. The major limitation of this approach was that while integration of the device into workflow was tested, the full impact and functionality of the software could not be evaluated. Greater need for full connectivity is to be expected as testing progresses into beta stages.

Creating Staff Buy-in
Throughout the process, we took great care to inform pharmacy staff of both the purpose of the testing and the potential long term uses and benefits of the product within the department. Not only did such communication increase the number of individuals who were able to provide valuable feedback on the device, it also served to mitigate anxiety generated by the process. The introduction of new technologies can be exciting, but often simultaneously raises concerns. Furthermore, veteran staff may be reluctant to embrace new technology, particularly when it will involve substantial modification to existing workflow patterns. Additionally, because the handheld devices being tested tracked order time and accuracy as a component of their functionality, concerns were raised regarding potential job performance measurement or position obviation. To address these concepts, we openly discussed the process with staff during the testing periods, but limited actual involvement and training to a few select individuals that would be directly involved. In addition to reducing anxiety resulting from perceived performance measurement, limiting personnel involvement had the added benefit of minimizing workflow interruption and thus also minimizing any potential impact on patient care. By involving staff members early in the process and highlighting the potential benefits of the technology, a situation that easily could have damaged morale, became an opportunity for morale improvement. Indeed, several staff members expressed excitement about the devices and indicated their appreciation that a clear effort for inclusion had been made.

Building Lucrative Partnerships
For the vendor, the ultimate goal of an alpha testing process, besides the introduction of services to a new client base, is the identification of areas in need of improvement as well as areas for functionality growth. In our case, a vendor representative was on hand for the majority of the testing period and was allowed to observe the pharmacy’s workflow processes in detail. In addition to the data collected from the handheld devices, the vendor received regular updates and a summary document of departmental suggestions for product improvement at the completion of the testing process. Implementing the technology in a real world setting revealed numerous areas of programming and process deficiencies, while multiple testing periods, with sufficient time between them, allowed the vendor to address concerns raised during those periods and remedy those deficiencies. In response to staff concerns, the number of scans required to transition an order out of the pharmacy was reduced and portions of the pharmacist verification process were automated. The original software required a pre-specified database of users. Because we wanted a system to not only track order delivery, but also to track the individual accepting delivery of the order, every possible end user would have needed to be included in that database. As most of the individuals who ultimately receive medication orders at our hospital are not pharmacy department employees, maintaining such a database would have been prohibitive. Instead, the vendor was able to adjust their program to add new users on the fly with a single ID badge scan and a signature, allowing delivery even to temporary employees and individuals with no prior experience with the devices.

Due to the potential implications to patient care, testing a product live in the pharmacy may seem daunting. However, with thoughtful preparation, alpha testing can be performed in a controlled, safe manner. While device size and software complexity will determine the feasibility of such a project, partnering with a vendor to provide alpha-testing services allows for the creation of a jointly lucrative venture. Such testing offers ground floor evaluation of new technologies, can be performed in a relatively cost-free format, and not only delivers an early estimation of potential technological value for the department, but also has the added benefit of enabling a concurrent examination of existing workflow. Ultimately, pharmacy’s design improvement suggestions can yield a final product that complements the key needs of your institution. In our experience, a controlled roll out of the testing process minimized workflow impact, yet provided adequate opportunity for product evaluation and appraisal. The boost in morale that resulted from pharmacy staff experiencing and expressing appreciation for the effort to include them throughout the process proved to be an unexpected benefit for management. All of these factors in combination may significantly increase the potential attractiveness of such projects to hospital leadership.

Ian R. Willoughby, PharmD, is a PGY2 Pharmacy Administration resident at University of North Carolina Hospitals. He received his doctoral degree from the UNC Eshelman School of Pharmacy in Chapel Hill, North Carolina. Ian also completed a PGY1 in Pharmacy Practice residency at UNC Hospitals.

Robert Granko, PharmD, MBA, is the associate director of pharmacy at the University of North Carolina Hospitals. He graduated with his Bachelor of Science in Pharmacy from Long Island University, School of Pharmacy. Robert received his Doctor of Pharmacy from the University of North Carolina at Chapel Hill and his Master of Business Administration from Pfeiffer University, School of Graduate Studies.