Summary
Sleep ISR was developed by the AASM in 2010 to help sleep facilities meet the inter-scorer reliability requirement for accreditation. At its core, this system posts a new sleep record consisting of 200 epochs each month that technologists can score, comparing their results to those of a gold standard panel. There are currently more than 8000 individuals belonging to more than 2000 facilities actively using this web application. Sleep ISR was completely overhauled in late 2018 to improve the scoring functionality and modernize the administrative dashboard.
Sleep ISR consists of two separate applications, an administrative dashboard and a sleep study scoring system. Written in VB.NET using version 4 of the .NET Framework, these web applications follow a traditional n-tiered architecture. All database interactions occur within the data layer using ADO.NET and stored procedures. ASP.NET Membership and Role Management are incorporated for user validation and authorization. The sleep record scoring system is primarily a client side application built with JavaScript and ASMX web services.
Improved Layout
The overhaul began with drastic changes to the look and feel of the administrative dashboard. The layout was converted to a customized bootstrap implementation, and the entire site was transitioned away from using an outdated version of Telerik UI. Considering this is a web forms application running on version 4 of the .NET framework we chose to use Update Panels to limit full post backs and increase app responsiveness. Update panels allowed us to easily transition from the previously used Telerik RadAjaxPanels with minimal effort. Getting acquainted with this older technology turned out to be a lot of fun.
To maintain backward compatibility to IE 8, we chose Bootstrap 3 along with respond.js for this project. Several sleep facilities that rely on Sleep ISR still use older browsers, so maintaining compatibility was an extremely high priority.
Updated Authorization Model
The system was previously split into two account types, facilities and individuals. A facility account was required to purchase subscriptions, while individual accounts were required to score records. A facility account would purchase either annual subscriptions or individual credits to give its staff access to score records and would manage staff access to these subscriptions. Over time it was apparent that this model created several problems. If a person managed multiple facilities, they would need a separate facility account for each. If a facility administrator wanted to also score records, they’d need a facility account and an individual account. If an individual that was not linked to any facility wanted access to score records, they would need to create a dummy facility account to purchase a subscription. This model needed to be redesigned to simplify registration and subscription purchasing.
We solved this problem by converting all accounts to individual type accounts and following a permissions based approach. Facilities are no longer a separate account type requiring a separate login. Individuals are given administrative permissions to a facility when they create that facility. Furthermore, administrators can assign administrative privileges to other staff that are linked to the facility. Each facility can now have multiple administrators, and each individual can now be an administrator for multiple facilities. Subscriptions have also been expanded to include individuals eliminating the need to create dummy facilities just to gain access to score records. Overall, the registration and subscription process was greatly simplified.
The transition to this model was a difficult one. Ideally, we would have automatically merged duplicate accounts, but this was not feasible. Each account has a different username, password, contact information, etc., and we have no way of determining the preferred data for the merged account. Our solution was to offer a merge tool and to detect potential duplicate accounts the next time the user logs in providing a notification within the user’s dashboard. This merge tool allows the user to select the preferred username, address, email, phone number, etc. of the merged account before completing the process. The user verifies ownership of the duplicate accounts that will be merged by entering the username and password for each, and they confirm the merge by entering a new password for the merged account. After merging, they can manage their staff for all of their facilities and score records from a single account.
Improved Notification System
Sleep ISR traditionally relied solely on emails for all platform notifications. An external email was sent for each major event, including staff requests, pass/fail notifications, new record notifications, upcoming subscription expirations, and low credit notifications. None of these notifications were present within the dashboard, and the opt-out feature for the external emails was extremely minimalistic. A user could opt-out of pass and fail notifications individually, but that was the extent of the opt-out granularity.
As part of the upgrade, the notification system was completely overhauled. All notifications are now present within the user dashboard in the form of either simple notifications or alerts. Any notification that requires user interaction (e.g. subscription expiration, staff requests, etc.) is presented as an alert with an action link that the user can follow to complete the task. All other simple notifications are listed beneath a notification bell icon, and a page was created to allow the user to review all past notifications. Each alert triggers an external email to the user, and the user now has the option to granularly opt-out of these emails.
Improved Reporting
Sleep ISR previously relied on Telerik to generate reports. Instead of purchasing an upgraded version of Telerik we chose to seek out a more cost effective approach to reporting that allowed for deeper customization. First, we integrated with amCharts to offer more immersive and interactive graphing of data. This library does provide exporting features, but we needed a better solution for generating printable reports. We settled on jsreport and the associated dotnet NuGet package to solve this problem. This library allows us to generate completely custom, downloadable files of just about any type. In addition to the tabular data generated for these reports, images of the dynamically generated charts are included with these downloadable files.
More detailed reports were created to complement this improved reporting system. Facilities now have access to an end of record report that is automatically generated for them when a record is no longer available for their staff. Each new record is available to score for 3 months. At the conclusion of the scoring period, a report is generated for each facility that shows how well their staff did compared to all other facilities within the system. This report shows the facility’s overall average scores and percentile ranking as compared to all other facilities, and provides a breakdown of how each individual staff member performed on the record.
Pediatric Records
Sleep ISR has traditionally only offered adult specific sleep studies. One major feature change that has been frequently requested since Sleep ISR was first released is to offer pediatric specific sleep records. Sleep studies for children differ substantially from those for adults, and pediatric focused facilities did not have a convenient system for satisfying the inter-scorer reliability requirement for accreditation while focusing on studies pertinent to their practice. Sleep ISR implemented this feature, launching monthly pediatric specific records beginning on January 15, 2019.
One of the major challenges with this release was to accommodate an additional signal that is present within pediatric sleep studies. Known as End-Tidal CO2 (EtCO2), this signal is a representation of the amount of CO2 that is exhaled with each breath. We discovered that sleep studies can represent this in one of two ways. The first is to show the full wave form represented as mm Hg, and the second is to have two wave forms in which one shows the unitless shape of the waveform and the other shows the peak value for each exhalation in mm Hg. Our challenge was two fold. First, we needed the ability to combine the unitless waveform and subsequent peak value signal into a single wave form that shows mm Hg at every step of the exhalation process. The second was to determine the peak value for each breath in this combined signal.
To create a new record within Sleep ISR, we are given an EDF file that represents the data from an actual sleep study. This EDF file contains thousands of time based data points for several signals that were recorded during the patient’s sleep study. Our system iterates over these signals and converts them in to bitmap images that are used within Sleep ISR. With the EtCO2 signal, my task was to create an algorithm that not only combines the waveform and value signals in to a single waveform, but also determines the peak values for a signal with a constantly changing frequency. The problem of combining the signals was quickly reduced to a unit conversion process. The shape waveform is unitless, but the bottom of each wave form can be assumed to be equivalent to 0 mm Hg, and the value for the peak of each waveform is represented in the value signal. Using these two data points for each breath, the value of each point within the waveform can be converted to mm Hg. As for discovering and labeling the peak values of the combined waveform, the algorithm I converged on labels the peak for each and every breath, and will label intermediate peaks if the exhalation exhibits an anomaly where the pressure dips drastically but does not return to 0 before peaking again.
As a web developer, it’s not often I get to work on traditional computer science problems!
Archived, Free, And Bonus Records
The updated system removed a free trial period that was being exploited by users in favor of a single free record that was given to each new account. In addition to this free record, Sleep ISR now allows the AASM to identify bonus records that may be given to every individual that logs in within a specific time period, and a new subscription tier was added that opens up the entire AASM sleep record archives for users to score and/or rescore. These archives were made available to allow technicians to reacquaint themselves with historical records that they may have had difficulty with when they first scored them. The archive feature also gives facility administrators a tool to further help their technicians improve their scoring abilities.
Sleep ISR traditionally allowed users to only have one record opened at a time. If they opened a record, they could not start scoring another record until the first record was complete. With the addition of so many more accessible records the system needed to allow individuals to have multiple records open simultaneously, and Sleep ISR was modified to allow this.
Conclusion
The Sleep ISR overhaul was an extremely challenging and rewarding project based in extensive modification of legacy code using older technologies. This project brought forth several remarkable improvements:
- A more user friendly and intuitive dashboard.
- A simplified registration and subscription process.
- A permissions based authorization model allowing a single account to serve as an administrator for multiple facilities.
- Better reporting on staff and facility scoring results.
- The addition of pediatric sleep records.
- An integrated notification system.
- The inclusion of a subscription tier that includes all of the AASM archived records.
- The ability for users to score and rescore free, bonus, and archived records.
With this project I realized that older technologies are still relevant. Legacy code can easily be updated to give web applications a modern look and feel without using modern frameworks. Sometimes the best approach is to stick with and learn the technology that was originally used for a project when upgrading that project.