Call to action: an open-source pipeline for standardized performance evaluation of sleep-tracking technology  

Extract

Over the last decade, the field of sleep research and clinical sleep medicine has dramatically expanded in its capability for measuring sleep naturalistically, longitudinally, and on a large scale. At the forefront of this sleep revolution is the exponential growth of miniaturized sleep-tracking technology like wearables and nearables (e.g. multisensor wristbands, noncontact radar/sonar bedside devices, dry EEG-based sleep headbands), mainly within the consumer space. For example, in 2021, Statista reported 111 million users of Fitbit devices [1], which is still only the tip of the iceberg when considering the widespread use of sleep-tracking technology. These devices offer the enticing opportunity to monitor sleep (e.g. sleep/wake patterns, sleep timings, sleep stages, naps) and related events (e.g. nighttime heart rate and its variability, atrial fibrillation, oxygen desaturation, breathing patterns) through analytical approaches (e.g. machine-learning algorithms) that rely on sensors to collect physiological, environmental, and behavioral data.

However, the rapid advancement of sleep technology in a largely unregulated space has outpaced the independent evaluation of its performance, leading to a lack of confidence in its adoption in scientific research and confusion about which devices, models, and approaches are reliable. The algorithms applied in commercial devices are proprietary (“black-box”), meaning that they cannot be uncoupled from the device and independently verified. There is an urgent need for standardized, rigorous, and open-source tools that the community can access in a timely manner to systematically evaluate the performance of new technology for measuring sleep [2, 3]. The sleep field acknowledges the need for such rigorous evaluation of how sleep is measured, and several independent and joint initiatives pushed this concept forward from different perspectives, including authoritative positions by the Sleep Research Society [4], the National Sleep Foundation, and the American Academy of Sleep Medicine [5, 6].