Keeping Pace with Wearables: A Living Umbrella Review of Systematic Reviews Evaluating the Accuracy of Consumer Wearable Technologies in Health Measurement
Level 1 - systematic review of randomized trials
Systematic umbrella review of systematic reviews and validation studies
OpenAlex W4401112964 · doi:10.1007/s40279-024-02077-2
What was done
This living umbrella review searched MEDLINE, Embase, CINAHL, and SPORTDiscus for systematic reviews or meta-analyses that evaluated the validation of consumer wearable devices against reference standards. Outcomes extracted included mean absolute percentage error (MAPE), pooled absolute bias, intraclass correlation coefficients (ICCs), and mean absolute differences across multiple physiological metrics.
What was found
The search identified 24 systematic reviews, encompassing 249 non-duplicate validation studies and 430,465 participants (43% female). Only ~11% of available consumer wearables have been validated for at least one outcome (3.5% of total required evaluations). Reported performance metrics were: - Heart rate: mean bias of ± 3%. - Arrhythmia detection: pooled sensitivity 100%, specificity 95%. - VO2max: overestimated by ± 15.24% in resting tests and ± 9.83% in exercise tests. - Physical activity intensity: mean absolute error of 29% to 80%. - Step counts: mostly underestimated (MAPE -9% to 12%). - Energy expenditure: mean bias of -3 kcal/min (-3%), error ranging from -21.27% to 14.76%. - SpO2: mean absolute difference up to 2.0%. - Sleep: overestimated total sleep time (MAPE typically > 10%).
Why it matters
While consumer wearables offer strong accuracy for basic heart rate and arrhythmia detection, they have large measurement errors for complex physiological calculations like energy expenditure, VO2max, activity intensity, and sleep duration.
Limits
The findings are constrained by pervasive heterogeneity in primary study protocols and outcome reporting across reviews. Furthermore, roughly 89% of commercially released wearable models have never been validated against a reference standard in published literature.
Cited by
- contradicts Smartwatches that rely solely on photoplethysmography (PPG) are accurate for measuring heart rate at rest but are inaccurate during exercise.