Quantifying the repeatability and reproducibility of Dr.Noon CVD, an artificial intelligence software as medical device for cardiovascular risk assessment via retinal imaging
Abstract
Purpose
To evaluate the repeatability and reproducibility of the Dr.Noon CVD, an artificial intelligence software as a medical device that assesses cardiovascular risk from retinal photographs by providing risk scores and classifying patients into three categories: category 0 (low risk), category 1 (intermediate risk), and category 2 (high risk).
Methods
In this prospective, single-center study, participants underwent nonmydriatic fundus photography. For repeatability assessment, one examiner captured 3 consecutive images per eye. For reproducibility, a second examiner independently acquired 1 image per eye. Intraclass correlation coefficients (ICCs), within-subject standard deviations, and coefficients of variation were calculated. Differences by risk category and lens status were assessed using ANOVA and independent t tests.
Results
Overall, Dr.Noon CVD demonstrated excellent reliability, with a repeatability ICC of 0.997 (95% confidence interval [CI]: 0.996–0.998) and a reproducibility ICC of 0.999 (95% CI: 0.998–0.999). When analyzed by risk categories, repeatability, and reproducibility ICCs were 0.985 (95% CI: 0.974–0.997) and 0.995 (95% CI: 0.990–0.997) for category 0, 0.960 (95% CI: 0.918–0.983) and 0.969 (95% CI: 0.921–0.988) for category 1, and 0.965 (95% CI: 0.943–0.980) and 0.984 (95% CI: 0.971–0.992) for category 2. In terms of lens status, phakic eyes showed repeatability and reproducibility ICCs of 0.998 (95% CI: 0.996–0.998) and 0.999 (95% CI: 0.998–0.999), respectively, while pseudophakic eyes showed slightly lower but still excellent values of 0.989 (95% CI: 0.980–0.995) and 0.994 (95% CI: 0.988–0.997).
Conclusions
Dr.Noon CVD demonstrated high precision with excellent repeatability and reproducibility across all risk levels and lens statuses, supporting its reliability for cardiovascular risk screening using retinal images.