Linked Sensitivity Analysis, Calibration, and Uncertainty Analysis Using a System
Dynamics Model for Stroke Comparative Effectiveness Research
Yuan Tian, MSc; Kristen Hassmiller-Lich, PhD; Nathaniel D. Osgood, PhD; Kirsten Eom, MPH;David Matchar, MD
Introduction
Dynamic simulation models have been extensively used in informing medical decisions and health resource allocation. However, they are often complex and are involving a great degree of uncertainty in initial conditions, calibration, model input, and outputs. Securing a better understanding of the uncertainty is extremely valuable in achieving more effective calibration, inform future data collection and building confidence among stakeholders in decision making.
We demonstrate a systematic approach linking calibration, sensitivity and uncertainty analysis to inform better decision making under various type of uncertainty, especially for complex dynamic models.
We demonstrate a systematic approach linking calibration, sensitivity and uncertainty analysis to inform better decision making under various type of uncertainty, especially for complex dynamic models.
Analysis Overview
Our model starts with 60 uncertain parameters which need to be estimated. We applied a very rigorous analysis procedure to secure the impact of parameter uncertainty and calibration uncertainty on stroke outcomes. The motivation and results of the analysis are illustrated in Figure 2 and Figure 3.
Calibration
We employed the Powell hill-climbing optimization method to systematically and extensively linear search through the input space. We also applied Generalized likelihood uncertainty estimation (GLUE) (1) to explore the uncertainty in calibration, and we end up with 1,000 baselines. The descriptive results of these 1,000 baselines are presented in Table 1.
Reference:
1. Beven K, binley A. The Future of Distributed Models - Model Calibration and Uncertainty Prediction. Hydrological Processes. 1992;6(3):279-98.
Reference:
1. Beven K, binley A. The Future of Distributed Models - Model Calibration and Uncertainty Prediction. Hydrological Processes. 1992;6(3):279-98.
Table 1. Descriptive Results of 1,000 Calibration Baselines
Scatter Plot of QALYs of considered Stroke Interventions over 5 Years
Reference
Tian, Y., Lich, K. H., Osgood, N. D., Eom, K., & Matchar, D. B. (2016). Linked Sensitivity Analysis, Calibration, and Uncertainty Analysis Using a System Dynamics Model for Stroke Comparative Effectiveness Research. Medical Decision Making, 0272989X16643940.
Tian, Y., Lich, K. H., Osgood, N. D., Eom, K., & Matchar, D. B. (2016). Linked Sensitivity Analysis, Calibration, and Uncertainty Analysis Using a System Dynamics Model for Stroke Comparative Effectiveness Research. Medical Decision Making, 0272989X16643940.