Kunst NR, Wilson E, Glynn D, Alarid-Escudero F, Baio G, Brennan A, Fairley M, Goldhaber-Fiebert JD, Jackson C, Jalal H, Menzies N, Strong M, Thom H, Heath A, on behalf of the Collaborative Network for Value of Information (ConVOI). Computing the Expected Value of Sample Information Efficiently: Practical Guidance and Recommendations for Four Model-Based Methods. Value in Health, 2020;23(6):734-742.
Published in:
Abstract
Background
Value of information (VOI) analyses can help policy makers make informed decisions about whether to conduct and how to design future studies. Historically a computationally expensive method to compute the expected value of sample information (EVSI) restricted the use of VOI to simple decision models and study designs. Recently, 4 EVSI approximation methods have made such analyses more feasible and accessible. Members of the Collaborative Network for Value of Information (ConVOI) compared the inputs, the analyst’s expertise and skills, and the software required for the 4 recently developed EVSI approximation methods. Our report provides practical guidance and recommendations to help inform the choice between the 4 efficient EVSI estimation methods. More specifically, this report provides: (1) a step-by-step guide to the methods’ use, (2) the expertise and skills required to implement the methods, and (3) method recommendations based on the features of decision-analytic problems.
Methods
The reviewed methods included a regression-based (RB) method, an importance sampling (IS) method, a Gaussian approximation (GA) method and a moment matching (MM) method. These methods were selected because they place limited restrictions on the complexity of the underlying decision-analytic model or data collection exercise when calculating EVSI.
Results
The EVSI R package contains several graphical displays to present EVSI and related quantities. EVSI results can be loaded into the EVSI package, irrespective of the computation method used. These graphics can be displayed directly in R or explored using a dynamic graphical display launched from within R. Analysts unfamiliar with R can explore these graphics using an online interface at EVSI. The optimal sample size estimated using VOI methods can also be presented in form of a curve of optimal sample size (COSS). The COSS graphically displays optimal sample sizes for proposed study designs over a range of willingness-to-pay thresholds, including the impact of variation and uncertainty in VOI parameters on those optimal sample sizes.
Conclusions
VOI analysis has the potential to guide policy makers in the prioritization and design of future research studies, thereby improving decision making. Increasingly, HTA agencies are acknowledging the potential of VOI analyses and may start recommending EVSI to prioritize and design future research. In this study, members of ConVOI have provided practical guidance and recommendations to facilitate the implementation of 4 EVSI approximation methods. Our report outlines the inputs and software required to use each of these methods and also summarizes the recommended analysts’ skills and expertise needed to implement them based on the features of decision-analytic problems. The direction of future research in VOI is highlighted in Rothery et al, but future research should also focus on improving VOI implementation and its communication in clinical practice.