
Impact of Correlation between Multiple Time Point Measurements on Pooled Effect Measures in Meta-analysis
Correspondence Address :
N Sreekumaran Nair,
Professor, Department of Biostatistics, Jawaharlal Institute of Postgraduate Medical Education and Research, Puducherry-605006, India.
E-mail: nsknairmanipal@gmail.com
Introduction: Effect estimates obtained from multiple time points based on the same set of subjects are observed to be correlated. There is a need to integrate these correlations in the derivation of pooled summary measures to improve the precision of estimates. The conventional meta-analysis does not consider this dependency into account.
Aim: To compare the results obtained from meta-analysis which incorporate various levels of correlation in repeated measures data to the traditional meta-analysis.
Materials and Methods: The present statistical analytical study was conducted at Jawaharlal Institute of Postgraduate Medical Education and Research, Puducherry, India, from January 2021 to February 2022 on data from a systematic review that studied the effect of analgesics in reducing orthodontic pain using Visual Analogue Scale (VAS) pain score measured at three time points were used for demonstration. This study attempted to illustrate two distinct approaches to deal with dependency between measurements obtained from different follow-ups by adopting constant and degenerating correlation structures.
Results: The pooled effect estimates and confidence intervals obtained from models which incorporated correlation were different from the results of traditional approach. Naproxen fared to be better when compared to other two treatments. Pooled effect estimates and confidence intervals from Model 2 and Model 3 hovered around the same values. Non significant difference was observed in the Akaike Information Criterion (AIC) values of Model 2 and Model 3 for all three treatments. The between study variance ranged from 0.07 to 1.46, 1.25 to 3.17 and 0.01 to 0.98 for Acetaminophen, Naproxen and Ibuprofen, respectively.
Conclusion: The models which took care of dependency had a better fit to the data over conventional meta-analysis.
Correlated effect estimates, General linear mixed model, Repeated measures, Visual analog scale score
In healthcare research, outcomes are measured at different follow-up times to monitor the health status of patients. It has been reported that the outcomes obtained in such a way as heart rate, VAS score and pulmonary function, Forced Expiratory Volume in the first second (FEV1) tend to be highly correlated across different time points (1),(2),(3). Ignoring this stochastic dependency during the analysis will overestimate p-values for within-subject or within-cluster comparison and underestimate in between-subject or between-cluster comparison (4). Diverse statistical techniques like paired t-test, Wilcoxon signed rank test, Mc-Nemar’s test repeated measures Analysis of Variance (ANOVA), linear mixed models, generalised estimating equations etc., are used to address this dependency (5),(6),(7). The effect estimates obtained from multiple time points in longitudinal studies may likewise be related since they are procured from the same set of study units. In the conventional meta-analytic approach, a separate meta-analysis is done for each time point without considering this correlation. Multivariate meta-analysis is an alternative which can model the dependent effect estimates (8). Likewise, a meta-analysis of multiple time points can also be carried out in the General Linear Mixed (GLM) model framework which can account for correlation between time points (9),(10),(11).
This study has examined, whether the summary results and conclusions from the meta-analytic models of a set of data measured at multiple time points that incorporates dependency differ from the same data meta-analysed by ignoring dependency.
This study was conducted at Jawaharlal Institute of Postgraduate Medical Education and Research, Puducherry, India, from January 2021 to February 2022.
Procedure
Data source: The data for the present study was taken from a published systematic review that studied the efficacy of analgesics in controlling orthodontic pain: a systematic review and meta-analysis (12). The study compared the efficacy of three different analgesics: ibuprofen, acetaminophen and naproxen on the relief of pain in orthodontic treatment compared to the placebo among patients with orthodontic pain. Change in VAS score measured at 2, 6 and 24 hours in different studies were chosen for the demonstration of meta-analysis of longitudinal studies. The effect size was the standardised difference in means.
Statistical Analysis
Meta-analytic framework for modeling covariance: The effect estimates from multiple time points were pooled through different models using the GLM model framework (11). A more flexible unified modeling framework proposed by Sera F et al., was performed for a set of effect sizes measured at ki times in study as given below (11):
Where, θ, g, gi, and hi are fixed and random effect coefficients for intercepts and slopes. Three different models were built by specifying combinations of three different variants of within study variance covariance structures like independent, heterogeneous compound symmetry and heteroscedastic autoregressive as given below:
Model 1: Independent random time effects model (Zero correlation): This model assumes the effect size to be independent and will yield similar results to those of traditional univariate meta-analysis carried out for each time point separately. An independent variance, covariance structure that assumes that zero correlations between the effect estimates obtained from any two time points was used.
Model 2: Correlated within study model with constant correlation: This model is an extension of Model 1, where dependency between effects estimates obtained from different time points are considered. However, this within study serial correlation between longitudinal effect sizes is assumed to be a constant. Compound symmetry variance covariance which assumes correlation to be the constant across any two time points was adopted. A moderate correlation (r=0.50) was assumed between any two time points in the current study.
Model 3: Correlated within study model with degenerating correlation: In this model, which is also again an extension of Model 1, dependency between effect estimates obtained at different time points are considered to be degenerating as duration increases. A heteroscedastic autoregressive variance covariance matrix in which the correlation exponentially decreases with an increase in the time lag between the time points was used. If the correlation between first and second time point was assumed to be ‘r’, then the correlation between first and third was assumed to be r|3-1|. In the current study, a correlation of 0.50 was assumed between the first time point and second time point whereas the correlation diminished to 0.25 for the first and third time point. Likewise, the correlation between second and third was 0.50.
The between study variance covariance matrix was assumed to be independent in models 1, 2 and 3. The results were reported in terms of pooled effect estimates along with their 95% Confidence Intervals (95% CI) for all three time points of interest. The results obtained from meta-analysis with the assumption of independence were compared to the models which incorporated the correlations. The change in the treatment effect estimates and their precisions were compared. The deviance measures like Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) were calculated and compared across models to identify the best fitted model. A model with the lowest AIC or BIC values were considered to be a good fit for the data. All the analyses were carried out using mixmeta package in R software version 4.1.1 (13).
Among the included studies, there were six, four and nine trials studying the effectiveness of acetaminophen, naproxen and Ibuprofen 9treatments, respectively. The sample size of the trials ranged from 15 to 38, 17 to 30 and 10 to 41 for acetaminophen, naproxen and ibuprofen treatment comparisons. The results obtained through GLM models which considered the correlation are given in (Table/Fig 1). The usage of independent covariance structure (Model 1) yielded the same results as those of the multiple univariate meta-analyses in case of all treatment comparisons. A higher reduction in the pain score was observed at six hours in naproxen and ibuprofen whereas the acetaminophen showed the highest improvement after 24 hours. The reduction in pain score was observed to be almost same after 24 hours for the first two treatments, Acetaminophen and Naproxen. The trajectory of pain reduction was same over the period of time for the treatments, naproxen and ibuprofen.
A similar pattern of trend in the effect estimates across different time points were also found among the models which incorporated dependency. All the results were found to be statistically significant. Naproxen fared to be better when compared to other two treatments. The pooled effect estimates and confidence intervals obtained from models which incorporated correlation were different from the results of traditional approach. Pooled effect estimates and confidence intervals from Model 2 and Model 3 hovered around the same values (Table/Fig 2).
The AIC values of models which took care of the dependency into account were lesser when compared to the conventional method. No significant difference was observed in the AIC values of Model 2 and Model 3 for all three treatments. The between study variance ranged from 0.07 to 1.46, 1.25 to 3.17 and 0.01 to 0.98 for acetaminophen, naproxen and ibuprofen, respectively.
The present study has examined the impact of correlation between multiple time point measurements on pooled effect measures in meta-analysis. The models (model 2 and 3) which incorporated the dependency between time points gave substantially lesser AIC values when compared to model which did not incorporate correlation (Model 1) indicating them to be better fit to the data. The pattern observed was same in case of all the treatment comparisons. This emphasises the need for considering the correlation structure existing between multiple time points.
Additionally, the current study proposed a constant correlation between the effect sizes across time points (Model 2) whereas the previous studies assumed a constant correlation between effect estimates across different studies (8),(9),(10),(11). Ishak KJ et al., developed and compared several methods to handle correlations of longitudinal effect estimates in a meta-analysis using data on deep brain stimulation among patients with Parkinson’s disease measured at various time points (9). The outcome investigated was the Unified Parkinson Disease Rating Scale (UPDRS) motor function score, a continuous outcome. Three different models namely, study specific random effects, general multivariate method which can incorporate correlated within study residuals, and correlated time-specific random effects were used.
The model that used the multivariate approach provided a better fit to the data than the other two models. The models that used inherent correlation provided more precise summary estimates. Methods that consider stochastic dependencies between effect estimates obtained from subsequent time points was developed by Trikalinos TA and Olkin I (8). The outcome was survival rate at different follow-ups and the effect estimate used was the odds ratio. Different methods by assuming varying variance covariance structures were compared with univariate meta-analysis. Minor changes were observed in the magnitude of the effect sizes and their corresponding standard errors obtained from univariate and multivariate analyses. The study concluded that data acquired from multiple time points are multivariate in nature and therefore must be analysed using multivariate techniques.
Musekiwa A et al., compared five different GLM models by assuming different within and between variance covariance structures (10). GLM models are more flexible and can easily take care of new combinations of covariance structures like heterogeneous compound symmetry, heteroscedastic autoregressive, independence etc. The models were demonstrated using the same survival data used for demonstration by Trikalinos TA and Olkin I (8). The model which used a heteroscedastic autoregressive structure performed better.
Joint analysis of correlated effect estimates using an autoregressive covariance structure provided more precise estimates when compared to the compound covariance structure. All five models that incorporated correlation fitted better to the data when compared to the traditional approach. Only Ishak KJ et al., described meta-analysis of longitudinal time points using continuous data whereas all the other studies used binary outcomes (9).
In this study, Model 3 which assumed heteroscedastic autoregressive was found to be better when compared to other models in previous studies [10,11]. Model 2 and model 3 which used heterogeneous compound symmetry and heteroscedastic autoregressive gave similar AIC values in the current study. This may be due to the more similarity in the correlation structure since there was fewer number of time points.
Under both meta-analysis with and without considering dependency, all three treatments were found to be better when compared to the placebo group at all the time points and the results were found to be statistically significant. However, there was a change in the magnitude of pooled effect estimate after incorporation of the correlation. Intervention studies may produce an upward trend in effect estimates that may finally stabilise over a period of time, or they may cause effect estimates to plummet and revert to their baseline value. As a result, the effect measures either obtained from the subsequent time points will be more correlated as compared to those from the baseline and last time points. The changes brought to those time points after incorporating those correlations in the variance covariance structure can be closely observed depending upon the clinical significance.
Limitation(s)
One limitation of the current study was that, it did not addressed the analysis of categorical outcome and adjusting for the effect of the covariate. Also, the number of time points were less in order to study the decay of correlation with time.
It was observed that the correlation between the repeated measurements has an influence in the pooled effect estimates at different time points. Hence, deriving pooled effect estimates by incorporating correlation during meta-analysis of repeated measures data would be a better choice. The more flexible general framework incorporating the correlation may result in more valid estimates.
DOI: 10.7860/JCDR/2023/59761.17994
Date of Submission: Aug 22, 2022
Date of Peer Review: Nov 26, 2022
Date of Acceptance: Dec 24, 2022
Date of Publishing: Jun 01, 2023
AUTHOR DECLARATION:
• Financial or Other Competing Interests: None
• Was Ethics Committee Approval obtained for this study? NA
• Was informed consent obtained from the subjects involved in the study? NA
• For any images presented appropriate consent has been obtained from the subjects. NA
PLAGIARISM CHECKING METHODS:
• Plagiarism X-checker: Aug 27, 2022
• Manual Googling: Dec 01, 2022
• iThenticate Software: Dec 14, 2023 (19%)
ETYMOLOGY: Author Origin
- Emerging Sources Citation Index (Web of Science, thomsonreuters)
- Index Copernicus ICV 2017: 134.54
- Academic Search Complete Database
- Directory of Open Access Journals (DOAJ)
- Embase
- EBSCOhost
- Google Scholar
- HINARI Access to Research in Health Programme
- Indian Science Abstracts (ISA)
- Journal seek Database
- Popline (reproductive health literature)
- www.omnimedicalsearch.com