Assessment of the methodological quality of meta-analyses on interventions to reduce juvenile reoffending
Main Article Content
Abstract
The methodological quality used in research impacts its results and may produce biases. Therefore, a systematic review of other systematic reviews and meta-analyses on the effectiveness of psychosocial intervention programs to reduce reoffending among young offenders was carried out, aiming to describe their characteristics and evaluate their methodological rigor. Among the 25 articles included, it was found that the majority (n=15) came from the United States, and most of the studies (n=20) performed meta-analytic calculations rather than narrative syntheses. Furthermore, the articles included showed an increasing production from the year 2000 onwards, with publications between 1985 and 2023. The methodological quality of most articles (n=18) was classified as critically low by the AMSTAR-2 tool. The characteristics of the included articles can be understood through historical data on the sociopolitical development of the juvenile justice system and the scientific development of the meta-analytic method itself. In turn, the low methodological quality observed points to a lack of transparency in the production of meta-analysis reports in the field of study focused on the effectiveness of psychosocial interventions. Thus, some possible ways to bridge the existing gaps are discussed.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Obra disponible bajo una Licencia Creative Commons Atribución-NoComercial-CompartirIgual 4.0 Internacional (https://creativecommons.org/licenses/by-nc-sa/4.0/deed.es).
References
Blackburn, S. (2019). Juvenile Justice in Pennsylvania: Mission‐Driven; Performance‐ Based; Outcome‐Focused. Juvenile and Family Court Journal, 70(1), 73–87. https://doi.org/10.1111/jfcj.12127 DOI: https://doi.org/10.1111/jfcj.12127
Cathey, D. (2019). Relatório Mundial 2020: Rights Trends in Estados Unidos. Human Rights Watch. https://www.hrw.org/pt/world-report/country-chapters/united-states
Cheng, S.-Y., Davis, M., Jonson-Reid, M., y Yaeger, L. (2021). Compared to What? A Meta-Analysis of Batterer Intervention Studies Using Nontreated Controls or Comparisons. Trauma, Violence & Abuse, 22(3), 496-511. https://doi.org/10.1177/1524838019865927 DOI: https://doi.org/10.1177/1524838019865927
Cullen, F. T. (2013). Rehabilitation: Beyond Nothing Works. Crime and Justice, 42(1), 299–376. https://doi.org/10.1086/670395 DOI: https://doi.org/10.1086/670395
Elliott, D. S., Buckley, P. R., Gottfredson, D. C., Hawkins, J. D., y Tolan, P. H. (2020). Evidence‐based juvenile justice programs and practices: A critical review. Criminology & Public Policy, 19(4). https://doi.org/10.1111/1745-9133.12520 DOI: https://doi.org/10.1111/1745-9133.12520
Glass, G. V. (2015). Meta-analysis at middle age: a personal history. Research Synthesis Methods, 6(3), 221–231. https://doi.org/10.1002/jrsm.1133 DOI: https://doi.org/10.1002/jrsm.1133
Goshe, S. (2019). How Contemporary Rehabilitation Fails Youth and Sabotages the American Juvenile Justice System: A Critique and Call for Change. Crit Crim, 27, 559–573. https://doi.org/10.1007/s10612-019-09473-5 DOI: https://doi.org/10.1007/s10612-019-09473-5
Moosavi, L. (2019). A Friendly Critique of 'Asian Criminology' and 'Southern Criminology'. The British Journal of Criminology, 59(2), 257–275. https://doi.org/10.1093/bjc/azy045 DOI: https://doi.org/10.1093/bjc/azy045
Lipsey, M. W. (2009). The Primary Factors that Characterize Effective Interventions with Juvenile Offenders: A Meta-Analytic Overview. Victims & Offenders, 4(2), 124–147. https://doi.org/10.1080/15564880802612573 DOI: https://doi.org/10.1080/15564880802612573
Lipsey, M. W. (2020). Revisited: Effective use of the large body of research on the effectiveness of programs for juvenile offenders and the failure of the model programs approach. Criminology & Public Policy, 19(4), 1329–1345. https://doi.org/10.1111/1745-9133.12530 DOI: https://doi.org/10.1111/1745-9133.12530
Lipsey, M. W., Howell, J. C., y Tidd, S. T. (2007). The standardized program evaluation protocol (SPEP): A practical approach to evaluating and improving juvenile justice programs in North Carolina. Final evaluation report. Nashville, TN: Vanderbilt University. Available from author, Retrieved from http://mark.lipsey@vanderbilt.edu
López-Nicolás, R., López-López, J.A., Rubio-Aparicio, M. et al (2022). A meta-review of transparency and reproducibility-related reporting practices in published meta-analyses on clinical psychological interventions (2000–2020). Behavior Research Method, 54, 334–349. https://doi.org/10.3758/s13428-021-01644-z DOI: https://doi.org/10.3758/s13428-021-01644-z
Mulvey, E. P., Schubert, C. A., Jones, B., y Hawes, S. (2020). A validation of SPEP™ in Pennsylvania. Harrisburg, PA: Pennsylvania Commission on Crime and Delinquency. Retrieved from https://www.pccd.pa.gov/Juvenile-Justice/Documents/A%20Validation%20of%20SPEP%20in%20PA%20Report.pdf
Pigott, T. D., y Polanin, J. R. (2020). Methodological Guidance Paper: High-Quality Meta-Analysis in a Systematic Review. Review of Educational Research, 90(1), 24-46. https://doi.org/10.3102/0034654319877153 DOI: https://doi.org/10.3102/0034654319877153
Polanin, J. R., Hennessy, E. A., y Tsuji, S. (2020). Transparency and Reproducibility of Meta-Analyses in Psychology: A Meta-Review. Perspectives on Psychological Science, 15(4), 1026-1041. https://doi.org/10.1177/1745691620906416 DOI: https://doi.org/10.1177/1745691620906416
Redpath, D. P., y Brandner, J. K. (2010). The Arizona standardized program evaluation protocol (SPEP) for assessing the effectiveness of programs for juvenile probationers: SPEP rating and relative recidivism reduction: An update to the January 2008 Retrieved from https://my.vanderbilt.edu/spep/files/2013/04/AZ-SPEP-Recidivism-Analysis_Update_-2010.pdf
Schwarzer, G., Chemaitelly, H., Abu‐Raddad, L. J., y Rücker, G. (2019). Seriously misleading results using inverse of freeman‐tukey double arcsine transformation in meta‐analysis of single proportions. Research Synthesis Methods, 10(3), 476-483. https://doi.org/10.1002/jrsm.1348 DOI: https://doi.org/10.1002/jrsm.1348
Sera, F., Armstrong, B., Blangiardo, M., y Gasparrini, A. (2019). An extended mixed-effects framework for meta-analysis. Statistics in Medicine, 38(29), 5429-5444. https://doi.org/10.1002/sim.8362 DOI: https://doi.org/10.1002/sim.8362
Shadish, W. R., y Lecy, J. D. (2015). The meta-analytic big bang. Research Synthesis Methods, 6(3), 246–264. https://doi.org/10.1002/jrsm.1132 DOI: https://doi.org/10.1002/jrsm.1132
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., et al. (2017). AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ [Internet]. https://doi.org/10.1136/bmj.j4008 DOI: https://doi.org/10.1136/bmj.j4008
Tanner-Smith, E.E., Lipsey, M.W. y Wilson, D.B. Juvenile drug court effects on recidivism and drug use: a systematic review and meta-analysis. J Exp Criminol, 12, 477–513 (2016). https://doi.org/10.1007/s11292-016-9274-y DOI: https://doi.org/10.1007/s11292-016-9274-y
Wong, J. S., y Bouchard, J. (2023). Do Meta-Analyses of Intervention/Prevention Programs in the Field of Criminology Meet the Tests of Transparency and Reproducibility? Trauma, Violence, & Abuse, 24(3), 1522-1542. https://doi.org/10.1177/15248380211073839 DOI: https://doi.org/10.1177/15248380211073839