Open Journal of
Clinical and Medical Images
Research Article - Open Access, Volume 3
Interobserver variability in assessing treatment response of hepatocellular carcinoma to transarterial chemoembolization: A comparative study of LIRADS TR and mRECIST criteria in South America
Luis Fernando Bustamante-Cristancho, MD, MS1; Enrique Carlos García-Pretelt, MD2*; Martín Eduardo Renjifo, MD3; Diana Carolina Rodriguez-Gonzalez, MD4; Alfonso Holguin-Holguin, MD5
1Radiologist, Emergency Physician, Intensivist, Department of Critical Care Medicine, Valle del Lili foundation, Cali, Colombia.
2Radiology Resident, Icesi University - Valle del Lili foundation, Cali, Colombia.
3Radiologist, Body Imaging, Department of Radiology, Valle del Lili foundation, Cali, Colombia.
4Radiologist, Oncological Imaging, Department of Radiology, Valle del Lili foundation, Cali, Colombia.
5Interventional Radiologist, Head of the Department of Radiology, Valle del Lili foundation, Cali, Colombia.
*Corresponding Author: Enrique Carlos García-Pretelt
Radiology Resident, Icesi University - Valle del Lili foundation, Cali, Colombia.
Tel: 57-315-3904772;
Email: enritrelt@gmail.com
Received : Nov 16, 2023
Accepted : Dec 15, 2023
Published : Dec 22, 2023
Archived : www.jclinmedimages.org
Copyright : © García-Pretelt EC (2023).
Abstract
Introduction: Hepatocarcinoma (HCC) is a major cause of worldwide morbidity and mortality. Transarterial chemoebolization (TACE) is a frequent management strategy in cases of unresectability. The imaging response assessment to treatment is complex and there are different strategies that are not universal and whose performance has not been compared. In Latin America there are no studies regarding the interobserver agreement, and in the world it is an issue that has not yet been clarified.
Objective: To determine the interobserver agreement in the assessment of LIRADS and mRECIST after chemoembolization in patients with hepatocarcinoma.
Methodology: A random retrospective sampling of patients with cirrhosis and HCC who underwent chemoembolization between 2012 and 2022 was carried out. The selected sample (20/358) was clinically characterized. Two masked radiologists with experience in oncology analyzed pre- and post-treatment images and determined the number of lesions, index lesion size, pretreatment LIRADS, post-treatment LIRADS, and mRECIST. Statistical analysis was performed to compare results.
Results: The main etiology of cirrhosis was cryptogenic (40%), followed by non-alcoholic (35%). The number of lesions, index lesion size, and pretreatment LIRADS were similar between the 2 radiologists (p-value 0.91, 0.067, and 1, respectively). Interobserver agreement for pretreatment LIRADS was 94% (Kappa value 0.80), posttreatment LIRADS was 75% (Kappa value 0.33), and mRECIST was 60% (Kappa value 0.47).
Conclusions: Interobserver agreement in pre-treatment LIRADS is GOOD. In LIRADS post treatment it is WEAK. In mRECIST it is MODERATE. Strategies are required to improve the interobserver agreement of the post-treatment LIRADS.
Keywords: Hepatocarcinoma, Trans arterial chemoembolization, LIRADS, mRECIST.
Abbreviations: HCC: Hepatocelular carcinoma; TACE: Transarterial chemoebolization; LIRADS: Liver imaging reporting and data system; mRECIST: Modified response evaluation criteria in solid tumors.
Citation: Bustamante-Cristancho LF, García-Pretelt EC, Renjifo ME, Rodriguez-Gonzalez DC, Holguin-Holguin A. Interobserver variability in assessing treatment response of hepatocellular carcinoma to transarterial chemoembolization: A comparative study of LIRADS TR and mRECIST criteria in South America. Open J Clin Med Images. 2023; 3(2): 1157.
Introduction
Hepatocellular carcinoma (HCC) represents the fifth most common neoplasm in the world and is the third leading cause of cancer-related death after lung and gastric carcinomas [1]. As a result, various therapeutic approaches have been developed that vary according to clinical staging and imaging. Transarterial chemoembolization (TACE), using Lipiodol, involves the direct administration of chemotherapeutic agents into the tumor through arterial infusion, guided by digital substraction angiography (DSA). It has been established as a minimally invasive option for the treatment of unresectable HCC, whether in a palliative manner or as a bridge pre-transplant therapy [2].
Different imaging-based criteria for assessing treatment response have been developed. They include mRECIST (modified Response Evaluation Criteria in Solid Tumors), EASL (European Association for the Study of the Liver), and LIRADS TR (Liver Imaging Reporting and Data System Treatment Response). Each one has its advantages and limitations, nevertheless, to date, there is no global consensus on which criteria offer the best performance in terms of histopathological correlation, prognosis, and other important variables such as interobserver agreement [3]. This is particularly relevant considering the recent introduction in 2017 of the LIRADS TR criteria, which from our experience, are not yet widely applied in our setting.
Unlike previous criteria, LIRADS TR includes an equivocal category and additional findings beyond arterial phase enhancement, such as washout and enhancement similar to pre-treatment [4]. Previous studies assessing the performance of LIRADS TR after TACE have yielded variable results [5-7]. However, to our knowledge, they have not been extensively compared with other strategies, and such studies are lacking in Latin America. In this regard, the purpose of this study is to conduct a pilot test to assess interobserver variability in the evaluation of treatment response with TACE in patients with hepatic cirrhosis and HCC, using the mRECIST and LIRADS TR criteria in a high-complexity reference center. This will enable the future development of projects aimed at enhancing the interpretation performance of these images. Although mRECIST is intended to evaluate the overall tumor burden at a per-patient level, it was used as a perlesion criterion in our study [8].
Materials and methods
A database was created for all adult patients who underwent TACE by searching for the procedure code in the electronic medical record system of a level four hospital in Cali, Colombia, in the period between January 2012 and December 2022, resulting in a total of 358 patients. A random sampling was conducted for this pilot study, selecting 20 patients with a diagnosis, either pathological or through imaging, of cirrhosis and HCC who had undergone invasive therapy with TACE. The patients were required to have abdominal magnetic resonance imaging (MRI) with dynamic post-contrast acquisitions adhering to the technical standards of the LIRADS 2018 group, performed before and after treatment. The follow-up images were taken between 2 and 9 months after the treatment.
Demographic and hepatopathy characterization was performed, including etiology, staging, use of sorafenib, and blood chemistry. This data was obtained retrospectively from medical records. The post-treatment images were analyzed retrospectively in the institutional Picture Archiving and Communication System (AGFA_PACS) by two radiologists with expertise in oncology (a general radiologist with 15 years of experience and a radiologist with a fellowship in oncology imaging). These radiologists, masked to the clinical characteristics, patient history, and previous image interpretations, determined various parameters for each case including the number of lesions, maximum size of the treated lesion, pre-treatment LIRADS category, posttreatment LIRADS TR category, and post-treatment mRECIST category. The study was approved by the institutional ethics committee, in accordance with national and international regulations.
Note: The mRECIST strategy is typically used to objectify treatment response globally and includes quantifying the total tumor burden (all lesions compatible with hepatocellular carcinoma). However, in our study, it was applied only to the index lesion treated with TACE to facilitate comparison with LIRADS TR, which assesses lesions independently.
Statistical analysis: A descriptive statistical analysis was conducted. Continuous variables were expressed as either mean and standard deviation or median and interquartile range. The comparison of medians was performed using the Wilcoxon signed-rank test. Categorical variables were presented as proportions, and the comparison between them was carried out using either the Chi-square test or Fisher’s exact test, depending on the case.
To assess the agreement between raters, the kappa coefficient was calculated and interpreted as follows: poor (<0.20), weak (0.21-0.40), moderate (0.41-0.60), good (0.61-0.80), and very good (0.81-1). Statistical significance was defined as a pvalue less than 0.05. For the data analysis, STATA 14.0 software was used.
Results
A total of 358 patients underwent TACE between January 2012 and December 2021. A random sample of 20 patients meeting inclusion criteria was selected, their clinical and demographic characteristics are detailed in Table 1. Among them, 17 (85%) were male and 3 (15%) were female, with a mean age of 68.1 years. The most common etiology of liver disease was cryptogenic (40%), followed by non-alcoholic steatohepatitis (NASH, 35%). Notably, viral etiology was observed in only 1 patient (5%). Most patients were categorized as Child-Pugh class A (85%) and Barcelona Clinic Liver Cancer stage B (85%).
The evaluation of pre-treatment images is detailed in Table 2. It did not show significant differences between the two radiologists (p>0.05). The interobserver agreement for pre-treatment LIRADS assessment was 94.7% with a kappa coefficient of 0.8081, categorized as good.
In the assessment of post-treatment images, the discrepancy increased as described in Table 3. The interobserver agreement for treatment response assessment using LIRADS TR was 75%, with a kappa coefficient of 0.3333, categorized as weak. The interobserver agreement using mRECIST was 60%, with a kappa coefficient of 0.477, categorized as moderate.
However, both radiologists concurred in detecting a significant decrease in the maximum diameter of the treated lesion: radiologist 1 quantified an average decrease of 31.7% (p 0.0247), and radiologist 2 measured a decrease of 40.6% (p 0.0093) (Figure 1).
Table 1: Clinical and demographic baseline.
| n (20) | % | |
|---|---|---|
| Identification data | ||
| Age* | 68.1 ± 7.4 | |
| Gender | ||
| Female | 3 | 15 |
| Male | 17 | 85 |
| Underlying disease data | ||
| Etiology of liver disease | ||
| Alcohol | 3 | 15 |
| NASH | 7 | 35 |
| Cryptogenic | 8 | 40 |
| Viral | 1 | 5 |
| Other | 1 | 5 |
| Child-Pugh | ||
| A | 17 | 85 |
| B | 3 | 15 |
| BCLC | ||
| A | 3 | 15 |
| B | 17 | 85 |
| Use of sorafenib | ||
| No | 16 | 80 |
| Yes | 4 | 20 |
| Laboratory tests prior to the First TACE | ||
| AFP (ng/ml) | 15.72 (4.895 - 97.5) | |
| Albumina (gr/dl) | 3.64 (3.34 - 4.04) | |
| Bilirrubina (mg/dl) | 1.18 (0.51 - 1.57) | |
| Creatinina (mg/dl) | 0.89 (0.74 - 1.065) | |
| PT | 14.2 (13.25 - 15.1) | |
| PTT | 33.1 (28.9 - 34.6) | |
| INR | 1.16 (1.02 - 1.32) | |
| AST (IU/L) | 52.1 (33.1 - 69.6) | |
| ALT (IU/L) | 44.95 (28.9 - 57.35) | |
Average ± standard deviation. **Median (IQR). AFP: Alpha-feto- protein. INR: International normalised ratio. NASH: Non-Alcoholic Ste- atoHepatitis. PT: Prothrombin time. PTT: Partial thromboplastin time. TACE: Transarterial Chemoembolization.
Table 2: Pre-TACE imaging features.
| Radiologist 1 | Radiologist 2 | P value | |
|---|---|---|---|
| Number of lesions | 1 (1 - 1) | 1.5 (1 - 3) | 0,067 |
| Maximum diameter (mm) | 42.5 (33 - 55) | 43 (31 - 60) | 0,9129 |
| LIRADS | |||
| LR-4 | 2 | 3 | 1 |
| LR-5 | 17 | 16 | |
| LR-M | 1 | 0 | |
| No data | 0 | 1 | |
TACE: Transarterial chemoebolization; LIRADS: Liver Imaging Reporting and Data System
Table 3: Pos del TACE imaging features.
| Radiologist 1 | Radiologist 2 | P value | |
|---|---|---|---|
| Number of lesions | 1 (1 - 1) | 2 (1 - 2) | 0,0271 |
| Maximum diameter (mm)** | 29 (25 - 38) | 25.5 (19 - 31) | 0,1562 |
| LIRADS | |||
| LR-TR Nonviable | 6 | 2 | 0,235 |
| LR-TR viable | 14 | 17 | |
| LR-TR equivocal | 0 | 1 | |
| mRECIST | |||
| SD | 3 | 9 | 0,246 |
| PD | 5 | 3 | |
| CR | 4 | 3 | |
| PR | 8 | 5 | |
TACE: Transarterial chemoebolization; LIRADS: Liver Imaging Reporting and Data System; mRECIST: modified Response Evaluation Cri-mRECIST: modified Response Evaluation Criteria in Solid Tumors; CR: Complete response; PR: Partial response; SD: Stable disease; PD: progressive disease.
Discussion
In this study, the LIRADS TR and mRECIST criteria were compared. The mRECIST defines viable tumor based solely on late arterial enhancement [9], while LIRADS additionally includes assessment of washout and enhancement similar to the pretreatment image [4] (Figure 2). According to our study, the assessment of post-TACE treatment response in HCC is complex regardless of the criteria used. We demonstrated that the interobserver agreement between two radiologists to establish tumor viability in magnetic resonance imaging is weak when using LIRADS TR and moderate when using mRECIST.
The interobserver agreement for LIRADS TR has been studied independently (not compared to mRECIST). Bartnik et al., found moderate agreement when using computed tomography (Kappa 0.70), especially in the nonviable tumor category (kappa 0.80) [10]. Abdelrahman et al., reported almost perfect interob- interobserver agreement for arterial phase hyperenhancement, washout, enhancement similar to pretreatment and DWI findings in all treated HCCs, when using MRI (kappa 0.815, 0.837, 0.826 and 0.81 respectively) [11].
Only two studies have compared the performance of LIRADS TR and mRECIST at the same time, however, they included patients with different kinds of loco-regional treatmen. Seo et al., noted better performance when using mRECIST (good) vs LIRADS TR (moderate) (kappa coefficient 0.713 vs 0.560, respectively) [8], highlighting better performance of both criteria in computed tomography (k 0.800 vs 0.693, respectively). In the article by Bae et al., LIRADS TR showed substantial agreement for both CT (kappa: 0.69) and HBA-enhanced MRI (kappa 0.69); mRECIST also demonstrated substantial interobserver agreement for viable tumors with both CT (Kappa 0.74) and HBAenhanced MRI (kappa, 0.64) [12].
Nevertheless, LIRADS TR has achieved high specificity and moderate sensitivity in the detection of viable tumor post-TACE with histopathological correlation, being non inferior when compared to mRECIST [13]. This finding confirms its robustness as an algorithm. Therefore, we consider that likely the obtained interobserver variability results are due to its relatively recent introduction to the radiological lexicon, and that future prospective studies with a larger number of patients may demonstrate better performance. On the other hand, as secondary outcomes, we were able to demonstrate good interobserver agreement in the assessment of pre-treatment LIRADS and adequate correlation in quantifying the reduction in tumor size post-TACE, results that have already been widely reported in other studies [14-16].
Finally, the clinical characterization in this study showed that the primary etiology of hepatic cirrhosis in our patients is cryptogenic (40%), different from what is described in other countries in the region where viral cirrhosis (20-37%), alcohol-related cirrhosis (21.5-70%), or NASH (5-45%) prevail [17-19]. There is even national variability; Lara et al., demonstrated that the main etiology is alcoholic (45%) followed by Hepatitis C (15.7%) [20]. These findings could be due to the fact that our institution is a local reference hospital where complex pathologies prevail.
Conclusion
In conclusion, the interpretation of pre-treatment HCC images showed consistency among radiologists. Post-treatment assessment, on the other hand, requires strategies to improve interobserver agreement. The development of a database allowing for prospective studies with a larger number of patients could be a first step on this path.
Acknowledgements
Funding: There was no private or public funding for the study.
Conflicts of interest: The authors declare that there were no conflicts of interest.
References
- Chidambaranathan-Reghupaty S, Fisher PB, Sarkar D. Hepatocellular carcinoma (HCC): Epidemiology, etiology and molecular classification. Adv Cancer Res. 2021; 149: 1-61. doi: 10.1016/bs.acr.2020.10.001. Epub 2020 Nov 28. PMID: 33579421; PMCID: PMC8796122.
- Han K, Kim JH. Transarterial chemoembolization in hepatocellular carcinoma treatment: Barcelona clinic liver cancer staging system. World J Gastroenterol. 2015; 21(36): 10327-35. doi: 10.3748/wjg.v21.i36.10327. PMID: 26420959; PMCID: PMC4579879.
- Gregory J, Dioguardi Burgio M, Corrias G, Vilgrain V, Ronot M. Evaluation of liver tumour response by imaging. JHEP Rep. 2020; 2(3): 100100. doi: 10.1016/j.jhepr.2020.100100. PMID: 32514496; PMCID: PMC7267412.
- American College of Radiology. Liver imaging reporting and data system (LI-RADS). American College of Radiology Web site. https://www.acr.org/Clinical-Resources/Reporting-and-DataSystems/LI-RADS/CT-MRI-LI-RADS-v. 2017.
- Kim TH, Woo S, Joo I, Bashir MR, Park MS, Burke LMB, Mendiratta-Lala M, Do RKG. LI-RADS treatment response algorithm for detecting incomplete necrosis in hepatocellular carcinoma after locoregional treatment: a systematic review and meta-analysis using individual patient data. Abdom Radiol (NY). 2021; 46(8): 3717-3728. doi: 10.1007/s00261-021-03122-8. Epub 2021 May 23. PMID: 34027566; PMCID: PMC9358967.
- Kierans AS, Najjar M, Dutruel SP, Gavlin A, Chen C, Lee MJ, Askin G, Halazun KJ. Evaluation of the LI-RADS treatment response algorithm in hepatocellular carcinoma after trans-arterial chemoembolization. Clin Imaging. 2021; 80: 117-122. doi: 10.1016/j.clinimag.2021.06.009. Epub 2021 Jun 24. PMID: 34303189.
- Youn SY, Kim DH, Choi SH, Kim B, Choi JI, Shin YR, Oh SN, Rha SE. Diagnostic performance of Liver Imaging Reporting and Data System treatment response algorithm: a systematic review and meta-analysis. Eur Radiol. 2021; 31(7): 4785-4793. doi: 10.1007/s00330-020-07464-7. Epub 2021 Jan 6. PMID: 33409795.
- Seo N, Kim MS, Park MS, Choi JY, Do RKG, Han K, Kim MJ. Evaluation of treatment response in hepatocellular carcinoma in the explanted liver with Liver Imaging Reporting and Data System version 2017. Eur Radiol. 2020; 30(1): 261-271. doi: 10.1007/s00330-019-06376-5. Epub 2019 Aug 15. Erratum in: Eur Radiol. 2021 Jun;31(6):4400-4401. PMID: 31418085; PMCID: PMC7485122.
- Lencioni R, Llovet JM. Modified RECIST (mRECIST) assessment for hepatocellular carcinoma. Semin Liver Dis. 2010; 30(1): 52-60. doi: 10.1055/s-0030-1247132. Epub 2010 Feb 19. PMID: 20175033.
- Bartnik K, Podgórska J, Rosiak G, Korzeniowski K, Rowiński O. Inter-observer agreement using the LI-RADS version 2018 CT treatment response algorithm in patients with hepatocellular carcinoma treated with conventional transarterial chemoembolization. Abdom Radiol (NY). 2022; 47(1): 115-122. doi: 10.1007/s00261-021-03272-9. Epub 2021 Sep 28. PMID: 34581927; PM-CID: PMC8776670.
- Abdelrahman AS, Ekladious MEY, Badran EM, Madkour SS. Liver imaging reporting and data system (LI-RADS) v2018: Reliability and agreement for assessing hepatocellular carcinoma locoregional treatment response. Diagn Interv Imaging. 2022; 103(11): 524-534. doi: 10.1016/j.diii.2022.06.007. Epub 2022 Jul 2. PMID: 35787988.
- Bae JS, Lee JM, Yoon JH, Kang HJ, Jeon SK, Joo I, Lee KB, Kim H. Evaluation of LI-RADS Version 2018 Treatment Response Algorithm for Hepatocellular Carcinoma in Liver Transplant Candidates: Intraindividual Comparison between CT and Hepatobiliary Agent-enhanced MRI. Radiology. 2021; 299(2): 336-345. doi: 10.1148/radiol.2021203537. Epub 2021 Mar 2. PMID: 33650901.
- Kierans AS, Najjar M, Dutruel SP, Gavlin A, Chen C, Lee MJ, Askin G, Halazun KJ. Evaluation of the LI-RADS treatment response algorithm in hepatocellular carcinoma after trans-arterial chemoembolization. Clin Imaging. 2021; 80: 117-122. doi: 10.1016/j.clinimag.2021.06.009. Epub 2021 Jun 24. PMID: 34303189.
- Abdel Razek AAK, El-Serougy LG, Saleh GA, Abd El-Wahab R, Shabana W. Interobserver Agreement of Magnetic Resonance Imaging of Liver Imaging Reporting and Data System Version 2018. J Comput Assist Tomogr. 2020; 44(1): 118-123. doi: 10.1097/RCT.0000000000000945. PMID: 31939892.
- Ren AH, Zhao PF, Yang DW, Du JB, Wang ZC, Yang ZH. Diagnostic performance of MR for hepatocellular carcinoma based on LI-RADS v2018, compared with v2017. J Magn Reson Imaging. 2019; 50(3): 746-755. doi: 10.1002/jmri.26640. Epub 2019 Jan 15. PMID: 30648327.
- Abdelrahman AS, Madkour SS, Ekladious MEY. Interrater reliability and agreement of the liver imaging reporting and data system (LI-RADS) v2018 for the evaluation of hepatic lesions. Pol J Radiol. 2022; 87: 316-e324. doi: 10.5114/pjr.2022.117590. PMID: 35892071; PMCID: PMC9288199.
- Valentina Berrutti,etal. Hepatocarcinoma en el Hospital Pasteur, 2007-2016. An Facultad Med (Univ Repúb Urug). 2018; 5(1): 97-107.
- Torres F, et al. Tl 1 – Hepatocarcinoma: etiología prevalente en dos hospitales de santiago de chile. Gastroenterol. Latinoam. 2017; 28: 2.
- Piñero F, et al. Intermediate-advanced hepatocellular carcinoma in Argentina: Treatment and survival analysis. World J Gastroenterol. 2019; 25(27): 3607-3618.
- Lara Cárdenas JP, et al. Sobrevida de los pacientes con carcinoma hepatocelular tratados con quimioembolización con microesferas cargadas - experiencia en un hospital de alta complejidad. Tesis de grado