Software Program Can Help Reduce Errors, Cut Time in Evaluating mRCC Response to Therapy

Software Program Can Help Reduce Errors, Cut Time in Evaluating mRCC Response to Therapy

Dr. Brian C. Allen
A computer-assisted response evaluation (CARE) system reduced errors in assessing tumor response to therapy in a retrospective analysis of patients with metastatic renal cell carcinoma (RCC). The system also reduced the amount of time needed to assess responses (Abstract 432).

RCC metastases are vascular tumors, and antiangiogenic agents, such as sunitinib, are considered first-line therapy. RECIST 1.1 is used to evaluate response to those therapies, but its utility is limited by its reliance on unidimensional changes to tumor length. Other response criteria have been developed, including the Choi criteria and the Morphology, Attenuation, Size, and Structure (MASS) criteria.

The standard-of-care method for evaluating response in all of those cases involves a series of steps, all but one of which are done manually. “Someone has to identify target lesions, make measurements, and those measurements have to be transferred into the electronic medical record,” said Brian C. Allen, MD, a radiologist at Duke University Medical Center, who presented the new study data. The measurements must then be evaluated for response, and changes to tumor burden must be calculated. With currently used systems, all of this is manual with the exception of archival of annotated images, Dr. Allen said.

The CARE system is a software platform that can assist or automate all parts of response assessments. “We know that manual methods are prone to errors,” Dr. Allen said. Among the common errors in tumor response evaluation are target lesion selection errors, measurement errors, data transfer and recording errors, a failure to identify marked decrease in attenuation, errors in mathematical calculations of size or attenuation, and errors in the actual classification of response.

The new study aimed to test whether the CARE system could reduce those errors, as well as the time involved to evaluate response. It involved 11 radiologist readers from 10 institutions. They reviewed CT images from 20 patients with metastatic RCC with clear cell histology who were included in a completed phase III trial that compared sunitinib with interferon; the images were from baseline and after one cycle of sunitinib.

CARE walks users through the assessment process,  identifies errors in real time, and allows the user to correct them. It automates calculations and data transfer processes, among other steps.

The 11 readers were divided into two pools. The first pool read images for patients 1 through 10 by using standard-of-care methods, and then used CARE for the other 10 patients. After a 2-week washout period, the readers then read each group with the opposite method. The second pool read patients 11 through 20 with standard of care and patients 1 through 10 with CARE, and then they switched after 2 weeks.

The study found that readers made several types of errors in target lesion selection, but CARE helped eliminate them. Errors in data transfer and calculation were more common; for example, an incorrect calculation in percentage change in attenuation occurred in 6.0% of patients, whereas CARE resulted in no errors. Errors were also made in response categorization across the three types of response criteria; CARE eliminated those errors as well.

Dr. Alessandro Volpe
In total, 30.5% of patients assessed had at least one error with standard of care, compared with zero errors with CARE (p < 0.001). Unsurprisingly, there were more total errors with the Choi and MASS criteria (24.5% and 23.0% of patients, respectively) than with RECIST 1.1 (11.0%), because those are more complicated sets of criteria. CARE also cut the evaluation time approximately in half, at a mean of 6.4 minutes compared with 13.1 minutes with standard of care (p < 0.001).

Dr. Allen noted that the results are limited by its retrospective design and that it is not clear exactly how clinically important the errors made really are. Alessandro Volpe, MD, of the University of Eastern Piedmont, Maggiore della Carità Hospital, in Italy, was the Discussant for the abstract, and he also noted that information about the cost of the CARE system is still lacking and that assessment of the learning curve associated with the system would be valuable as well.

Still, he noted that the advantage of a system like CARE may be even more significant than found in this study, because this study was limited to experienced radiologists at academic centers. The future of tumor assessment in metastatic RCC, according to Dr. Volpe, is likely to undergo a substantial change in the next decade with systems such as this one, “with potential impact on clinical decision-making, and, most importantly, on patient outcomes.”

– David Levitan