Project nGene.org® - Pixel-Level semi-automatization of Parmar's methodology

Pixel-Level semi-automatization of Parmar's methodology

Estimation of hazard ratios from Kaplan-Meier survival curves for a meta-analysis

System, method, and computer program for conducting pixel-level semi-automatization of Parmar's methodology (Frank, Patent Application 1020200088882 (2020.07.17) & PCT Patent Application).

1. Introduction

Cochrane Training [1] introduces a meta-analysis of time-to-event data and illustrates a methodology proposed by Parmar et al. (hereafter referred to as Parmar's methodology) [2] to estimate a hazard ratio from published Kaplan-Meier (KM) plots for subsequent meta-analyses [3, 4]. Concerning Parmar's methodology, Tierney et al. published a "HR calculated spreadsheet" (hereafter referred to as Tierney's spreadsheet) [5], which has facilitated the process of estimating a hazard ratio from the KM plot given survival probabilities extracted from survival curves. However, before applying either Parmar's methodology or Tierney's spreadsheet, the preparation of these survival probabilities from survival curves requires an understanding of either Parmar's methodology or Tierney's spreadsheet, even for a researcher who routinely employs the KM plot. This requirement may act as a barrier to a more active meta-analysis of time-to-event data.

The accuracy and precision of the survival probabilities collected from the KM plots affect the accuracy and precision of the calculated hazard ratios during the use of Parmar's methodology and, in turn, determine the statistical significance of meta-analyses on hazard ratios. Traditionally, when collecting the survival probabilities from the KM plot, a researcher might use a ruler to locate certain coordinates on a printed KM plot. Alternatively, Guyot et al. [6] recommended using a general-purpose plot digitizer software, DigitizeIt [7], which functions as a digital ruler, to better extract the x and y values of the coordinates from a KM plot image file using an input device such as a mouse. However, it is laboriously difficult for a user, whether using a ruler or generalized software, to manually locate the coordinates one by one. More importantly, improvements could be made on traditional methods with respect to, firstly, the precision of a coordinate pointed out by a user and, secondly, the accuracy between the coordinate pointed out by a user and the coordinate intended by the user.

In the present study, SC2HR.py (Survival Curve To Hazard Ratio) is introduced as an open-source, all-in-one software tool written in the Python programming language [8]. All improvements for the scope of current studies stem from the idea of approaching the KM plot and Parmar's methodology at the pixel level. In order not to request information that requires an understanding of Parmar's methodology or Tierney's spreadsheet, SC2HR.py takes the strategy of requesting subsets of information that can be provided with routine knowledge of the KM plot (Fig. 1A). Then, SC2HR.py corrects user inputs to better recognize the KM plots in terms of precision and accuracy at the pixel level (Fig. 1B), and then allows the software to reconstruct information, which researchers traditionally had to provide based on their knowledge of Parmar's methodology, by letting the software enumerate survival curves from the origin on the x-axis to the right, pixel by pixel (Fig. 1C).

Figure 1. Overview of the pixel-level semi-automation of Parmar's methodology by SC2HR.py

(A) Without needing an understanding of Parmar's methodology, a researcher with routine knowledge of the KM plot can manually input information using a mouse or keyboard. This includes specifying an origin on the x-axis and an origin on the y-axis, a tick on each axis (with the y-axis tick, which specifically corresponds to a 1.0 survival probability), the numerical value of the tick on the x-axis, and the initial numbers of participants in the research and control groups, as well as the survival curves of both groups. (It is noted that this process may be easier than it appears after viewing Supplementary File S1. The numerical indices, such as "1101", "1102", "3001", etc., are used as per its patent application to minimize misidentification.) (B) SC2HR.py corrects a coordinate selected by a user to the intended coordinate at the pixel level. This enhances the precision and accuracy of the information provided by the user. (C) SC2HR.py automatically generates, in place of the user, the information necessary to compute a hazard ratio according to Parmar's methodology. Specifically, the "time interval" in SC2HR.py is programmed to be one pixel of the image file. This obviates the need for the user to manually select time intervals and then indicate the corresponding points on the survival curves.

2. Methods and Materials

2.1. Pixel-level Semi-automation of Parmar's Methodology (Figure 1)

2.1.1. Circumventing the Requirement of Understanding Parmar's Methodology (Figure 1A)

In traditional methods, such as using a ruler or general-purpose plot digitizer software like DigitizeIt [7], an understanding of Parmar's methodology is required, especially when selecting coordinates on the KM plot whose y-values are later converted to survival probabilities. More specifically, the process involves: first, dividing the x-axis, which represents time, into non-overlapping intervals or "time intervals" (not necessarily of equal length [2]), as depicted by the distance between imaginary vertical dotted lines in Figure 1C. Second, for each time interval from the origin on the x-axis to the right, the coordinate information of survival curves for control and research groups, represented as green and blue crosses, must be identified and marked, using either a pencil on a printed KM plot or the DigitizeIt software [7] on a digital image file [6]. Third, these y-values of coordinates should be computationally converted into survival probabilities, for input into Tierney's spreadsheet [5] or for manual calculation according to Parmar's methodology [2].

However, SC2HR.py has been designed to simplify the preparation steps, requiring only routine knowledge of KM plots. Firstly, SC2HR.py prompts the user to indicate four specific points on the KM plot using an input device (e.g., a mouse): origins on the x- and y-axes, a tick on the x-axis, and a tick on the y-axis corresponding to a 1.0 survival probability (Note: the origin on the x-axis may not always coincide with the origin on the y-axis on the KM plot). Secondly, the user is asked to enter the numerical value of the x-axis tick and the starting numbers of participants in the research and control groups. Thirdly, the user must indicate the survival curves of both groups using a mouse, as shown in the demonstration video (Supplementary File S1). In this way, SC2HR.py eliminates the need for the user to (i) determine the "time interval," (ii) locate coordinates of the research and control groups at the time intervals from the origin on the x-axis, and (iii) convert the y-values of the coordinates into survival probabilities.

2.1.2. Improving Accuracy and Precision in the Context of Parmar's Methodology (Figure 1B)

Whenever a user selects a specific spot on the KM plot, SC2HR.py searches for the nearest spots on KM plots based on the current context of information being requested, such as the origin of the coordinate or the survival curve of the control group. First, concerning the origins or ticks of the KM plots, SC2HR.py investigates the local minima in the funneling landscape of RGB values of pixels to find the intersection of the two lines nearby, aiming to enhance the precision and accuracy of acquiring the four points marked by a user to the pixel level. Second, for survival curves of either the control or research group, SC2HR.py is designed to collect the Y-values of survival curves, which are later computed into survival probabilities. For programmers, it's noteworthy that SC2HR.py reacts to the mouse's curve movement direction, focusing more on the horizontal or vertical aspects of nearby survival curves to capture information based on the user's mouse movement context.

2.1.3. Eliminating Manual Identification of Coordinates on Survival Curves (Figure 1C)

Traditionally, researchers had to manually identify several coordinates on the survival curves of the KM plots at the imaginary dotted lines representing non-overlapping time intervals. This method of manually and sparsely collecting information has been significantly improved by SC2HR.py, which is programmed to enumerate every pixel on the x-axis from the origin to the right, collecting as many y-values of survival curves of both research and control groups as possible. The narrower the time interval, the more information is gathered for computing a hazard ratio, aligning with Parmar's recommendation to choose time intervals that keep the event rate within the interval relatively small [2].

2.2. Evaluation

2.2.1. Accuracy and Precision of Corrected Coordinates

The accuracy of the coordinates corrected by SC2HR.py will be visually illustrated using a pixel-level magnifying-glass feature. For any specific mouse location, a pixel table with RGB values will be visually displayed on the right side of the GUI application. This feature magnifies the location pointed by the user and the location corrected by SC2HR.py, enabling a clear comparison.

Regarding precision, a specific point on the survival curves, measured by both DigitizeIt (a general-purpose plot digitizer software) and SC2HR.py, will be repeatedly measured to investigate the variance of measurements by SC2HR.py compared to DigitizeIt. This comparison reflects the degree of precision. Notably, both DigitizeIt and SC2HR.py require the coordinates of an origin on the X-axis, an origin on the Y-axis, a tick on the X-axis, and a tick on the Y-axis, along with the numerical value for the tick on the X-axis to calculate the value of the particular point (or, the tick on the X-axis). These steps will be repeated ten times for an F-test to determine whether the variance of SC2HR.py is lower than that of DigitizeIt.

Additionally, upon completing the steps required by SC2HR.py, a file named SC2HR.xlsx will be available, showcasing reconstructed pixel-level KM curves as perceived by SC2HR.py. More specifically and technically, SC2HR.py is programmed to consider that at least three pixels in a row must indicate the same Y-axis value, within a margin of error of +/- one pixel, before taking the Y-value into account for computing the survival probability according to Parmar's methodology. An uppercase "C" or "R" represents the pixel location identified by SC2HR.py where the Y-axis value remains exactly the same for three consecutive pixels. In contrast, a lowercase "c" or "r" represents a location where the Y-axis value deviates by just one pixel from a group of nearby pixels, such that adjusting that single "c" or "r" by one pixel would align all three Y-values for at least three consecutive times.

2.2.3. Verifying the Correctness of HR Computation Results

Censoring should be briefly mentioned. Censoring reflects amounts that cannot be accurately evaluated due to a lack of knowledge about what happened thereafter for reasons other than the outcome of interest (e.g., illness or death from other causes, withdrawal of consent, etc.) [9]. The appendix section of Parmar's methodology [2] proposed how to accommodate censoring to better estimate hazard ratios based on the assumption of a constant rate of censoring and the employment of minimum and maximum follow-up information (if available). Thus, with respect to the numerical dataset of Parmar et al. (pages 2825 ~ 2827), survival probabilities acquired from the KM plots using a mouse device will be used to compute hazard ratios and checked against the published hazard ratios.

However, the constant rate-of-censoring assumption is not always the case, and the information of minimum and maximum follow-up in the trial tends to be unavailable. In fact, censoring is typically encountered and indirectly noticed when comparing the proportion of survival curves and the information in tables for the number of patients at risk at a given time (Supplemental Figure 2A), such that, to the best of our knowledge, the appendix section of Parmar's methodology cannot be applied for the purpose of testing the correctness of SC2HR.py against the published hazard ratios. In the meantime, Combes et al. published a KM survival curve [10] where the proportion of the survival curve matches the table of the number of patients at risk at a given time (Supplementary Figure 2B). Therefore, in this study, testing of SC2HR.py was performed for the dataset of Combes et al. [10], without applying the appendix part of Parmar's methodology to accommodate censoring [2].

Figure 2. Pixel-level correction

SC2HR.py enhances precision and accuracy by adjusting the coordinates selected by users. Given the challenges users face in pinpointing an exact location on an image with a mouse-like device to the precision of a single pixel, SC2HR.py automatically corrects the manually marked point (shown in red) to the intended point (shown in green).

3. Results

In terms of the accuracy of coordinates, SC2HR.py visually displays both the actual location marked by researchers and the corrected location intended by researchers at the pixel level. This feature allows for the visual verification of the pixel-level accuracy of the corrected locations, as illustrated in Figure 2. SC2HR.xlsx demonstrates the reconstructed pixel-level KM curves, showcasing how SC2HR.py interprets survival curves at the pixel level (Figure 3). The precision of SC2HR.py was compared against DigitizeIt by repeatedly measuring a specific point on a survival curve to observe the variability of each tool (Supplemental Data 3). Measurements taken with DigitizeIt software were as follows: (0.400, 98.0), (0.428, 98.1), (0.438, 97.7), (0.427, 98.2), (0.428, 98.1), (0.438, 98.0), (0.419, 98.2), (0.438, 97.9), (0.438, 97.9), (0.419, 98.0), and (0.438, 98.0). Measurements taken with SC2HR.py were: (0.433, 97.9), (0.433, 97.9), (0.433, 98.0), (0.426, 97.9), (0.419, 97.9), (0.433, 97.9), (0.426, 98.0), (0.426, 98.2), (0.426, 98.0), and (0.426, 97.9). The variances in X-values for DigitizeIt and SC2HR.py were 0.0001491 and 0.00002232, respectively, and the F-test results indicated that the variance of SC2HR.py was significantly lower than that of DigitizeIt (p = 0.00464). The variances in Y-values for DigitizeIt and SC2HR.py were 0.02178 and 0.00933, respectively; however, the F-test did not indicate a significant difference in variance between SC2HR.py and DigitizeIt for Y-values (p = 0.1114).

When testing SC2HR.py with the dataset from Parmar et al. [2] (Supplementary Data 3A), the computed log hazard ratio was XXXX, compared to -0.244 according to Parmar's computation [2] and -0.0235 in the original authors' report [2, 11], matching up to three decimal points. For the dataset from Combes et al. [10] (Supplementary Data 3B), SC2HR.py matched the reported hazard ratios up to the third decimal point, without applying the appendix part of Parmar's methodology.

Figure 3. Reconstructed pixel-level Kaplan-Meier curves

SC2HR.xlsx illustrates the interpretation of survival curves at the pixel level by SC2HR.py. "C" and "c" denote the control group, while "R" and "r" represent the research group. Uppercase "C" or "R" indicates the pixel locations identified by SC2HR.py where the Y-axis value is consistent across three consecutive pixels. In contrast, lowercase "c" or "r" marks locations where the Y-axis value differs from the surrounding pixels by just one pixel. Adjusting this "c" or "r" by one pixel would align all three Y-values for at least three consecutive instances. The X-axis and Y-axis represent time and survival probabilities, respectively. For the X-axis, the first row displays the pixel-level location in the 10th digit, and the second row shows the index in the 1st digit. The third row indicates the time, computed based on the origin on the X-axis, the tick on the X-axis, and its value. Similarly, for the Y-axis, the first descriptive column under "===[...]===", labeled "Pixel," refers to the pixel-level location. The second column, labeled "Percent," represents the survival probability multiplied by 100. The third and fourth columns display the number of patients in the control and research groups at that specific pixel on the Y-axis, respectively.

4. Discussion

This study introduces SC2HR.py, an all-in-one software solution designed to enhance meta-analyses of time-to-event data. SC2HR.py allows researchers with basic knowledge of Kaplan-Meier (KM) plots to intuitively estimate hazard ratios from survival curves with improved precision and accuracy. Compared to previous general-purpose plot digitization software, SC2HR.py has demonstrated superior or at least equivalent precision in corrected coordinates, as validated by F-test results. Users of SC2HR.py can verify the accuracy of these coordinates themselves by consulting the pixel table displayed on the right side of the GUI application.

SC2HR.py's innovative approach to Parmar's methodology at the pixel level enables it to utilize general information about KM plots without requiring knowledge of Parmar's methodology. This facilitates the correction of user inputs for increased precision and accuracy at the pixel level and allows for the automatic construction of survival probabilities to apply Parmar's methodology. Thus, it eliminates the need for manually identifying coordinates on survival curves using a ruler or generic software, along with an understanding of Parmar's methodology.

Notably, this study does not propose modifications to Parmar's methodology itself but rather offers specialized software for the pixel-level semi-automation of this methodology, particularly enhancing the pre-processing phase of acquiring survival probabilities. The results can be verified against referenced values or using Tierney's spreadsheet, as discussed in the Results section. Furthermore, each line of open-source code relevant to Parmar's methodology has been written with readability in mind and peer-reviewed for correctness in terms of variable naming and process sequencing.

However, SC2HR.py has its limitations. Firstly, the default GUI size in version 1.0 is optimized for a 15-inch laptop monitor, suggesting that larger image files could potentially improve accuracy and precision in estimating hazard ratios. Nevertheless, Parmar has noted that published survival curves may be too small for accurate reading, indicating that simply enlarging the image might not be beneficial [2]. Secondly, the estimation of hazard ratios (HR) from a survival curve via SC2HR.py significantly depends on the collected survival probabilities, which in turn are contingent on the quality of the images, such as clarity and the scale of numerical axes [6]. Lastly, while the pixel-level semi-automation of Parmar's methodology addresses several issues within the scope of this study, further advancements in facilitating the conversion from KM plots to hazard ratios will require additional expertise in this domain.

Supplemental Figure 2. Censoring Issue and Applying the Constant Rate of Censoring Assumption

Censoring results from a lack of knowledge about what happens beyond the study endpoint for reasons unrelated to the outcome of interest. Parmar et al. proposed reflecting censoring by assuming a constant rate of censoring in their methodology's appendix section. However, since a constant rate of censoring often does not hold in real-life examples, and information on minimum and maximum follow-up in trials tends to be unavailable, applying the algorithm introduced in Parmar's appendix is typically not feasible.

(A) In practice, censoring is commonly encountered and can be indirectly observed when comparing the proportion of survival curves with the information in tables for the number of patients at risk at given times, which is often attributed to censoring. For instance, in the dataset and KM plot published by Takauji et al., at 10 hospital days, the survival probabilities for ECMO and Control groups - approximately 0.92 and 0.66, respectively - seem to correspond with the ratio of patients at risk to the initial patient numbers, 23/25=0.92 (A-1) and 59/89=0.66 (A-2). Conversely, at 50 hospital days, the survival probabilities for ECMO and Control groups, 0.6 and 0.34, do not align as expected with the ratios, 10/25=0.4 (A-3) and 16/89=0.18 (A-4), indicating the influence of censoring.

(B) In the absence of censoring, the proportions derived from survival curves should match the information in tables for the number of patients at risk at a given time. In the KM survival curve from Combes et al., at 50 hospital days, the survival probabilities for ECMO and control groups, 0.67 and 0.58, closely match the ratios of patients at risk to the initial patient numbers, 83/124=0.67 (B-3) and 72/125=0.58 (B-4). Therefore, for the dataset from Combes et al., SC2HR.py was tested without applying the algorithm introduced in the Appendix section of Parmar's methodology.

5. Conclusion

SC2HR.py is designed to facilitate the estimation of hazard ratios from survival curves by providing a GUI environment in which users with basic knowledge of KM plots can intuitively input information necessary for the integrated computer system to employ Parmar's methodology. This enhancement in the accuracy and precision of user input recognition, and consequently in hazard ratio estimates, positions SC2HR.py alongside several prior efforts aimed at streamlining Parmar's methodology. By equipping researchers with a tool that incorporates different developers' perspectives, SC2HR.py enhances confidence in analyzing and reporting hazard ratios and their variances.

Software Instruction in Relation to Patent Application

Installation

After installing the latest version of Python (version 3.x or greater) on your operating system, please open SC2HR.py using Python's Integrated Development and Learning Environment (IDLE). To install the required Python libraries such as PIL, numpy, and openpyxl, you can open a terminal window and type the following commands:

sudo pip3.8 install Pillow

sudo easy_install-3.8 numpy

sudo easy_install-3.8 openpyxl

Python IDLE allows you to execute SC2HR.py by pressing the F5 button or selecting "Run Module" from the menu. The Graphical User Interface (GUI) of SC2HR.py will then be displayed on your monitor. The GUI is divided into two parts: the left part, known as a canvas, where the Kaplan-Meier survival curve image file will be loaded, and the right part, which contains two tabs. The 'Step' tab highlights the current instruction step in red font; completing a step de-highlights it and highlights the next step. The 'Pixel' tab, functioning like a magnifying-glass feature, displays both the actual location marked by a researcher and the corrected location intended by the researcher at the pixel level. Users can manually select one of these tabs, but the relevant tab for the current stage of Parmar's methodology will be automatically chosen as needed.

To START, click the beige canvas on the left.

Upon first executing SC2HR.py, this instruction will be highlighted in yellow with the font in reddish bold. Please click on the beige-colored part of the GUI on the left, which will open a pop-up directory folder. Select your graphical image of the Kaplan-Meier plot in file extension formats 'png', 'jpg', or 'gif'.

An origin on the x-axis (1101) => A tick on the x-axis (1102)

On the loaded Kaplan-Meier survival curve image, mark the origin on the X-axis by clicking and holding (mouse-down) and then mark the largest tick on the X-axis by releasing the mouse button (mouse-up).

A tick on the y-axis (1103) => An origin on the y-axis (1104)

Similarly, mark the tick on the Y-axis that represents a survival probability of 1.0 by clicking and holding (mouse-down) and then mark the origin on the Y-axis by releasing the mouse button (mouse-up). Concentric circles will appear at the X-coordinate of the origin on the X-axis and the Y-coordinate of the tick on the Y-axis.

Survival curve@Control (1105)

Mark the survival curve of the control group by pressing and holding the mouse button from the center of the concentric circles to the end of the curve.

Survival curve@Research (1106)

Similarly, mark the survival curve of the research group by continuing to press down all the way.

Fill in the following forms:

Type in the number of patients in the research and control groups in "Effective number@Control" and "Effective number@Research", respectively, representing the "effective number alive at the start of the time interval." Fill in the entry for "Tick@X-axis," which requires the largest number indicating the rightmost tick on the X-axis.

FOR ANALYSIS, click the previous canvas on the left

Clicking the canvas on the left side of the GUI will save the output in SC_output.xlsx, and log information will be available in the Python shell.

[Result] ln(HR): 0.0, ln(variance): 0.0

The estimated natural log of the HR and log variance from the Kaplan-Meier plot will then be available.

Reference

1. Smith, C.T. Meta-analysis of time-to-event data. 2018 [cited 2019 March, 29th]; Available from: https://training.cochrane.org/resource/meta-analysis-time-event-data.
2. Parmar, M.K., V. Torri, and L. Stewart, Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. Stat Med, 1998. 17(24): p. 2815-34.
3. Chu, D.K., et al., Mortality and morbidity in acutely ill adults treated with liberal versus conservative oxygen therapy (IOTA): a systematic review and meta-analysis. Lancet, 2018. 391(10131): p. 1693-1705.
4. Roh, H.F., et al., Pulmonary resection for patients with multidrug-resistant tuberculosis based on survival outcomes: a systematic review and meta-analysis. Eur J Cardiothorac Surg, 2017. 52(4): p. 673-678.
5. Tierney, J.F., et al., Practical methods for incorporating summary time-to-event data into meta-analysis. Trials, 2007. 8: p. 16.
6. Guyot, P., et al., Enhanced secondary analysis of survival data: reconstructing the data from published Kaplan-Meier survival curves. BMC Med Res Methodol, 2012. 12: p. 9.
7. Bormann, I. Digitizer software - digitize a scanned graph or chart into (x,y)-data.; Available from: http://www.digitizeit.de/.
8. Foundation, P.S. Welcome to Python.org. Available from: https://www.python.org.
9. Rich, J.T., et al., A practical guide to understanding Kaplan-Meier curves. Otolaryngol Head Neck Surg, 2010. 143(3): p. 331-6.
10. Combes, A., et al., Extracorporeal Membrane Oxygenation for Severe Acute Respiratory Distress Syndrome. N Engl J Med, 2018. 378(21): p. 1965-1975.
11. Ingle, J.N., et al., Randomized trial of tamoxifen alone or combined with aminoglutethimide and hydrocortisone in women with metastatic breast cancer. J Clin Oncol, 1986. 4(6): p. 958-64.
12. Tierney, J.F., et al., Response to: Practical methods for incorporating summary time-to-event data into meta. Authors' reply. Trials, 2013. 14: p. 391.
13. Takauji, S., et al., Respiratory extracorporeal membrane oxygenation for severe sepsis and septic shock in adults: a propensity score analysis in a multicenter retrospective observational study. Acute Med Surg, 2017. 4(4): p. 408-417.
14. Hoyle, M.W. and W. Henley, Improved curve fits to summary survival data: application to economic evaluation of health technologies. BMC Med Res Methodol, 2011. 11: p. 139.
15. Pocock, S.J., T.C. Clayton, and D.G. Altman, Survival plots of time-to-event outcomes in clinical trials: good practice and pitfalls. Lancet, 2002. 359(9318): p. 1686-9.

Note:

As long as this article and the name of the trademark are formally and reasonably referred to, and as long as a user accepts the license agreement form (https://ngene.org/download.html), it will be an honor for the source code being openly employed by the public to contribute to accelerating the advancements in the methodology of meta-analysis and relevant fields.