Introduction

The advent of Intensity-Modulated Radiotherapy (IMRT) and Volumetric-Modulated Arc Therapy (VMAT), which provide highly conformal radiotherapy delivery, has significantly heightened the intricacy and complexity of radiotherapy planning and delivery. Modulated planning dose distributions exhibit much higher levels of heterogeneity and conformality, achieved through optimization of dynamic dose rates and movements of Multi-Leaf Collimator (MLC), gantry, couch, and collimator. However, the intricate nature of MLC movements, coupled with limitations in Treatment Planning System (TPS) dose calculations, introduces several factors that may contribute to discrepancies between planned and delivered doses, which impacts treatment accuracy.

Patient-specific RapidArc Quality Assurance (PSQA) is an essential part of the implementation of clinical radiotherapy. The primary objective of the IMRT/VMAT routine pre-treatment verification procedure is to ensure the safety and accuracy of the treatment process.

Different detectors and devices have been used to perform the pre-treatment IMRT/VMAT QA verification. Compared to ion chamber-based detector arrays, an Electronic Portal Imaging Device (EPID) has more measurement points with high-resolution two-dimensional digital data [ 1 - 3 ]. Radiochromic film exhibits a high resolution in dose measurement; however, the process is time-consuming and expensive [ 4 ]. Consequently, EPIDs play a crucial role in dose verification [ 5 ].

The gamma analysis method proposed by Low et al. is widely accepted and utilized to evaluate PSQA . The conventional approach of PSQA has the limitation of revealing the real-time trend of the QA process in the radiotherapy department. The current PSQA analysis, as indicated in the study by Palaniswaamy et al. [ 8 ], adopts a universal tolerance threshold, which raises the risk of false negatives and false positives, leading to treatment delays and reduced efficiency. Many departments apply the same tolerance criteria across various lesions to overcome this limitation. The American Association of Physicists in Medicine (AAPM) Task Group-218 report [ 9 ], and previous studies [ 10 , 11 ] have given Statistical Process Control (SPC) to address the variation in IMRT or VMAT PSQA and establish equipment-specific and site-specific process-based tolerance and action limits

In the classical approach, a sample’s mean and standard deviation are generally used to characterize a process. SPC provides a more robust method for monitoring the stability of process variability over time. SPC is a quality control method that uses statistical techniques to monitor and control a process, ensuring it operates at its full potential. The main benefit of SPC is its ability to detect shifts or trends in a process before they lead to clinically significant problems, allowing for early intervention and process improvement. In PSQA, SPC determines action limits when universal limits are not appropriate. A control chart is mostly used to detect deviations from an ideal state of statistical control, even when the process remains within clinical action limits, to determine process stability and enhance performance by minimizing variation. In PSQA analysis, the limits can vary depending on equipment, site-specific factors, treatment technique, or the gamma criteria used for PSQA analysis. Therefore, tolerance limits are defined within, which a process usually operates, subject only to random errors. Action limits set the minimum performance level such that PSQA measurements outside these limits could negatively impact patient outcomes. Therefore, the I-MR (Individual-Moving Range) is used to identify out-of-control behavior in PSQA measurements and establish limits, helping to monitor individual values and their variation over time. The Exponentially Weighted Moving Average (EWMA) chart is designed to detect process drift as small as 1σ by emphasizing recent data points, making it sensitive to small, gradual changes in the process. Furthermore, the Cumulative Sum (CUSUM) chart tracks cumulative deviations from the target, allowing for early detection of small but persistent shifts, which might indicate a drift in treatment delivery quality. This combined approach addresses issues before clinically reaching unacceptable thresholds or action limits.

Previous studies have demonstrated the utility of SPC tools in monitoring the output constancy of linear accelerators [ 12 - 14 ]. However, the available evidence remains insufficient to establish robust, site-specific tolerance and action limits. Additionally, the intricacies associated with head and neck treatment planning and dose delivery, particularly due to the proximity of organs at risk, have been highlighted in earlier research [ 15 , 16 ]. Furthermore, more complexity in head and neck treatment plans is required to comply with the tolerance limit during PSQA. Therefore, a specific selection of gamma criteria is required, depending on the treatment technique and measurement system.

Van Esch et al. [ 17 ] were the first to propose an approach for establishing QA standards through statistical analysis and integrating them into the treatment preparation process. In addition, by employing specific control charts within a QA process, the random and systematic errors can be differentiated in a stream of time-resolved data using action thresholds developed through statistical process control, as proposed by Pawlicki et al., [ 18 ]. The objective of an optimal PSQA procedure is to minimize both the number and magnitude of systematic errors, which can be quantitatively achieved by establishing an appropriate action threshold. Process behavior charts, which were first developed by Shewhart as described by Reynolds et al. form the foundation of SPC [ 19 ]. Additionally, it is essential to consider the impact of sample size on the control chart, including the Lower Control Limit (LCL), Center Line (CL), and Upper Control Limit (UCL). LCL corresponds to the threshold limit for gamma passing rate, commonly set at 95%, widely used in radiotherapy centers. However, this threshold may vary depending on factors, such as the selected gamma criterion, resolution of the measurement device, pathology, and the treated area [ 20 , 21 ]. Furthermore, the complexity of radiotherapy planning varies across different clinical sites, which requires setting corresponding tolerance and intervention limits. Previous studies have primarily focused on setting Action Limits (AL) and Control Limits (CL) for PSQA based on single gamma (global) criteria or data collected from different equipment. Also, limited literature focuses on interventions for LCL when it falls below the recommended threshold.

The current study aimed to assess the application of SPC to evaluate tolerance and action limits of PSQA for different gamma criteria for head and neck, brain, thorax, and pelvis sites for portal dosimetry.

Material and Methods

Study Cohort

In this analytical study, a two-stage approach is employed to analyze PSQA data obtained from gamma analysis of the brain (25), head and neck (50), thorax (25), and pelvis (50) treatment plans using various gamma criteria. Initially, PSQA results, which include percent gamma passing rates for different gamma criteria, were used to establish control limits. The patient cohort had varying prescription doses and fractionation schedules based on their cancer stage. Subsequently, to evaluate the established limits, the PSQA results of 375 HN treatment plans were monitored over a year to assess action and tolerance limits for a larger population. Additionally, to identify systematic and random errors in the process, three distinct statistical control methods were employed: Individual moving range (I-MR), EWMA, and CUSUM were applied for this analysis, presenting a novel approach to the study.

Treatment Planning and generation of the verification plan

All RapidArc plans were created using two full arcs using a 6 MV photon beam flattening-filtered photon beam using two complete arcs, one in the clockwise (CW) and the other in the counterclockwise (CCW) direction with a collimator rotation of 30 or 330 in the Varian Eclipse treatment planning system (TPS; version 15.6, Varian Medical Systems, Palo Alto, CA). An Anisotropic Analytical Algorithm (AAA) with a 2.5 mm grid size was used to calculate the dose. A verification plan was created for each patient based on the composite plan at the actual treatment angle. The Varian TrueBeam SVC with a 6 MV photon beam at nominal dose rates of 600 MU/min and 120 multi-leaf collimators (Millennium MLC; 5 mm central 40 leaf pairs spanning 1 cm and 5.0 mm leaf pairs in the periphery) were used to deliver all verification plans.

RapidArc QAs using Portal Dosimetry

For this current study, we used the Varian Portal Dosimetry System with amorphous silicon (a-Si) EPID for the QA of RapidArc. In this study, the Varian Electronic Portal Dosimetry System, equipped with an amorphous silicon (a-Si) EPID, was utilized for RapidArc quality assurance. The Varian Portal Dosimetry system (version 11.0) is composed of three key elements: (i) the portal imager used for capturing images, (ii) the Portal Dose Image Prediction (PDIP) module within the Eclipse TPS, and (iii) the ARIA portal dosimetry review workspace, which is utilized for analyzing the RapidArc QA test.

The Varian TrueBeam PortalVision imager (aS1200 amorphous silicon) has an active area of 40×40 cm² at a Source-to-Detector Distance (SDD) of 100 cm and an array of 1190×1190 pixels and pixel pitch of 0.336 mm. The EPID images were calibrated with dark field and flood field and scaled to 1 Calibrated Unit (CU)=1 cGy at the isocenter plane at 600 MU/min. A profile correction using a diagonal beam profile measured at the d_max in water using a 40×40 cm² field was applied to calibrate the EPID detector following the vendor’s instruction.

Gamma Evaluation

The Gamma analysis quantitatively compares the measured dose distribution with a calculated dose distribution by calculating each point’s Gamma value. The agreement between the measured and calculated dose distributions is assessed based on two criteria: Dose difference (DD) in % and Distance-to Agreement (DTA) in mm. The Gamma analysis produces Gamma Index Values, and ≤1 indicates passed or otherwise failed. The percentage of passing points in the Gamma distribution is called the Gamma Pass Rate (GPR). Further, in the current study, we have used 3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm gamma criteria to set the tolerance and action limits for various sites for both local and global gamma with a 10% threshold.

Control charts

A control chart is a graphical representation that displays data over time, with the horizontal axis representing time and the vertical axis showing the measured parameter. The chart features a Center Line (CL), which represents the average value of the vertical axis variable across the time period. Additionally, it includes an Upper Control Limit (UCL) and an LCL. From a statistical perspective, the process is considered to be in control at any given point on the chart as long as the process variable falls between the UCL and LCL range.

Time-weighted charts viz Exponentially Weighted Moving Average (EMWA) charts and CUSUM charts were created to detect more minor changes. To identify smaller shifts, specialized charts, such as EMWA and CUSUM were developed, both of which incorporate time-weighting techniques.

This work uses three control charts within SPC: the I-MR chart, the EWMA chart, and the CUSUM chart. The control charts are typically adopted under the assumption of normality. Accordingly, the data were tested for normal distribution as a pre-requisite to the use of control charts. To evaluate the normality of the hypothesized distribution, the Anderson-Darling test was employed with a significance level of 0.05. Additionally, normal probability plots and histograms were generated. The control charts presented in this study were created using Minitab® 20 software, a specialized statistical tool designed for quality improvement and process control analysis. The software offers comprehensive features for creating and analyzing control charts, enabling detailed statistical evaluation. SPSS version 25.0 software (SPSS Inc.) was also used for statistical analysis.

This research employs a conventional control chart known as the I-chart. Given that the maximum achievable pass rate is 100%, the I-chart in this study lacks an Upper Control Limit (UCL). Instead, the CL and LCL for an I-chart are used at the tolerance limits set in AAPM TG-218. The CL and LCL for an I-chart are calculated using the following equation:

$\begin{array}{cc}\mathrm{CL}=\frac{1}{n}{\sum }_1^{n}x& \mathrm{(1)}\end{array}$

$\begin{array}{cc}\mathrm{LCL}=\mathrm{CL}-2.660\stackrel{\_}{\mathrm{mR}}& \mathrm{(2)}\end{array}$

Where x stands for an individual gamma pass rate per cent for RapidArc QA, n is the total number of measurements, and $\stackrel{\_}{\mathrm{mR}}$ represents the average moving range of all gamma pass rate (in percentage) data.

$\begin{array}{cc}\stackrel{\_}{\mathrm{mR}}=\frac{1}{n-1}{\sum }_{\mathrm{i=2}}^n{x}_i-x_{\mathrm{i-1}}& \mathrm{(3)}\end{array}$

Exponentially weighted moving average (EWMA)

The EWMA chart assigns the highest weight to the most recent data point while the influence of earlier data decreases, with more distant data receiving exponentially less weight. The remaining data points are weighted based on their chronological closeness to the current data. The combination of I-MR and EWMA charts may provide the most effective approach for real-time process monitoring [ 22 ]. The following mathematical equation defines the EWMA,

$\begin{array}{cc}{\mathrm{EWMA}}_i=\lambda x_i+(1-\lambda ){\mathrm{EWMA}}_{\mathrm{i-1}}& \mathrm{(4)}\end{array}$

Where EWMA_i-1 is the EWMA value of the x_i-1^th data point, and x_i is the observation data at the time i, and x represents an individual gamma pass rate percentage of RapidArc QA. λ is the weighting factor varying from (0<λ≤1). A larger λ assigns more weight to recent data. When λ is set to 1, the EWMA chart becomes an I-Chart. Typically, the value of λ is chosen between 0.15 and 0.3. For this study, we have selected λ=0.2 [ 14 ].

The mathematical formulas for calculating the EWMA control limits are provided below, calculated using the following equations,

$\begin{array}{cc}\mathrm{CL}=\frac{1}{n}{\sum }_1^n{\mathrm{EWMA}}_i& \mathrm{(5)}\end{array}$

$\begin{array}{cc}\mathrm{LCL}=\mathrm{CL}-\mathrm{2.66}\stackrel{\_}{\mathrm{mR}}\sqrt{\frac{2}{2-\lambda }[1-{(1-\lambda )}^{\mathrm{2i}}]}& \mathrm{(6)}\end{array}$

$\begin{array}{cc}\mathrm{UCL}=\mathrm{CL}-\mathrm{2.66}\stackrel{\_}{\mathrm{mR}}\sqrt{\frac{2}{2-\lambda }[1-{(1-\lambda )}^{\mathrm{2i}}]}& \mathrm{(7)}\end{array}$

$\begin{array}{cc}\stackrel{\_}{\mathrm{mR}}=\frac{1}{n-1}{\sum }_{\mathrm{i=2}}^n{\mathrm{EWMA}}_i-{\mathrm{EWMA}}_{\mathrm{i-1}}& \mathrm{(8)}\end{array}$

Where n represents the total number of measurements, and $\stackrel{\_}{\mathrm{mR}}$ indicates the average moving ranges for all gamma pass rate percentages.

Cumulated sum (CUSUM)

CUSUM analyses subgroup averages relative to a specified target value and identifies movements by comparing them with historical statistics. This is an objective that I-charts and EWMA charts do not achieve, as they compute statistics based on sample observations rather than set target values [ 23 ].

A CUSUM chart aggregates deviations exceeding and falling short of the target value in two distinct variables, C⁺ and C^-. The charts for the two variables are upper and Lower CUSUM, respectively. In computing the deviation, a set value is often used, commonly assumed to be half the standard deviation of the samples [ 19 ], as follows:

$\begin{array}{cc}{C}_i^{+}=\max [0,x_i-(T+K)+C_{\mathrm{i-1}}^{+}]& \mathrm{(9)}\end{array}$

$\begin{array}{cc}{C}_i^{-}=\min [0,x_i-(T-K)+C_{\mathrm{i-1}}^{-}]& \mathrm{(10)}\end{array}$

Where T represents the target value, whereas K denotes the slack value or leeway, contingent upon the extent of change that the CUSUM chart must detect, time-varying control charts are typically employed to identify variations over one standard deviation. The Upper and Lower Control Limits for CUSUM charts are predetermined. The process is deemed out of control when either the top or lower CUSUM line surpasses certain thresholds.

Control limits for CUSUM charts function as thresholds, indicating that the CUSUM chart is deemed out of control when exceeded. To assess the consistency of each PSQA utilizing CUSUM control charts, K is selected as 0.5, and the decision interval is set to 5.

Before collecting the data for each PSQA, it was verified that daily QA displays predictive behavior and is within the process behavior limits using a daily QuickChek quality assurance device.

Action limits and control limits

The action limits are established to define a minimum acceptable standard for performance. Typically, the range for action limits is more lenient compared to the more stringent parameters set by control or warning limits. There are two primary forms of action limits. The first type consists of standardized recommendations that are invariant from institute to institute. The second type is tailored to the individual institute, relying on a thorough analysis of the specific data available to that institution. The AAPM Task Group 218 advocates for this second, localized approach as the preferred approach to calculating action limits.

The action limit is determined using the following equation:

$\begin{array}{cc}\mathrm{\Delta AL}=\beta \sqrt{{\sigma }^2+{(\stackrel{-}{X}-T)}^2}& \mathrm{(11)}\end{array}$

Where ∆AL denotes the difference between the upper control limit and the width of the action limits, typically expressed as ±AL/2, and T represents the process target value to be achieved. For a known target, generally in gamma analysis, the gamma pass rate value is chosen to be 100%. σ and $\stackrel{-}{X}$ are standard deviations also measured as process variance and process average, respectively. Based on existing literature, β=6 is an appropriate value for the current study [ 9 ]. The Lower Action Limit (LAL) is set from the above-derived width.

To assess the baseline performance of the control charts, we included data from brain (25), head and neck (50), thorax (25), and pelvis (50) to establish the LCL, which is most important for the PSQA. The first ten patients’ data was used to calculate the I-MR chart to set the tolerance and action limits. Then, the EWMA and Cumulative Sum (CUSUM) chart were used to study slight drift in the control chart. Finally, 375 head and neck patients were used to monitor the process behavior for a large population to establish limits after setting the tolerance and action limits.

Results

The SPC methods were applied to the parameters measured by the portal dosimetry setup for 6 MV photons. Initially, I-MR charts detected major deviations, followed by EWMA charts to confirm the findings, resulting in two sets of control charts for each parameter. The analysis assessed the effectiveness of SPC in our treatment delivery in RapidArc planning. Control charts feature a blue centre line and red upper and lower control limits. Points falling within the limits are blue, while those outsides are red. The CUSUM chart highlights a noticeable shift in the process mean from the target value, as it crosses the specification limit early in the data points and continues to show a consistent slope over time.

All control charts (I-MR, EWMA, and CUSUM) were generated for local and global gamma criteria of 3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm. The corresponding charts are presented in Figure 1A, B, C, and D depict the Individual-Moving Range (I-MR) charts for the gamma pass rate percentage for 3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm global gamma criteria. In contrast, Figure 2A, B, C, and D illustrate the variations in the LCL across different local gamma criteria for head and neck sites for local gamma with 3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm criteria. The first 10 data points were selected to calculate the initial control limits. During the chart preparation, if the data point exceeded the initial LCL, it was excluded, and the control limits were recalculated based on the remaining data. The identification, elimination, and recalculation process were repeated until all data points were within the control limits. Table 1 compares the LAL and AL obtained for the brain (25), head and neck (50), thorax (25), and pelvis (50) and the evaluation of all CL and LCL limits. The I-chart and EWMA charts indicate that the process has reached stability.

Figure 1.A, B, C, and D show the Individual-Moving Range (I-MR) charts for 3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm global gamma criteria. These I-MR control charts provide a detailed analysis of the variations of individual values in patient-specific quality assurance from Gamma Analysis, using different dose difference and distance-to-agreement criteria for head and neck site.

Figure 2.A, B, C, and D show the Individual-Moving Range (I-MR) charts for 3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm using local gamma criteria. These I-MR control charts illustrate the variations in the Lower Control limit (LCL) across different local gamma criteria for head and neck site.

From the I-Chart analysis, it was observed that, for the head and neck site, the CL (%) and LCL (%) under different gamma criteria (3%/3 mm, 3%/2 mm, 2%/2 mm, and 2%/2 mm) were as follows: 99.39%, 99.01%, 98.45%, 97.37%, and 96.82%, 96.0%, 94.05%, 89.42% for global gamma, and 97.72%, 96.61%, 96.01%, 93.82%, and 91.40%, 88.54%, 88.03%, 83.06% for local gamma. The LAL (%) was 96.35%, 94.34%, 91.88%, and 86.60% for global gamma and 87.37%, 82.33%, 79.93%, and 70.73% for local gamma criteria.

It is observed that, except for the 2%/2 mm global criterion, all the resulting PSQAs were higher than the LCL of 90%. On the contrary, only the 3%/3 mm local criterion had an LCL exceeding 90%.

Figures 3 and 4 present EWMA charts for both global and local gamma criteria, illustrating the gamma analysis results for the 3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm criteria for head and neck sites. These charts emphasize recent data more heavily, effectively highlighting subtle trends and shifts in quality assurance metrics over time. Figures 5 and 6 also illustrate CUSUM charts, summarizing the cumulative deviations from the target values established in the gamma analysis for the same global and local gamma criteria. These charts effectively capture small but persistent shifts in quality assurance data, which may indicate a gradual drift from acceptable treatment delivery standards.

Figure 3.A, B, C, and D illustrate the Exponentially Weighted Moving Average (EWMA) charts, which provide an observation of the gamma analysis results for the 3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm global gamma criteria for the head and neck site. These charts emphasize recent data more heavily than earlier observations, effectively highlighting subtle trends and shifts in quality assurance metrics over time.

Figure 4.A, B, C, and D show Exponentially Weighted Moving Average (EWMA) charts for the gamma analysis results using 3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm local gamma criteria for head and neck site. The local criteria enhance sensitivity to specific regions in these areas, with the EWMA charts emphasizing recent data and effectively capturing trends and shifts in quality assurance metrics over time.

Figure 5.A, B, C, and D display Cumulative Sum (CUSUM) charts summarizing the cumulative deviations from the target values established in the gamma analysis for the 3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm global gamma criteria for the head and neck site. These charts effectively capture small but persistent shifts in quality assurance data, which may indicate a drift from acceptable treatment delivery standards.

Figure 6.A, B, C, and D display Cumulative Sum (CUSUM) charts that track cumulative deviations from target values based on the gamma analysis for the 3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm local gamma criteria for the head and neck site. The CUSUM chart identifies small, consistent shifts in treatment accuracy over time by focusing on localized dose variations, allowing for the early detection of gradual drifts in quality assurance.

The I-MR, EWMA, and CUSUM of the head and neck patients for 3%/2 mm global criteria are shown in Figure 7A. It presents the I-MR chart, which monitors individual measurements and their moving range to identify shifts in pre-treatment quality assurance results. Figure 7B shows the EWMA chart, which applies exponentially decreasing weights to past observations of recent data and provides a smoothed view of trends over time. 7C features the CUSUM chart, which tracks cumulative deviations from target values, allowing for the detection of small but consistent shifts in treatment accuracy.

Figure 7. Illustrates the variations in Control Charts of Individual-Moving Range (I-MR), Exponentially Weighted Moving Average (EWMA), and Cumulative Sum (CUSUM) charts, for a large cohort of samples analyzed under the 3%/2 mm global gamma criteria for head and neck sites. A) presents the I-MR chart, which monitors individual measurements and their moving range to identify shifts in pre-treatment quality assurance results. B) shows the EWMA chart, which applies exponentially decreasing weights to past observations of recent data and providing a smoothed view of trends over time. C) features the CUSUM chart, which tracks cumulative deviations from target values, allowing for the detection of small but consistent shifts in treatment accuracy.

The I-MR, EWMA, and CUSUM evaluation details can be found in the supplementary file.

From the I-Chart analysis, it was observed that, for the brain site, the CL (%) and LCL (%) under different gamma criteria (3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm) were as follows: 99.64%, 99.43%, 99.05%, 98.37%, and 98.16%, 96.94%, 96.69%, 94.17% for global gamma, and 98.89%, 97.80%, 97.21%, 95.34%, and 95.73%, 91.45%, 91.88%, 85.77% for local gamma. The LAL (%) was 95.77%, 93.18%, 90.90%, and 84.00% for global gamma and 89.93%, 88.70%, 87.05%, and 78.68% for local gamma criteria.

From the I-Chart analysis, it was observed that for the thorax site, the CL (%) and LCL (%) under different gamma criteria (3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm) were as follows: 98.29%, 97.59%, 95.40%, 93.21%, and 94.83%, 92.48%, 88.10%, 83.84% for global gamma, and 97.34%, 95.58%, 94.32%, 90.48%, and 93.50%, 89.58%, 88.00%, 80.64% for local gamma. The LAL (%) was 92.34%, 89.05%, 82.45%, and 70.75% for global gamma and 88.45%, 80.98%, 75.90%, and 59.94% for local gamma criteria.

From the I-Chart analysis, it was observed that for the pelvis site, the CL (%) and LCL (%) under different gamma criteria (3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm) were as follows: 98.50%, 97.96%, 96.21%, 92.64%, and 94.84%, 92.11%, 89.09%, 78.49% for global gamma, and 97.21%, 95.16%, 93.62%, 89.19%, and 91.95%, 86.32%, 83.42%, 73.44% for local gamma. The LAL (%) was 92.85%, 90.24%, 83.71%, and 66.65% for global gamma and 87.42%, 77.87%, 71.84%, and 64.78% for local gamma criteria.

Discussion

The PSQA is an important quality assurance test for intensity-modulated planning, typically performed in a radiotherapy department before actual treatment delivery. In the current study, we adopted SPC tools to analyze and establish Tolerance Limits (TL) and AL using portal dosimetry for different sites with different gamma criteria (3%/3 mm, 3%/2 mm, 2%/3 mm, and 2%/2 mm). Furthermore, utilizing I-MR, EWMA, and CUSUM charts, we monitored the PSQA results and assessed the established action and tolerance limits for a larger population. The continuous process of charting data points and identifying systematic errors represents an ongoing endeavor to minimize variations in a process. Head & neck emerges as a challenging site in terms of planning and delivery complexities. In the current study, the LCLs for various sites, as presented in Table 1, have been assessed, and these LCLs are the cut-off limit for separating systematic errors from random variation. Moreover, all plans in this study were developed using the RapidArc technique. The advantage of this technique is smoother dose distribution, more conformal plans, and treatment time [ 24 ].

The tolerance and action limits were 96.82%, 96.0%, 96.35%, and 94.34% from the global gamma criteria of 3%/3 mm and 3%/2 mm, respectively. These values align with the stipulated requirements in the AAPM TG-218 report, where recommended tolerance and action limits were set at 95% and 90%. However, a notable decrease in local passing rates was observed compared to global criteria, particularly as the gamma criteria transitioned towards more stringent measures of 2%/3 mm and 2%/2 mm. For the H&N site, both the 2%/3 mm and 2%/2 mm criteria failed to meet the recommended tolerance level for global and local gamma criteria. However, 3%/3 mm global criteria had achieved the recommended action limit. Therefore, this process-based approach gives a tighter control of quality assurance procedures, reducing variability in outcomes. At the departmental level, integrating SPC into PSQA verification can enhance decision-making and improve the overall reliability of treatment delivery, ensuring that tolerance and action limits are consistently maintained for future PSQA processes.

The variation in gamma passing rates is influenced by normalization and criteria, as highlighted by Bailey et al., [ 25 ]. Consequently, our study aligns with this observation, demonstrating consistently higher passing rates in global gamma compared to local gamma under the same criteria and using an identical lower dose threshold. Furthermore, global gamma evaluation with 3%/3 mm criteria has raised concerns due to its reported lack of sensitivity and specificity for detecting the delivery of clinically significant patient dose errors. It is considered less clinically usable, as described by Yu et al., [ 26 ]. Therefore, our current study focused on the assessment of variation in the behavior of gamma evaluation for both local and global gamma evaluations with different criteria.

Figures 1 and 2 show the variation of individual values in patient-specific quality assurance, based on gamma analysis for both global and local gamma, using 3%/2 mm, 2%/3 mm, and 2%/2 mm criteria. The I-chart demonstrates that monitoring sudden changes can help detect abnormal plans. Therefore, if the data stays within the control limits, it suggests that the errors are purely random. On the contrary, if the data falls outside control limits, it signals to the physicist that an immediate review of the QA results is necessary. If the origin of the error is identified from the points outside the control limits, those errors are categorized as systematic. They should be corrected, ensuring the system returns to a controlled state. Sanghangthum et al. [ 27 ] found that VMAT planning gives more confidence and better efficiency in terms of QA compared to IMRT. Their lower control limit was 90% for VMAT and 85.0% for IMRT in the head and neck sites.

The I-Chart analysis showed that gamma passing rates declined as criteria became more stringent across all anatomical sites, with the greatest variability observed under the 2%/2 mm criteria. For the Brain site, CL dropped from 99.64% (3%/3 mm) to 95.34% (2%/2 mm), while the pelvis and thorax sites exhibited similar trends, especially in local gamma assessments, where LCL values fell as low as 73.44% for the pelvis. These findings align with recent studies by Xu et al. [ 28 ] and Russo et al. [ 29 ], highlighting the need for adaptive PSQA frameworks to address variability across different sites and more complex treatment plans. Incorporating SPC, as recommended by AAPM TG-218, offers a data-driven approach to improve PSQA consistency and optimize gamma passing rates, particularly when using stricter criteria.

According to Xiao et al. [ 30 ], it is suggested that a sample size exceeding 300 is necessary to confirm the permanent limits of individual control charts. Moreover, to overcome issues related to non-normality, a substantial sample size of at least 100 is required. We reviewed more than 300 samples in this study, demonstrating the accuracy and reliability of the control charts. Moreover, the passing rate parameter shows an increasing pattern from 160th to 235th duration of the collected data shown in Figure 7B, suggesting that an internal process parameter may be responsible for the gradual decline in the passing rate. However, this conclusion cannot be drawn solely from the I-MR chart analysis, as it is not sensitive to minor reductions in control. Reynolds et al. [ 19 ] and Vysakh et al. [ 31 ] found that the EWMA chart can be combined with the I-chart to detect large and small changes promptly. The advantage of the EWMA control chart is that it can detect the MLC problems responsible for the drift in the control chart. Therefore, the current study assessed the QA record and indicated that the MLC a T-nut was replaced and lubricated, and the dose calibration was repeated on the EPID. In addition, if there are two different EPIDs, it is challenging to ensure the measurement results. Hence, it is suggested by Sanghangthum et al. [ 32 ] that the control limits and tolerance limits should be calculated for the individual machine.

A CUSUM plot from Figure 7C illustrates the level of control established over the process, clearly indicating when actual values deviate from the set target value of the parameter. The target is achieved in typical PSQA for 3%/2 mm when a gamma passing rate is more than 97%. However, depending on the complexity of the planning and delivery, this rate may vary. In an ideal case with only random process variation, the CUSUM plot should exhibit random fluctuations near zero, as indicated by Pal et al. [ 13 ]. Random variation was observed in the CUSUM chart in the plot up to the 150th point. However, in our study, the process remains under control within the set limits.

Cui et al. [ 33 ] investigated dynamic IMRT cases using EPID. The results revealed CL, LCL, and AL of 98.13%, 96.05%, and 94.00%, respectively, mainly in cases of cervical and rectal cancer patients. Additionally, they identified a systematic error originating from the EPID, which indicates regular monitoring for accuracy and stability to prevent such trends. Moreover, they reported that significant shifts could result not only from systematic errors but also from the complexity of the plan. Therefore, they recommend investigating relevant parameter settings if failures persist, including DD and DTA criteria, suppression of low dose volume, type of normalization, dose registration (both measured and distributed), and the sample size for each control chart.

In a control chart, as studied by different researchers, there can be categories for the complex plans throughout the process. The primary machine parameters leading to IMRT QA failure are beam uniformity, symmetry, dose rate, output, and MLC calibration [ 34 ]. However, manual errors are primarily attributed to factors such as unadjusted SSD, incorrect comparisons, and erroneous additional build-up [ 20 ]. The advantages of using EPID in the current study are that it is a simple process, the absence of the need for extra build-up, no requirement for plane selection during planning, and convenience in positioning in RapidArc QA. More et al. [ 35 ] demonstrated that EPID portal dosimetry is an effective and reliable method for verifying intensity-modulated treatment plans, serving as a key tool in pre-treatment quality assurance.

Moreover, there may be some discrepancies between linac’s performance during the delivery of QA plans and its performance at the time of TPS commissioning, despite the use of periodic QA to mitigate systematic mistakes. Moreover, inherent mistakes arise from the application of various dosage calculation methods inside the TPS. Szczurek et al. [ 36 ] have shown that the impact of various algorithms or systematic flaws of the linac may obscure the negligible dose variances resulting from gantry or collimator defects. Mishra et al. [ 37 ] further showed that the selection of algorithm and the gamma criterion directly impact the gamma passing rate. Consequently, it is advisable to establish locally specified action levels attainable through the implementation of process-based tolerance and action limits. Action levels established locally must be tailored to the individual equipment, procedures, and case types, in addition to the physicist’s expertise. SPC provides a robust framework for detecting process drifts and ensuring consistent quality inpatient treatments. A two-stage approach is recommended for implementing SPC in PSQA analysis. This proactive approach helps radiotherapy departments identify and rectify deviations before they impact clinical outcomes, thereby enhancing radiotherapy treatments’ overall reliability and safety. However, proper selection of samples and knowledge of SPC tools are essential before implementation.

Conclusion

The present study provides valuable insights into the systematic and random errors associated with PSQA and establishes tolerance and action limits for different sites with different gamma criteria using portal dosimetry based on AAPM TG 218 guidelines. The combination of I-MR, EWMA, and CUSUM quality control charts demonstrated effective process control, identifying small and large drifts and maintaining results within the tolerance limits for RapidArc PSQA. Therefore, regular monitoring of the process of a control chart is essential.

Our findings indicate that a single gamma criterion cannot be universally applied to all sites due to the complexity of planning involving target and organ-at-risk locations. Therefore, institutions must develop their tolerance and action limits based on site- and technique-specific factors while also comparing these limits with international guidelines. The current study demonstrated the use of SPC in establishing tolerance and action limits for EPID-based PSQA. Future research should compare these limits across different dosimetry systems.

Authors’ Contribution

S. Manna participated in the design, data collection, data analysis, and manuscript draft, BK. Singh participated in the research design, data analysis, and manuscript draft, KJM. Das participated in the research design, data analysis, and manuscript draft. All the authors read, modified, and approved the final version of the manuscript.

Ethical Approval

Permission was obtained from KSSSCI, Lucknow, to conduct the research and use the resources needed.

A part of this study was presented at the International Conference on Medical Physics (ICMP), held December 6–9, 2023, in Mumbai, India. The abstract was published in the conference proceedings in Medical Physics International Journal, Vol. 11, No. 2, 2023, p. 87.

Funding

There is no funding source for this study.

Conflict of Interest

None

References

1. Bakhtiari M, Kumaraswamy L, Bailey DW, De Boer S, Malhotra HK, Podgorsak MB. Using an EPID for patient-specific VMAT quality assurance. Med Phys. 2011; 38(3):1366-73. DOI | PubMed
2. Boggula R, Birkner M, Lohr F, Steil V, Wenz F, Wertz H. Evaluation of a 2D detector array for patient-specific VMAT QA with different setups. Phys Med Biol. 2011; 56(22):7163-77. DOI | PubMed
3. Ibrahim AG, Mohamed IE, Zidan HM. Dosimetric comparison of amorphous silicon epid and 2d array detector for pre-treatment verification of intensity modulated radiation therapy. International Journal of Medical Physics, Clinical Engineering and Radiation Oncology. 2018; 7(4):438-52. DOI
4. Niroomand-Rad A, Chiu-Tsao ST, Grams MP, Lewis DF, Soares CG, Van Battum LJ, et al. Report of AAPM Task Group 235 Radiochromic Film Dosimetry: An Update to TG-55. Med Phys. 2020; 47(12):5986-6025. DOI | PubMed
5. Dogan N, Mijnheer BJ, Padgett K, Nalichowski A, Wu C, Nyflot MJ, et al. AAPM Task Group Report 307: Use of EPIDs for Patient-Specific IMRT and VMAT QA. Med Phys. 2023; 50(8):e865-903. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
6. Low DA, Harms WB, Mutic S, Purdy JA. A technique for the quantitative evaluation of dose distributions. Med Phys. 1998; 25(5):656-61. DOI | PubMed
7. Low DA, Dempsey JF. Evaluation of the gamma dose distribution comparison method. Med Phys. 2003; 30(9):2455-64. DOI | PubMed
8. Palaniswaamy G, Scott Brame R, Yaddanapudi S, Rangaraj D, Mutic S. A statistical approach to IMRT patient-specific QA. Med Phys. 2012; 39(12):7560-70. DOI | PubMed
9. Miften M, Olch A, Mihailidis D, Moran J, Pawlicki T, Molineu A, et al. Tolerance limits and methodologies for IMRT measurement-based verification QA: Recommendations of AAPM Task Group No. 218. Med Phys 2018; 45(4):e53-83. DOI | PubMed
10. Anetai Y, Sumida I, Kumazaki Y, Kito S, Kurooka M, Ueda Y, et al. Assessment of using a gamma index analysis for patient-specific quality assurance in Japan. J Appl Clin Med Phys. 2022; 23(10):e13745. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
11. Deng J, Liu S, Huang Y, Li X, Wu X. Evaluating AAPM-TG-218 recommendations: Gamma index tolerance and action limits in IMRT and VMAT quality assurance using SunCHECK. J Appl Clin Med Phys. 2024; 25(6):e14277. DOI
12. Binny D, Aland T, Archibald-Heeren BR, Trapp JV, Kairn T, Crowe SB. A multi-institutional evaluation of machine performance check system on treatment beam output and symmetry using statistical process control. J Appl Clin Med Phys. 2019; 20(3):71-80. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
13. Pal B, Pal A, Das S, Palit S, Sarkar P, Mondal S, et al. Retrospective study on performance of constancy check device in Linac beam monitoring using Statistical Process Control. Rep Pract Oncol Radiother. 2020; 25(1):91-9. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
14. Jassal K, Sarkar B, Munshi A, Roy S, Paul S, Mohanti BK, et al. Consistency analysis for the performance of planar detector systems used in advanced radiotherapy. Int J Cancer Ther Oncol. 2015; 3(1):1-9. DOI
15. Guo Y, Hu J, Li Y, Ran J, Cai H. Correlation between patient-specific quality assurance in volumetric modulated arc therapy and 2D dose image features. Sci Rep. 2023; 13(1):4051. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
16. Manna S, Singh BK, Das KJM. A comprehensive site-specific analysis of patient-specific quality assurance based on dose volume histogram of RapidArc treatment delivery. Rep Pract Oncol Radiother. 2024; 29(6):675-89. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
17. Van Esch A, Bogaerts R, Kutcher GJ, Huyskens D. Quality assurance in radiotherapy by identifying standards and monitoring treatment preparation. Radiother Oncol. 2000; 56(1):109-15. DOI | PubMed
18. Pawlicki T, Whitaker M, Boyer AL. Statistical process control for radiotherapy quality assurance. Med Phys. 2005; 32(9):2777-86. DOI | PubMed
19. Reynolds Jr MR, Stoumbos ZG. Should exponentially weighted moving average and cumulative sum charts be used with Shewhart limits? Technometrics. 2005; 47(4):409-24. DOI
20. Woon W, Ravindran PB, Ekayanake P, Vikraman S, Lim YY, Khalid J. A study on the effect of detector resolution on gamma index passing rate for VMAT and IMRT QA. J Appl Clin Med Phys. 2018; 19(2):230-48. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
21. Stedem AK, Tutty M, Chofor N, Langhans M, Kleefeld C, Schönfeld AA. Systematic evaluation of spatial resolution and gamma criteria for quality assurance with detector arrays in stereotactic radiosurgery. J Appl Clin Med Phys. 2024; 25(2):e14274. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
22. Woodall WH, Saleh NA, Mahmoud MA. Equivalences between multiple dependent state sampling, chain sampling, and control chart runs rules. Qual Eng. 2023; 35(1):142-51. DOI
23. Scouse RA. Introduction to statistical quality control. Plastics and Rubber International. 1985; 10(1):30-2.
24. Deng X, Han C, Chen S, Xie C, Yi J, Zhou Y, et al. Dosimetric benefits of intensity-modulated radiotherapy and volumetric-modulated arc therapy in the treatment of postoperative cervical cancer patients. J Appl Clin Med Phys. 2017; 18(1):25-31. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
25. Bailey DW, Nelms BE, Attwood K, Kumaraswamy L, Podgorsak MB. Statistical variability and confidence intervals for planar dose QA pass rates. Med Phys. 2011; 38(11):6053-64. DOI | PubMed
26. Yu L, Tang TLS, Cassim N, Livingstone A, Cassidy D, Kairn T, Crowe SB. Analysis of dose comparison techniques for patient-specific quality assurance in radiation therapy. J Appl Clin Med Phys. 2019; 20(11):189-98. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
27. Sanghangthum T, Suriyapee S, Kim GY, Pawlicki T. A method of setting limits for the purpose of quality assurance. Phys Med Biol. 2013; 58(19):7025-37. DOI | PubMed
28. Xu Y, Zhang K, Liu Z, Liang B, Ma X, Ren W, et al. Treatment plan prescreening for patient-specific quality assurance measurements using independent Monte Carlo dose calculations. Front Oncol. 2022; 12:1051110. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
29. Russo S, Saez J, Esposito M, Bruschi A, Ghirelli A, Pini S, et al. Incorporating plan complexity into the statistical process control of volumetric modulated arc therapy pre-treatment verifications. Med Phys. 2024; 51(6):3961-71. DOI | PubMed
30. Xiao Q, Bai S, Li G, Yang K, Bai L, Li Z, et al. Statistical process control and process capability analysis for non-normal volumetric modulated arc therapy patient-specific quality assurance processes. Med Phys. 2020; 47(10):4694-702. DOI | PubMed
31. Vysakh R, Raman RG, Niyas P, Aflah P. Statistical Process Control in Monitoring Radiotherapy Quality Assurance Program: An Institutional Experience. Iran J Med Phys. 2022; 19(3):189-98. DOI
32. Sanghangthum T, Suriyapee S, Srisatit S, Pawlicki T. Statistical process control analysis for patient-specific IMRT and VMAT QA. J Radiat Res. 2013; 54(3):546-52. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
33. Cui T, Peng JX, Jin XL, Chu XG, Zhang J, Cui LZ, Xing XF. Improving intensity-modulated radiation therapy quality assurance by adopting statistical process control. Cancer Radiother. 2022; 26(3):427-32. DOI | PubMed
34. Zhang H, Lu W, Cui H, Li Y, Yi X. Assessment of Statistical Process Control Based DVH Action Levels for Systematic Multi-Leaf Collimator Errors in Cervical Cancer RapidArc Plans. Front Oncol. 2022; 12:862635. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
35. More M, Jain V, Gurjar OP. Clinical Experience of Intensity Modulated Radiotherapy Pre-Treatment Quality Assurance for Carcinoma Head and Neck Patients with EPID and IMatriXX in Rural Center. J Biomed Phys Eng. 2020; 10(6):691-8. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
36. Szczurek L, Juszkat R, Szczurek J, Turek I, Sosnowski P. Pre-treatment 2D and 3D dosimetric verification of volumetric arc therapy. A correlation study between gamma index passing rate and clinical dose volume histogram. PLoS One 2019; 14(8):e0221086. Publisher Full Text | DOI | PubMed [ PMC Free Article ]
37. Mishra A, Pathak R, Raj Verma T, Srivastava AK, Prasad Mishra S, Mittal KK, Singh SK. Evaluation of Radiation Treatment Planning Algorithms in IMRT and VMAT: A Comparative Dosimetric Study in Lung Equivalent Heterogeneous Medium. J Biomed Phys Eng. 2023; 13(6):503-14. Publisher Full Text | DOI | PubMed [ PMC Free Article ]