Introduction

To ensure patients’ safety and maintain acceptable diagnostic equipment performances, national and international guidelines defined criteria and minimum accessibility values for all devices used in diagnostic exams and provided information about the Quality Assurance (QA) procedures during periodic quality controls (QC) [ 1 - 4 ]. Periodic QC procedures are important in order to guarantee consistent image quality of radiological equipment. Then QCs have an important role because they enable a complete evaluation of system status as well as image quality and allow the identification of degradation concerning the image quality itself and sources of any possible equipment malfunction, which may reduce the ability to detect and correctly interpret abnormal findings [ 5 , 6 ].

Walker et al., support the need for these procedures. In fact, since the exposure to different types of radiation must be as low as possible [ 7 ], these QCs cannot be done directly on patients. Moreover, if the device is not properly working, patients may be subjected to several risks.

Dental radiographic assessment represents one of the most frequently undertaken radiological examinations in the European Union [ 8 ]. As a matter of fact, it provides a useful aid in the diagnosis and treatment of oral diseases [ 9 ]. In this context, several professional groups, such as NCRP 145 and AAPM, proposed QA recommendations. Initially, the QC protocols were intended for film-based radiography: in fact, NCRP Report 145 refers to digital radiography by acknowledging that “the required standards, apparatus and software for dental system do not currently exist” [ 10 ]. In 2002, the American Association of Physicists in Medicine (AAPM) task group n. 175 report, concerning quality controls of dental image equipment was published [ 11 ].

However, with the development in computer technology and the introduction of digital systems, many specialists have converted from analog (film-based radiography) to digital imaging sensors. The latter has several advantages over film radiography (e.g.: immediate image production, interactive display and the possibility to store DICOM file format) [ 12 ]. Based on the image acquisition process, digital radiographic devices can be categorized as direct and indirect systems. The first ones acquire images using solid state detectors that are connected to a computer in order to display the image after exposure. The most used solid state detectors are charge couple devices (CCD) and complementary metal oxide semiconductors active pixel sensors (CMOS-APS). CCD is composed of an array of X-ray sensitive elements or wells on a silicon chip arranged in a rectangular matrix. In the readout process, resulting current is amplified, digitized, stored and displayed as digital images [ 13 ]. CMOS-APS are similar to CCDs, except they use an active pixel technology in which the pixels are isolated from their neighbors and directly accessed individually [ 14 ]. Indirect digital systems use photostimulable phosphor (PSP) plates. These imaging plates are coated with a radiation sensitive phosphor, storing a latent image after X-ray exposure [ 15 ]. The plate is scanned using a high-speed laser scanner, and the resulting light emitted by the stimulated phosphor is digitized and converted into displayable electronic information [ 16 ]. Today, a wide variety of intraoral digital systems is available in the dental market; in each case, the optimization of image reproducibility motivates the use of acquisition protocols that use dedicated phantoms to guarantee the maintenance of consistent image quality of the diagnostic equipment over lifetime [ 17 ].

Several useful phantoms have been built using different materials, such as Polymethyl methacrylate (PMMA), water, aluminum, etc., to simulate human body. These phantoms allow the measurement of different imaging parameters. For example, Mah et al., manufactured a phantom, in aluminum and PMMA to measure dynamic range, contrast perceptibility and spatial resolution at a source-to-detector distance that was representative of the one used in the clinical environment [ 18 ].

To study any possible source of error in intraoral radiographic exams, Yakoumakis et al., developed a phantom containing a real human tooth at its center, three air steps of different lengths and a background area for contrast and optical density measurements [ 19 ].

Phantoms are also useful to make comparison tests between different receptor systems; Yoshiura et al., manufactured a phantom in aluminum to compare film-based instrumentation performances with digital ones, obtaining a similar perceptibility curve in both cases and ensuring the validity of digital instruments as an alternative to classical diagnostic systems [ 20 ]. Dashpuntsag et al., performed an imaging quality comparison between the images acquired with photostimulable phosphor plates (PSPs) and the charge-coupled devices (CCDs) ones, showing that CCDs had better performances with respect to the PSPs, moreover a high variability in Gray value-dose relation was evaluated [ 21 ]. Furthermore, Pittayapat et al., tested different portable X-ray diagnostic systems using real human teeth as phantoms [ 22 ].

In this context, we developed a novel PMMA phantom to conduct image quality control of X-ray intraoral devices, regardless the X-ray source. This novel dedicated phantom was tested on different, direct and indirect, digital X-ray intraoral devices. In addition, in order to univocally perform the measurements in real time and/or during post processing, we also developed a dedicated LabView based software procedure. In fact, LabView allows easy interaction between acquisition instrumentation and operators [ 23 ]. It realizes virtual instruments in form of graphical programs running on personal computers or work- stations, which simplify the procedure for data acquisition, instrument control and/or data post-processing [ 24 ]. LabView implements a data flow paradigm in which the code is not written, but rather drawn or represented graphically, similarly to a flowchart diagram [ 25 , 26 ].

All the images were analyzed by three raters (two experienced and one novice) and the reliability of this study was evaluated by a detailed statistical analysis, better detailed in the following section.

Material and Methods

The focus of this experimental research regards the development of a novel phantom to conduct image quality controls on intraoral equipments.

Phantom description

The novel dedicated phantom consists of a PMMA circular insert, 3.5 cm in diameter. The phantom is 1.5 cm high and a PMMA grid with 0.2 × 0.2 mm² square holes is manufactured on the top surface. In the middle of this grid, corresponding to the center of the phantom, a circular hole is present. This hole is dedicated to the measurement of High Contrast Spatial Resolution (HCSR), when a high-density object (i.e. an aluminum wire), 1.0 mm in diameter, is inserted. It is depicted in Figure 1 (a) and (b).

Figure 1. The novel dedicated phantom used for X-ray intraoral image quality controls. Visible view: Front (1 (a)) and lateral (1 (b)) view. X-Ray view Front (1 (c)) and lateral (1 (d)).

Digital imaging systems sensors

The study was conducted on five different intraoral devices, all using digital sensors. In particular, three direct digital systems (one Carestream CS 2100 and two Vatech EzRay Premium Vex S100W) and two indirect digital devices (one Orix 70 and one Kodak 2100) with PSP technology were analyzed.

Test procedure

The novel phantom is able to perform a complete QC image program of X-ray intraoral equipment and allows the evaluation of the following image parameters:

1. – Image Uniformity (IU)
2. – High Contrast Spatial Resolution (HCSR)
3. – Low Contrast Spatial Resolution (LCSR)
4. – Image Linearity (IL)
5. – Artefacts.

In order to determine all the necessary imaging parameters, the phantom needs to be exposed only twice. The two different exposures are shown in Figure 1. The first exposure (Figure 1(c)) allows the evaluation of HCSR and LCSR, IL and the possible presence of artefacts; while the second one (Figure 1(d)) allows us to calculate the IU.

LabView developed software

The method for data processing uses a software for quantitative and qualitative image measurements developed for a LabView environment. The software is compatible with nonstandard and standard image formats (DICOM, BMP, TIFF, JPEG, JPEG2000, PNG, and AIPD). In order to perform image quality measurements, the procedure begins by opening the image under investigation and, using the developed tool, quantitative information was obtained.

The developed software consists of the Front Panel, which represents the visual interface, and the block diagram, that represents the main core of the software.

Image measurements

Three raters, two medical physicists, P1 and P2, and one dentistry, D, performed the QC on the X-ray acquired images. The digital images were acquired with the manufacturers’ proprietary software and they were stored in TIFF format in order to avoid information loss. In post processing, the raters, using the developed LabView software, analyzed the stored images. The radiographic parameters set in order to image the phantom is reported in Table 1.

Image Uniformity (IU) refers to the ability of the X-ray system to produce a constant signal response throughout the acquired object, imaged homogenous [ 27 ].

The IU measurement is performed using the dedicated section of the novel phantom depicted in Figure 1 (d). Three circular regions of interest (ROI) of diameter about 10% of the linear width of the phantom are drawn: the first one at the center of the phantom image, and the others on its left and right, respectively. For each ROI, a measurement of mean values of gray number is performed. The measurements are conducted using the LabView based software purposely developed. It opens the radiographic image, previously acquired, and measures the mean value of the gray numbers inside the ROI and its standard deviation (Figure 2 (a)). At acceptance, the difference between the central Gray number (G,c) and the peripheral Gray numbers (G,p) must not exceed 20 Gray Unit (GU).

Figure 2. The Front Panel of the developed software during Image Uniformity (IU) (2 (a)) and High Contrast Spatial Resolution (HCSR) (2 (b)) measurements.

|G,c-G,p|≤20 (1)

We chose the 20 GU difference value because it corresponds to about the 10% of the whole gray scale (i.e. 256 GU).

High Contrast Spatial Resolution (HCSR) measures the system’s ability to resolve high contrast objects. It is often determined having a large signal to noise ratio [ 28 ]. The HCSR measurement is performed using the dedicated section of the novel phantom depicted in Figure 1 (c) and the developed LabView based software procedure is used. A similar procedure was developed and validated in a previous study to determine the slice thickness in Computed Tomography (CT) and Magnetic Resonance (MR) devices [ 29 ]. For the evaluation of the HCSR, the procedure uses the “Line Profile” tool, which draws a line, along which gets the pixel values and intensities, on the opened image. Figure 2 (b) shows the “Front Panel” of the developed software, which allows opening the image, previously acquired, and drawing the “Line Profile”. In the same panel, the result of the line profile appears in real time as intensity vs. pixel number plot and the obtained curve is similar to an edge function.

The HCSR is obtained using the equation 2:

$\mathrm{HCSR}=\frac{\mathrm{D.L}}{P}$ (2)

Where D is the phantom diameter in mm, L represents the distance, expressed in pixels, between the minimum and the maximum of the path of rapid change in image intensity, and P is the total pixels number calculated by the LabView software.

Low Contrast Spatial Resolution (LCSR) measures the system’s ability to resolve low contrast objects of small sizes. It measures the capability of an imaging system to distinguish objects when there is no significant noise contribution [ 27 ]. The LCSR is performed using the dedicated section of the novel phantom depicted in Figure 1 (c), consisting of a grid with 0.2 × 0.2 mm² square holes. This PMMA grid has minimal differences (if converted to gray scale) from background. Different quantitative methods have been proposed. However, the most widely accepted [ 27 ] requires an observer that subjectively detects the objects (i.e. in this study the grid holes), as distinct. The grid with 0.2 × 0.2 mm² square holes was chosen to augment the differences (in the gray scale representation) between the phantom and the background.

Image Linearity (IL) (or geometrical distortion) refers to the ability of the system to reproduce the dimensions of an object [ 30 ]. The IL measurement is performed using the dedicated section of the novel phantom depicted in Figure 1 (c) using the LabView software. The LabView software opens the previously acquired X-ray image and four diameter values are measured. The IL is defined by the equation 3:

$\mathrm{IL}(\%)=\frac{D_T-\stackrel{-}{D_M}}{D_T}$ (3)

Where D_T is the physical diameter of the novel phantom and $\stackrel{-}{D_M}$ is the mean value of the distances measured in the image, using the LabView’s ruler tool. At the acceptance, the maximum IL (%) must not differ from 5%.

Artefacts are features that appear in an image but not in the original imaged object. The identification of imaging artefacts depends on the rater’s ability, being often based on subjective criteria [ 31 ].

Statistical analysis

A further validation of the proposed methodology was the statistical analysis conducted [using the MATLAB ‘lillietest’ function, MATLAB® (MathWorks Inc., Natick, MA) ver. R2018b] on the resulting data set. A Lilliefors test was conducted on every set and each rater to ensure that the collected datasets follow a normal distribution [ 32 ].

In order to confirm the validity of the obtained results, three different statistical methods were performed: a one-way analysis of variance (ANOVA) for quantitative, normal distributed parameters, a post-hoc Tukey’s test for multiple comparison of different datasets, and an analysis with the Fleiss’ Kappa coefficient for the qualitative LCSR parameter, always using MATLAB® (MathWorks Inc., Natick, MA) ver. R2018b.

The one-way ANOVA is a statistical model, which allows comparing different and independent datasets of the same variable to assume whether they are generated by the same distribution; this is the ”null hypothesis” on which the test is based [ 33 ].

It can be seen as a special case of linear model, described by the equation 4:

$y_{\mathrm{ij}}=\stackrel{-}{y_J}+{\epsilon }_{\mathrm{ij}}$ (4)

Where y_ij is the i^th measure of the j^th group, y_j is the population mean of the j^th group, and ε_ij is the random error, independent and normally distributed, with zero mean and constant variance. Therefore, the “anova1” MATLAB function is used; if the p-value returned by the function is higher than 0.05, the test fails to reject the null hypothesis that indicates the two datasets refer to the same distribution, with a probability proportional to the p-value [ 34 ].

The post-hoc Tukey’s test is a single-step multiple comparison test used to compare all possible pairs of means in a group of different datasets and to find which ones of them statistically differ from the others [ 35 ]. The ‘multcompare’ MATLAB function is used [ 36 ].

The Fleiss’ Kappa coefficient is an inter-rater statistical coefficient, which measures the reliability of different sets of measures by different raters. It is an “extension” of Cohen’s Kappa coefficient that is valid for two raters only. It can be calculated from a matrix and Fleiss’ Kappa can be computed from this matrix to show the degree of agreement between the psychiatrists above the level of agreement expected by chance [ 37 ]. The expression for the Fleiss’ Kappa is (eq. 5):

$K=\frac{\stackrel{-}{P}-\stackrel{-}{P_e}}{1-\stackrel{-}{P_e}}$ (5)

Where $\stackrel{-}{P}$ is the relative observed agreement among raters, given by the equation 6:

$\stackrel{-}{P}=\frac{1}{N}{\sum }_i{P}_i$ (6)

With P_i rater-rater pair agreement for subject i, while P_e is the hypothetical probability of chance agreement, given by the equation 7:

$\stackrel{-}{P}={\sum }_k{P}_k^2$ (7)

With p_k proportion of judgements for class k.

In this case, a self-made MATLAB script is used to calculate the Fleiss’ Kappa coefficient (reported in supplementary material section).

Results

Quantitative and qualitative results are following. In Table 2, IU and HCSR measurements conducted on the Vatech EzRay Premium Vex S100W (1) are reported. In particular, for IU measurements, the differences of the peripheral ROIs compared to the central one are highlighted. Two different measurements were performed by each rater and, in all cases, both the differences are less than 20 GU. For each evaluated difference, the error is calculated by using the method of quadratic sum and its value is also reported. In Table 2, the results of the HCSR measurements, carried out by the three raters, using the LabView dedicated software, are shown, and an error of 5%, obtained from software development parameters, for each measurement, was considered. Each rater performed four measurements and the obtained mean value ± standard deviation (SD) is also calculated.

The Lilliefors test, performed on HCSR results, assured that all the datasets had a normal distribution (p>0.05 in all datasets, 0.19 for P1, 0.19 for P2, and 0.13 for D). The results of the ANOVA test showed no statistical difference across datasets (p-value: 0.89) (Figure 3 (a)). In Table 3, the results of the D_M measurements, conducted on the Vatech EzRay Premium Vex S100W(1) and carried out by the three raters using the LabView software, are shown. In this case, an error of 5%, obtained from software development parameters, for each measurement was considered. Each rater performed four measurements and, in Table 3, the IL value, obtained using equation. 3, is also calculated. The IL error is calculated using the standard propagation of errors and reported on Table 3.

Figure 3. Box Plot of High Contrast Spatial Resolution (HCSR) (3 (a)) and Image Linearity (IL) (3(b)) values. The central mark represents the median of the observables and the edges of the box are the 25^th and 75^th percentiles. The whiskers extend to the most extreme data points that are not considered outliers.

The Lilliefors test identified that all the datasets had a normal distribution (p>0.05 in all datasets, 0.19 for P1, 0.19 for P2, and 0.50 for D). The ANOVA test provided, for the matrix containing the three datasets, a p-value of 0.50 (p > 0.05). In Figure 3 (b), the plot of the ANOVA test is reported. In this case, it is highlighted that no statistical differences are present.

In Table 4, the results of the image QC conducted on all the considered X-ray devices and the evaluation of each rater are reported. For the IU, the maximum difference values are considered for each device. In addition, the ROI mean values strictly depend on the Window Gray Level, set on the X-ray dental acquired images.

For each device, in order to compare the dataset of every physical parameter, the ANOVA test was performed. The obtained p-values are < 0.001 (IU and HCSR) and 0.39 (IL): it is possible to notice that the values for IU and HCSR do not follow the same distribution. Then we performed a post-hoc multiple comparison using the Tukey’s test, and we found statistical differences only between indirect digital devices and direct digital devices (p < 0.001 for all the comparisons), while we did not find statistical differences between devices of the same type.

In Figure 4, the box plots of IU (4 (a)) and HCSR (4(b)) are reported. In the Figure 4 , it is possible to notice how devices with the same receptor technology are statistically correlated.

Figure 4. Tukey’s test results that put in evidence how only devices of the same type (direct or indirect systems) are statistical correlated. (a) is referred to Image Uniformity (IU), while (b) is about High Contrast Spatial Resolution (HCSR).

Statistical analysis of the LCSR qualitative parameter was conducted using the Fleiss’ Kappa coefficient. The three raters were able to distinguish the visible square grid, except for the Orix 70 X-ray images, in which one rater was not able to clearly distinguish the grid. Therefore, we used a matrix with two columns (the two classes “grid” and “not grid”) and ten rows (two rows for every image obtained with a specific device, one row for the grid region and the other for the non-grid region) in order to calculate the Fleiss’ Kappa coefficient. The Fleiss’ Kappa resulting from this matrix is 0.86: this means that there is a high grade of agreement among raters. The three raters did not detect artefacts among the acquired images; for this reason no statistical analysis was conducted on this parameter.

Discussion

In this study, a novel phantom was developed to conduct a complete image QC program on intraoral devices. The reliability of this novel phantom was evaluated on five intraoral devices different in receptor technology. The developed phantom allows us to obtain information about the following image parameters: Image Uniformity, High Contrast Spatial Resolution, Low Contrast Spatial Resolution, Image Linearity and Artifacts. In this way, a complete QC program could be carried out with only two exposures. Moreover, the novel phantom is coupled with a dedicated LabView software, which provides an innovative, accurate and automated determination of the quantitative parameters, in real time and/or during post processing, with an intuitive visual programming language. The great advantage of the LabView software is that it is adaptable and modifiable according to the operator’s needs together with providing a huge free documentation. Thus, no dedicated software and phantom, distributed by manufacturers, are necessary.

Conclusion

QCs represent an important instrument to monitor the image quality and verify the maintenance of several physical characteristics of the medical imaging devices. Due to the impossibility of performing QCs directly on patients for safety issues, a sort of replacement is needed: for this reason, phantoms are realized in order to simulate human body or parts of it, observing the standard criteria defined by the various protocols for any application. Usually commercial QC phantoms are expensive and often suitable for evaluating just a few parameters during image quality test. Therefore, several dedicated phantoms are necessary in order to perform a complete image QC program.

With this research, we tried successfully to realize a phantom able to unify all the necessary characteristics for a complete image QC program. We obtained a good accuracy of both quantitative and qualitative measurements for all of the different systems, and no statistical differences were found between the raters with the inter-rater analysis, suggesting that a single phantom can be used to evaluate the performance of the X-ray intraoral devices.

The obtained results and the related statistical analysis showed the validity of this methodology and suggested that the novel phantom could be considered as an effective tool that allows an immediate determination of the image parameters interested in the image QC program; furthermore, it is easy to be used and could represent a reproducible instrument for routinely QC programs.

References

1. Judy P F, Balter S, Bassano D, McCullough E C, Payne J T, Rothenberg L. Phantoms for performance evaluation and quality assurance of CT scanners. AAPM Report No. 1; Chicago (IL): American Association of Physicists in Medicine; 1977.
2. Och J G, Clarke G D, Sobol W T, Rosen C W, Mun S K. Acceptance testing of magnetic resonance imaging systems: report of AAPM Nuclear Magnetic Resonance Task Group No. 6. Med Phys. 1992; 19(1):217-29. DOI | PubMed
3. Amendoeira I, et al. European guidelines for quality assurance in breast cancer screening and diagnosis. Belgium: European Commission; 2013. p. 55-71.
4. Goodsitt M M, Carson P L, Witt S, Hykes D L, Kofler J M. Real-time B-mode ultrasound quality control test procedures. Report of AAPM Ultrasound Task Group No. 1. Med Phys. 1998; 25(8):1385-406. DOI | PubMed
5. Acri G, Tripepi M G, Causa F, Testagrossa B, Novario R, Vermiglio G. Slice-thickness evaluation in CT and MRI: an alternative computerised procedure. Radiol Med. 2012; 117(3):507-18. DOI | PubMed
6. Boone J M, Cody D D, Fisher J R, Frey G D, Glasser H, Gray J E, Haus A G, et al. Quality control in diagnostic radiology. AAPM Report No. 74; Madison (WI): American Association of Physicists in Medicine; 2002.
7. Walker T F, Mah P, Dove S B, McDavid W D. Digital intraoral radiographic quality assurance and control in private practice. General Dentistry. 2014; 62(5):22-9. PubMed
8. Hatziioannou K, Psarouli E, Papanastassiou E, Bousbouras P, et al. Quality control and diagnostic reference levels in intraoral dental radiographic facilities. Dentomaxillofacial Radiology. 2005; 34(5):304-7. DOI | PubMed
9. Cederberg R. Intraoral digital radiography: elements of effective imaging. Compend Contin Educ Dent. 2012; 33(9):656. PubMed
10. National Council on Radiological Protection and Measurement. Radiation protection in dentistry. NCRP Report No. 145; Bethesda (MD): NCRP; 2003.
11. Kwan A L, Ching H, Gray J E, Massoth R J, McDavid W D, Platin E, et al. Acceptance Testing and Qual- ity Control of Dental Imaging Equipment. AAPM Report No. 175; Alexandria (VA): American Association of Physicists in Medicine; 2016.
12. Udupa H, Mah P, Dove S B, McDavid W D. Evaluation of image quality parameters of representative intraoral digital radiographic systems. Oral Surg Oral Med Oral Pathol Oral Radiol. 2013; 116(6):774-83. DOI | PubMed
13. Sanderink G C, Miles D A. Intraoral detectors. CCD, CMOS, TFT, and other devices. Dental Clinics of North America. 2000; 44(2):249-55. PubMed
14. Paurazas S B, Geist J R, Pink F E, et al. Comparison of digital imaging using CCD and CMOS-APS sensors with E-spead film in the detection of periapical bony lesions. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2000; 89:356-62. DOI | PubMed
15. Kashima I. Computed radiography with photostimulable phosphor in oral and maxillofacial radiology. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 1995; 80(5):577-98. DOI | PubMed
16. Sonoda M, Takano M, Miyahara J, Kato H. Computed radiography utilizing scanning laser stimulated luminescence. Radiology. 1983; 148(3):833-8. DOI | PubMed
17. Bonanno L, Marino S, Morabito R, Barbalace G, et al. Evaluation of US and MRI techniques for carotid stenosis: a novel phantom approach. Radiol Med. 2019; 124(5):368-74. DOI | PubMed
18. Mah P, McDavid W D, Dove S B. Quality assurance phantom for digital dental imaging. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2011; 112(5):632-9. DOI | PubMed
19. Yakoumakis E N, Tierris C E, Stefanou E P, et al. Image quality assessment and radiation doses in in- traoral radiography. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2001; 91(3):362-8. DOI
20. Yoshiura K, Kawazu T, Chikui T, Tatsumi M, Tokumori K, Tanaka T, Kanda S. Assessment of image quality in dental radiography, part 1: phantom validity. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 1999; 87(1):115-22. DOI | PubMed
21. Dashpuntsag O, Yoshida M, Kasai R, et al. Numerical evaluation of image contrast for thicker and thinner objects among current intraoral digital imaging systems. Biomed Res Int. 2017; 2017:5215413. Publisher Full Text | DOI | PubMed
22. Pittayapat P, Oliveira-Santos C, Thevissen P, et al. Image quality assessment and medical physics evaluation of different portable dental X-ray units. Forensic Science International. 2010; 201(1-3):112-7. DOI | PubMed
23. Collamati L, Filippetti F, Franceschini G, Pirani S, Tassoni C. Induction machine stator fault on-line diagnosis based on LabVIEW environment. Proceedings of 8th Mediterranean Electrotechnical Conference on Industrial Applications in Power Systems, Computer Science and Telecommunications (MELECON 96); Bari, Italy: IEEE; 1996. p. 495-8.
24. National Instrument. LabVIEW TM User Manual. April 2003. Available from: https://www.ni.com/pdf/manuals/320999e.pdf.
25. Elliott C, Vijayakumar V, Zink W, Hansen R. National instruments LabVIEW: a programming environment for laboratory automation and measurement. JALA. 2007; 12(1):17-24. DOI
26. Acri G, Testagrossa B, Sestito A, Bonanno L, Vermiglio G. CT and MRI slice separation evaluation by LabView developed software. Zeitschrift für Medizinische Physik. 2018; 28(1):6-13. DOI
27. Gulliksrud K, Stokke C, Martinsen A C. How to measure CT image quality: variations in CT-numbers, uniformity and low contrast resolution for a CT quality assurance phantom. Physica Medica. 2014; 30(4):521-6. DOI | PubMed
28. Racine D, Viry A, Becce F, Schmidt S, et al. Objective comparison of high-contrast spatial resolution and low-contrast detectability for various clinical protocols on multiple CT scanners. Med Phys. 2017; 44(9):e153-63. DOI | PubMed
29. Vermiglio G, Testagrossa B, Acri G, Tripepi M G, Marino S, Bramanti P. MRI and CT Slice Thickness Evaluation by Computerized Dedicated Procedures. World Congress on Medical Physics and Biomedical Engineering; Munich, Germany: Springer; 2009 p. 924-7.DOI
30. Price R R, Axel L, Morgan T, Newman R, Perman W, Schneiders N, et al. Quality assurance methods and phantoms for magnetic resonance imaging: report of AAPM nuclear magnetic resonance Task Group No. 1. Med Phys. 1990; 17(2):287-95. DOI | PubMed
31. Erasmus L J, Hurter D, Naudé M, Kritzinger H G, Acho S. A short overview of MRI artefacts. South African Journal of Radiology. 2004; 8(2):13-7. DOI
32. MathWorks [Internet]. Lilliefors test – MATLAB lillietest – MathWorks Italia; 2019 Feb 13 [cited 2019 Mar 11]. Available from: https://it.mathworks.com/help/stats/lillietest.html.
33. Hogg R V, Ledolter J. Engineering Statistics. New York: MacMillan; 1987. p. 442.
34. MathWorks [Internet]. One-way analysis of variance – MATLAB anova1 – MathWorks Italia; 2019 Feb 15 [cited 2019 Mar 11].
35. Hochberg Y, A Tamhane. Multiple Comparison Procedures. Hoboken, NJ: John Wiley; 1987.
36. MathWorks [Internet]. Multiple comparison test - MATLAB multcompare - MathWorks Italia; 2019 Feb 13 [cited 2019 Mar 11].
37. Fleiss J L, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement. 1973; 33(3):613-9. DOI