Introduction

Although spatial domain interpolation methods have been evaluated and compared, optimal performance evaluation for an interpolation algorithm is difficult to explain. Indeed, the suitability of a spatial domain interpolator is extremely associated with some specific factors considered. Furthermore, other items may also affect the choice of an interpolation method and the precision of its outputs. The definition of modelling is to create a series of frameworks in order to imitate a real system. A model will not be the same as the reality, and the model which imitates the reality. In many barriers, the processes performed are based on a model. In this article, the aim is to analyze modelling performed for the special problem of interpolationof medical images based on the directly original and indirectly estimated pixel selection models. Interpolation based on spatial estimation can be used for various purposes such as image de-noising and restoration [ 1 ], image zooming/magnification and reconstruction [ 1 ], image security [ 2 ], image fusion [ 3 - 4 ], image compression [ 5 ] and image de-mosaicking [ 5 - 6 ].Therefore, image interpolation approaches play a pivotal role in medical image processing.They are wanted for many feasible applications of medical imageprocessing such as computed tomography (CT),and magneticresonance imaging (MRI). In these images, the reconstruction could utilize interpolation. In new X-ray imagingsystems such as digital subtraction angiography (DSA),interpolation provides assistance to enable the computer-boosted regulation of the current radiograph and the mask image. In addition, interpolation for focusing or revolving medical images after their acquirement is usually utilized in diagnosis and cure, and interpolators are combined into systems for computer-aided diagnosis (CAD), computer-assisted surgery (CAS) and image transmission, communication systems [ 7 - 8 ] and surveillance systems [ 9 ]. For example, if under interpolation, the total size of an image is reduced to1/4 of its first size, it means that each dimension of the image has been reduced to a half size.

As follows, we firstly review the basic concepts of image resizing problems and digital resolution in digital images. In the third section, we introduce our reconstruction models based on two different pixel selection approaches. In fourth and fifth sections, evaluation scheme and numerical results are investigated. The final section is allocated to the conclusion.

Material and Methods

Image Fundamentals

This study is a technical research with focus on simulation and computer-assisted image modeling to investigate image reconstruction modeling.

The digitization of images can often constitute two simultaneous processes of sampling and quantization. These two procedures are briefly described in order for the consideration of image storage requirements. Sampling process is used to digitize the spatial information in an image. It is often reached by dividing an image into a square, rectangular and newly hexagonal array of sampling points. Each of the sampling spots is referred to as a photo elementcalled a pixel.Ademonstration of a digital image is provided when consisting of a central dark area with the light climbing towards the periphery, see Figure 1. The process may be summarized as the digitization of an analog image into an N×N array of pixels. A sample value for N is 128 for a nuclear medicine scan, 512 for CT and MRI scans, 1024 for a DSA image and 2048 for a computed radiography image and digital radiograph. Note that N has weights which are round number and powers of 2, because of the binary nature of modern computing sort. Naturally, the bigger number of pixelsdenotes higher spatial resolution of the digitized image.

Figure 1. A representation of a digital image

The number of bits, b, is essential to show an image in digital format. It is explicit that the bit number is given by b = N×N×M for a square array with N-pixel and the radiometric accuracy of M-bit. The following chart shows that the number of bits is vital to represent images that are digitized. It must be known that very huge values are required to attain the resolution, which is utilized in medical imaging. Spatial accuracy is not the only factor in determining the spatial resolution of an image, because, on the other hand, the dot per inch (DPI) value is another effective factor in the spatial resolution of an image. In Figure 2, although the dimensions of both images HR and HR’ are equal, HR’ image has less DPI than image HR, so this fact reveals the resolution of an image does not depend on its sole size. If no methods of spatial estimation (interpolation) are used for resizing the image, then, min{DPI_HR'}=1/4 DPI_HR, but the use of appropriate estimator (interpolator) causes 1/4 DPI_HR<DPI_HR'<DPI_HR. In the example shown in Figure 2, the rate of compression of data is 0.75, as Eq. (1), that is, 0.75% of the primary data has been omitted (we have assumed that any type of image coding has not been used except the resizing-based approach). Figure 3 shows the importance of image interpolation that prevents the blocking issue of the simple pixel replication (For more detail, see Table 1).

Compression Rate(C.R.)=1-(b'/b) (1)

If b'= 1/4b→C.R.=0.75

Figure 2. A workflow of compression process through the interpolation methods; HR, LR and HR’ denote high resolution, low resolution and reconstructed images, respectively.

Figure 3. Images magnified based on repetition and interpolation of pixels.

For a primary HR image with 700dpi resolution (dpi is the unit of DPI factor) which has the dimension of 512×512 (pixels) and the depth of 8-bit, the primary volume is 216.32 (bits), which in the end, the total volume of HR’ will be 216.32/4 bits, but with a DPI it is less than 300 dpi.

Image Reconstruction

In compression of the information, changing compressed information into an uncompressed state is called decompression. To do so, we use the methods that are called reconstruction methods. There are various methods of reconstruction used; this way, re-sampling is one of the most useful tools. Re-sampling can be the sum of two operations up-sampling for resizing and estimating methods like spatial domain interpolators. In many aspects of image processing, the spatial domain estimation means the same as interpolation method. The most important geometrical transform in order to do resizing in digital images is an affine linear transform as Eq. (2), where the output is consequently obtained by Eq. (3). According to Eq. (3), (X,Y) are the primary coordinates of pixels, (X',Y') and are the secondary coordinates of the pixels in resized (up-sampled) image. It is clear that after resizing, there is a possibility of producing new pixels, which should be estimated by an estimation method; the estimation is done based on the existing values of adjacent pixels.

$\left(\begin{array}{c}{X}_{\mathrm{HŔ}}\\ Y_{\mathrm{HŔ}}\\ Z_{\mathrm{HŔ}}\end{array}\right)=\left(\begin{array}{ccc}{c}_x& 0& 0\\ 0& c_y& 0\\ 0& 0& 1\end{array}\right)\left(\begin{array}{c}{X}_{\mathrm{LR}}\\ Y_{\mathrm{LR}}\\ Z_{\mathrm{LR}}\end{array}\right)$ (2)

$X_{\mathrm{HŔ}}=c_x{X}_{\mathrm{LR}}$

$Y_{\mathrm{HŔ}}=c_y{Y}_{\mathrm{LR}}$ (3)

Figure 4 shows the way of resizing (up-sampling) for the sample image with the size of 2×2 whose dimensions have been doubled in both of length and width c_x = c_y = 2. After performing the required rescaling, the next step is to estimate values for the new pixels through interpolation methods. To specify a better estimation method, a technical assessment is required by numerical standards of quality assessment (QA) which is based on similarity evaluation of outputs, different interpolators and the source HR image. In the science of computer graphics, it can be shown that when similarity of a reconstructed image is more then another one to the source HR image, then, its DPI value will be greater than the second case equivalently [ 1 ]. There are various estimation methods based on interpolations such as classic methods, cubic convolution (BL) and bi-linear (CC) [ 1 , 10 - 12 ] used in this study. We investigate pixel selection models under these two interpolation methods. More explanations of the interpolation methods can be found in [ 1 ].

Figure 4. Up-sampling based on standard linear affine transform.

Experiment Design

In order to investigate the effects of various estimators, we have to design a test. To this end, first, we have to change the primary image (HR) into LR version through down-sampling, then, through reconstruction, it is changed into HR’ which has equal dimensions to the source HR image. In order to have a numerical quality assessment (QA), we can use some metrics such as PNSR [ 13 ] or SSIM [ 3 ] which obtain the order of similarity between HR and HR’ images. PNSR is as Eq. (4) and is used here. Although it does not match human visual system (HVS) [ 4 ] well, it has been known as an acceptable standard metric, and is one of the most well-known standards of quality assessment in digital image processing (DIP).

$\mathrm{MSE}=\left(\frac{1}{\mathrm{MN}}\sum _{\mathrm{i=1}}^M\sum _{\mathrm{j=1}}^N{(f_{\mathrm{ij}}-{\mathrm{f́}}_{\mathrm{ij}})}^2\right)$

$\mathrm{RMSE}=\sqrt{\mathrm{MSE}}={\left(\frac{1}{\mathrm{MN}}\sum _{\mathrm{i=1}}^M\sum _{\mathrm{j=1}}^N{(f_{\mathrm{ij}}-{\mathrm{f́}}_{\mathrm{ij}})}^2\right)}^{\frac{1}{2}}$

$\mathrm{PNSR}=\mathrm{20log}\frac{255}{\mathrm{RMSE}}(\mathrm{for\; 8}-\mathrm{bit\; images})$ (4)

The mapping used between two images under re-sampling and down-sampling processes can be clearly effective on outputs and some parts of modeling where error comes from these two stages; however, our objective here is not the error of this section. Thus, for all tests, an equal mapping must be used in the stage of re-sampling (a design like Figure 4) and down-sampling (a design like Figure 5). The design used in down-sampling stage is based on Eq. (5) or equivalently Figure 5 under the condition of c_x = c_y = 1/2.

$X_{\mathrm{LR}}=\frac{X_{\mathrm{HR}}+{\left(c_x\right)}^{-1}-1}{{\left(c_x\right)}^{-1}}$

$Y_{\mathrm{LR}}=\frac{Y_{\mathrm{HR}}+{\left(c_y\right)}^{-1}-1}{{\left(c_y\right)}^{-1}}$ (5)

Figure 5. Down-sampling model used.

Results

Modeling Results

Our goal is to do compression with the rate of C.R.=1-(1/16)=0.94, that is, we compress images with the dimensions of 512×512up to 128×128. To do this, we will simply use Eq. (5) under c_x =c_y = 4. The problem which should be discussed is the number of re-sampling steps. It is clear that it is possible to do the image reconstruction directly and produce the HR’ image (or HR’_direct). In this state, we use Eq. (3) under c_x =c_y = 4, but another way is to perform re-sampling in two steps, we again use Eq. (2) under c_x =c_y = 4 in order to produce HR’ (or HR’_indirect), whose dimensions are 256×256 in the first step, and 512×512 in the second step. The reason of using multi-step state is that in many scientific articles dealing with interpolation methods, the number of steps is not usually investigated most of the time, this work is done in only one-step, whereas many new methods for spatial domain estimation are not capable to do the reconstruction in a step like LMMSE [ 3 - 4 ]. Thus, there is the potential to change such algorithms to the direct mode of estimation (i.e. one-step); if the direct approach in an application for a wide range of scalable (free-size) interpolators has better performance. For instance, LMMSE method is one, which will do the mentioned problem in two steps. Therefore, it is evident that, in a case of comparing this method with other methods, it is desired to change other methods such as LMMSE to produce another version, which is capable of estimating in just one-step (or directly). Basically, two interpolation methods should be compared when all conditions of modeling are the same as each other. It is therefore clear that the first step is the equality of the dataset, and the metrics of QA that are the general requirements of simulation; nonetheless, these two are not all the conditions, the implementation of models is also very important. In Table 2, the results of PNSR metric for test images including benchmarks such as Lena, Baboon, etc. are seen [ 14 ]; where all images have been tested in terms of four models based on original/direct and estimated/indirect pixel selection models and also two interpolators of BL and CC (they are therefore named Direct BL, Indirect BL, Direct CC, and Indirect CC). In Table 3, PNSRs of all models for 8 medical test images (Figure 6) are shown.

Figure 6. Medical test images contain various types of images, e.g. MRI, Ultrasound, X-ray images.

The results clearly show that despite the absolute preference of original pixel selection model in the first research [ 14 ], we cannot see this preference in medical dataset in which the results of BL interpolator for both tested models (original and estimated pixel selection models) are nearly the same as each other. For CC interpolator, we only see a relatively better preference for the original pixel selection model. Figures 7 and 8 also describe the data of Table 3 for BL and CC interpolators separately (under two different pixel selection models).

Figure 7. CC-based results for direct and indirect models in medical images.

Figure 8. BL-based results for direct and indirect models in medical images.

Eq. (6) is a simple metric for computing the distance or difference between two selected models. The results of using this metric are shown in Figure 9, and Table 4 describes its details. It is clear that “d13” has the least value among all labels, because Direct BL and Indirect BL models in medical dataset nearly have the same PSNRs.

$d_{\mathrm{ij}}=\sqrt{\sum _{\mathrm{all\; images}}{\left({\mathrm{PNSR}}_j\right)}^2}$ (6)

Figure 9. It shows the distances of all models compared each other.

Figure 10 shows some processed images under the mentioned models visually. See more details in its capture. Visual comparison has usually been a main factor for quality assessment.

Figure 10. Columns “B”, “C”, “D”, and “E” show the different models of Direct BL, Direct CC, Indirect BL, and Indirect CC, respectively; and the column “A” shows the source images before any processing including down-sampling and reconstruction. Direct CC model has the best quality in practice.

Discussion

This paper gives a complementary viewpoint on findings in [ 14 ]. The results clearly demonstrate that despite the absolute preference of the original pixel selection model in the first research, we cannot see this preference in medical dataset in which the results of BL interpolator for both tested models (original and estimated pixel selection models) are approximately the same as each other and for CC interpolator, we only see a relatively better preference for the original pixel selection model. The current research reveals the fact that selection models are not a general factor in reconstruction problems, and the structure of the basic interpolators is also a main factor which affects the final results. Therefore, in designing new magnification algorithms for direct zoom of medical images, it is possible that selecting original pixels might not have a considerable impact on the algorithm performance. In other words, some interpolators in medical dataset can be affected by the selection models, while, some cannot. In addition, computational complexity [ 15 ] for direct and indirect models should be further discussed in terms of number of operations. For a future work, we can consider these results as a guideline for improving non-freesize interpolators to a more free-size state or generally other interpolators [ 16 - 22 ] and image processing applications such as [ 9 , 23 - 24 ].

Conclusion

The analyses of the outputs show that ignoring the manner of reconstruction models is clearly effective in numerical outputs. The current research exhibited that selection models are not a general factor in reconstruction problems, and the structure of the basic interpolators is also a main factor which affects the final results. Therefore, in designing new magnification algorithms for direct zoom of medical images, it is possible that selecting original pixels will not have a considerable impact on the algorithm performance. In other words, some interpolators in medical dataset can be affected by selection models, but some cannot.

References

1. Gonzalez R C, Woods R E. Digital image processing. Prentice Hall: Upper Saddle River, NJ ; 2012.
2. Khosravi M, Rostami H. Improving the Binary Watermark-based Data Hiding Scheme in Remote Sensing Images. SID: ICAUCAE; Tehran ; 2016.
3. Khosravi M R, Sharif-Yazd M, Moghimi M K, Keshavarz A, Rostami H, Mansouri S. MRF-based multispectral image fusion using an adaptive approach based on edge-guided interpolation. Journal of Geographic Information System. 2017; 9:114-25.
4. Statistical Image Fusion for HR Band Colorization in Landsat Sensors. 20th Annual Conference of Computer Society of Iran (CSICC2015). CSI: Mashhad ; 2015.
5. Unser M, Thevenaz P, Yaroslavsky L. Convolution-based interpolation for fast, high-quality rotation of images. IEEE Trans Image Process. 1995; 4:1371-81. DOI | PubMed
6. Maeland E. On the comparison of interpolation methods. IEEE Trans Med Imaging. 1988; 7:213-7. DOI | PubMed
7. Alhihi M. Network Coding for Wireless Sensor Network Cluster over Rayleigh Fading Channel: Finite State Markov Chain. International Journal of Communications, Network and System Sciences. 2017; 10:1.
8. Alhihi M. Practical routing protocol models to improve network performance and adequacy. Journal of Computer and Communications. 2017; 5:114-24.
9. Moghimi M K, Pourghassem H. Shadow Detection Based on Combinations of Hessenberg Decomposition and Principal Component Analysis in Surveillance Applications. IETE Journal of Research. 2015; 61:269-84.
10. Keys R. Cubic convolution interpolation for digital image processing. IEEE Trans Acoust. 1981; 29:1153-60.
11. Hou H, Andrews H. Cubic splines for image interpolation and digital filtering. IEEE Trans Acoust. 1978; 26:508-17.
12. Parker J A, Kenyon R V, Troxel D E. Comparison of interpolating methods for image resampling. IEEE Trans Med Imaging. 1983; 2:31-9.
13. Khosravi M, Rostami H. A New Statistical Technique for Interpolation of Landsat Images. ICAUCAE 2016. SID: Tehran ; 2016.
14. Khosravi M R. Single- and Multi-Step Image Enlargement for Medical Image Coding. Current Signal Transduction Therapy. 2019; 14:1-2.
15. Energy efficient spherical divisions for VBF-based routing in dense UWSNs. 2nd International Conference on Knowledge-Based Engineering and Innovation (KBEI). IEEE: Tehran ; 2015.
16. Unser M, Aldroubi A, Eden M. Fast B-spline transforms for continuous image representation and interpolation. IEEE Transactions on pattern analysis and machine intelligence. 1991; 13:277-85.
17. Dodgson N A. Quadratic interpolation for image resampling. IEEE Trans Image Process. 1997; 6:1322-6. DOI | PubMed
18. Schultz R R, Stevenson R L. A Bayesian approach to image expansion for improved definition. IEEE Transactions on Image Processing. 1994; 3:233-42.
19. Thurnhofer S, Mitra SK. Edge-enhanced image zooming. Optical Engineering. 1996; 35:1862-71.
20. Grevera G J, Udupa J K, Miki Y. A task-specific evaluation of three-dimensional image interpolation techniques. IEEE Trans Med Imaging. 1999; 18:137-43. DOI | PubMed
21. Appledorn C R. A new approach to the interpolation of sampled data. IEEE Trans Med Imaging. 1996; 15:369-76. DOI | PubMed
22. Lee C, Eden M, Unser M. High-quality image resizing using oblique projection operators. IEEE Trans Image Process. 1998; 7:679-92. DOI | PubMed
23. A novel fake color scheme based on depth protection for MR passive/optical sensors. 2nd International Conference on Knowledge-Based Engineering and Innovation (KBEI). IEEE: Tehran ; 2015.
24. Khosravi M R, Rostami H, Samadi S. Enhancing the Binary Watermark-Based Data Hiding Scheme Using an Interpolation-Based Approach for Optical Remote Sensing Images. International Journal of Agricultural and Environmental Information Systems (IJAEIS). 2018; 9:53-71.