Medical image registration based on PCA and M_PSNR
Jin Jin^{1} , Xinrui Bi^{2} , Miaomiao Jiang^{3} , Jincai Chang^{4} , Jianzhong Cui^{5}
^{1, 2, 4}College of Sciences, North China University of Science and Technology, Tangshan, 063210, China
^{1, 2, 4}3D Modeling and Application Innovation Laboratory, North China University of Technology, Tangshan, 063210, China
^{3, 5}Department of Neurosurgery, Tangshan Gongren Hospital, Tangshan, 063000, China
^{4, 5}Corresponding authors
Journal of Complexity in Health Sciences, Vol. 3, Issue 1, 2020, p. 62-72.
https://doi.org/10.21595/chs.2020.21266
Received 23 December 2019; received in revised form 1 February 2020; accepted 10 February 2020; published 30 June 2020
JVE Conferences
Medical image registration is not only the premise of medical image fusion, but also the focus of medical image processing. It plays an important role in clinical diagnosis and treatment planning. Aiming at the problem of mutual information value in image registration, a PCA neural network and M_PSNR based method for rigid registration of reference and floating images was proposed. In this method, the centroid of the image needs to be figured out through the image matrix, and the rotation Angle and translation of the image registration can be calculated by PCA method to obtain the optimal rotation transformation and the initial registration value. Finally, M_PSNR method improved by PSNR was used for similarity test to increase the speed and efficiency of registration and simplify the calculation process.
Keywords: medical image registration, PCA neural network, PSNR, mutual information value.
1. Introduction
In recent years, medical image registration has played an important role in clinical diagnosis and treatment planning, domestic and foreign experts and scholars in the medical image registration study phase has made many achievements, in a growing number of surgical navigation system and virtual surgery system, rapid registration is played a crucial role, at present most allocation algorithm between MRI and CT, the existing medical image registration methods mainly include two categories: the method based on gray level and based on the characteristics of the method [1].
3D Slicer software in the field of medicine has strong function of Registration, the user simply import the CT and MRI data, select the General Registration Registration (Elastix) module, can quickly achieve the same patients with different mode of image Registration, also can according to the actual situation to adjust the required parameters and templates, as shown in the Figs. 1-3.
Fig. 1. CT
Fig. 2. MR
Fig. 3. CT and MR registration
Medical image registration is a kind of typical problem in the field of image processing. Typically, patients receive multiple tomographic radiography for medical diagnosis. For example, images from different acquisition devices, different time, different shooting angles and so on, these are called multimode images, and multimode images are very useful in medical image analysis, because images in different modes provide supplementary information. Multimodal image registration is a method of recording the volume of different modes, for example, computed tomography (CT) and magnetic resonance (MR). Since the 20th century, medical imaging technology has experienced rapid development from static to dynamic [2], from shape to function, and from plane to three-dimensional. Combining various images to display their information on the same image and provide diversified images for clinical medical diagnosis will be a technology of great application value, while accurate and efficient image matching criteria is a major difficulty.
2. The problem of image registration based on mutual information values
Define a stereo coordinate system: the $X$-axis – along the line scan direction; the $Y$-axis – along the column scan direction; the $Z$-axis – from the bottom to the top. General with MRI image as a reference image ($R$), CT images are floating images ($F$). The image sizes of $R$ and $F$ are both $M\mathrm{*}N$, $L$ grayscale, and the grayscale values at the point $(x,y)$ are respectively $r(x,y)$ and $f(x,y)$. Under some transformation $\alpha $, the joint probability density of $R$ and $F$ images can be obtained through normalization processing ${P}_{FR,\alpha}(f,r)$:
where, ${h}_{\alpha}(f,r)$ refers to the joint gray histogram of $R$ and $F$ under the transformation of $\alpha $, and the probability density of $F$ is:
Similarly, the probability density of $R$ is:
The information contained in the gray distribution of the image is defined as the entropy of the image, denotation as $H$. In the process of medical image registration, when the space of $R$ and $F$ images is consistent, their corresponding gray mutual information $I\left(\alpha \right)$ reaches the maximum value, and the mutual information calculation formula is as follows:
where $H\left(R\right)$ is the entropy value of the reference image, $H\left(F\right)$ is the entropy value of the floating image, and $H(R,F)$ is the joint entropy of the two. That is:
According to the above equation, an optimal transformation $\stackrel{~}{\alpha}$ can be found under the rigid transformation to obtain the maximum mutual trust value [3]:
Since $\alpha $ is translation and rotation in the three-dimensional space of medical images:
where $\theta $ is the Angle of rotation as $\alpha $ change.
It can be seen that the process of finding the best transformation $\stackrel{~}{\alpha}$ is actually a multi-parameter optimization problem. Unconstrained method is usually used to find the optimal solution in such problems. By referring to the literature [4] and using the mutual information of images as the similarity test, it is necessary to use the multi-parameter optimization method of Powell [5] and simplex method to obtain the parameters in translation and rotation changes. Due to the fact that many local minima are involved in the calculation of mutual information, these optimization methods tend to get people into the trouble of local optimization. In the calculation of the optimal solution, the calculation amount is very large, the process is very tedious, and a lot of cycles and iterations are required, which consumes a lot of search time. More importantly, this algorithm needs an accurate and proper initial value. If it is not selected properly, it will consume more search time and may even cause the failure of medical image registration. Therefore, it is necessary to find a method with low registration time.
3. PCA neural network model
PCA is a tool for finding patterns of high-dimensional data, images, etc. In machine learning practice, PCA is often used to preprocess the input neural network data. PCA is the nature of the direction of maximum variance as the main characteristics, through the gathering, rotation and scaling of data in each orthogonal direction will be “related to”, which make them in different orthogonal directions without correlation, this algorithm can get rid of some low variance of dimension and dimension reduction effect, this operation can improve the convergence rate of the neural network and the overall effect.
In order to improve the registration efficiency and increase the probability of registration success, the first task is to find the optimal initial value needed for medical image registration, which can greatly reduce the search time. First, PCA neural network [6] was used to calculate the first principal direction of the feature image, and then the first principal direction of the feature image was matched with the center of mass to achieve image registration. The following are the basic methods and steps of PCA algorithm in image registration.
3.1. Determine the centroid coordinates of the image
Let the $(a+b)$ order matrix of two-dimensional discrete function $f(x,y)$ be:
Among them, $a$, $b=$ 0, 1, 2,….
Its zero-order moment is defined as ${M}_{\mathrm{0,0}}$, when $a=0$, $b=0$:
When the first moment and the higher moment are divided by the zero moment ${M}_{\mathrm{0,0}}$, the center of mass of the image is independent of the size of the image.
The first moment is defined as ${M}_{\mathrm{0,1}}$ or ${M}_{\mathrm{1,0}}$, when $a=0$, $b=1$ and $a=1$, $b=0$:
According to the above equation, $(\stackrel{~}{x},\stackrel{~}{y})$ is the center of mass of the object in the image.
Similarly, in medical image registration, the zero-order moment and first-order moment of images $R$ and $F$ can be calculated to obtain their centroid coordinates as $({\stackrel{~}{x}}_{R},{\stackrel{~}{y}}_{R})$ and $({\stackrel{~}{x}}_{F},{\stackrel{~}{y}}_{F})$, respectively.
3.2. Calculate rotation angle
The calculation of rotation Angle is an important step in medical image registration. As long as the image is rotated, it is obvious that the rotation Angle is equal to the rotation Angle of the first main direction. The first main direction of the reference feature image ($R$) and floating feature image ($F$) was calculated by using PCA neural network method, and then the rotation Angle, namely the Angle between the two main directions, was obtained [7].
The calculation of the first principal direction requires the principal element analysis neural network method of $Oja$, which can be used to calculate the principal direction of the correlation matrix with a given random data set.
Let’s say the input matrix:
It’s a zero mean random matrix.
If ${C}_{k}=x\left(k\right)x(k{)}^{T}$ is defined, then PCA learning algorithm of $Oja$ can be described by the following random difference equation:
where, $\eta $ is the learning rate, and $w\left(k\right)$ meets the following formula:
If the learning rate $\eta $ meets some basic conditions, $w$ will converge to the first main direction. The learning algorithm of PCA neural network model is shown in Fig. 4.
Fig. 4. PCA neural network model algorithm schematic diagram
The reference image ($R$) and the floating image ($F$) are respectively included in the corresponding matrix $r\left(k\right)$ and $f\left(k\right)$, which satisfy the following equation:
Using$$PCA principal component analysis neural network method under $Oja$, the first principal direction of $R$ image can be obtained:
And $\Vert {W}_{R}\Vert =1$, similarly, the first principal direction of image $F$ is obtained:
$A{b}_{\alpha}$ is the absolute value of the Angle.
And $\Vert {W}_{f}\Vert =1$.
The calculation formula of the Angle $\alpha $ between these two vectors is as follows:
Function $\mathrm{s}\mathrm{g}\mathrm{n}\left(x\right)$ is defined as follows:
In general, the rotation Angle is within [–30°, 30°], so the value of the rotation Angle can be obtained according to the $\mathrm{s}\mathrm{g}\mathrm{n}\left(x\right)$ function:
3.3. Registration translation calculation
According to the centroid coordinates of reference image ($R$) and floating image ($F$) determined in Section 3.1 and the rotation Angle $\alpha $ derived in Section 3.2, a method for calculating the registration translation is given.
The process of shift calculation is as follows: the center of mass $({\stackrel{~}{x}}_{R},{\stackrel{~}{y}}_{R})$ of the reference image minus the center of mass $({\stackrel{~}{x}}_{F},{\stackrel{~}{y}}_{F})$ of the floating image:
$\mathrm{d}\mathrm{e}\mathrm{l}X$ is the distance that we shifted in the $X$ direction, $\mathrm{d}\mathrm{e}\mathrm{l}Y$ is the distance that we shifted in the $Y$ direction.
3.4. Application and necessity of neural network
Neural network has the ability to use training data sets to approximate complex human input output, which is widely used in image segmentation, and is often used in the segmentation of lung CT, brain MRI and breast cancer X-ray images. Similarly, neural network is widely used in medical image registration. Generally, a spatial transformation is sought to make the spatial position of two images consistent, and the relatively mature one is the registration of brain CT and MRI. In addition, neural network also has strong nonlinear mapping function and better ability to obtain data rules, which can be used in all aspects of CAD to find the diagnosis rules and improve the diagnosis efficiency. Combining neural network with other techniques can improve the performance of the diagnosis system.
3.5. Registration translation calculation
Translation and Rotation are important steps in the image registration process, and the Translation and Rotation operations in the Transforms module in 3D Slicer software can translate and rotate slices, taking MR as an example. The section was rotated and adjusted by Rotation. The axial midline adjustment (IS) and coronal midline adjustment (PA) were performed to make the yellow line of the cross line coincide with the midline, and the sagittal position was adjusted by Angle (LR). The comparison before and after translation and rotation is shown in Fig. 5.
Fig. 5. MR contrast diagram before and after translation rotation
3.6. The advantages of PCA neural networks
The characteristics and advantages of artificial neural networks are mainly manifested in the ability to find optimal solutions at high speed. Finding the optimal solution of a complex problem often requires a large amount of computations. By using a feedback artificial neural network designed for a certain problem and giving full play to the high-speed computing capacity of computers, the optimal solution will soon be found.
4. Image similarity test based on PSNR method
Based on the problem of mutual information value in medical image registration, it requires a large amount of calculation, a complex process and a long time, so it cannot directly reflect the difference between the reference image and the floating image as a whole. In order to evaluate the image similarity after registration more objectively, the PSNR method (image peak signal-to-noise ratio) [7] was finally adopted to test the similarity, so as to further reduce the time of image registration.
PSNR is a commonly used objective standard for evaluating images, which can directly show the gray scale difference between $R$ and $F$ on the whole. Generally, the larger the calculated PSNR is, the smaller the gray difference between $R$ and $F$ is, and the more similar the image is. On the contrary, the smaller$$PSNR is, the bigger the gray scale difference between $R$ and $F$ images is, and the bigger the difference between the two images is. The mathematical formula of is expressed as follows:
where, MSE is the mean square error between $R$ and $F$ images. Combining with Eq. (8), we can get:
Since PSNR algorithm has more advantages in terms of computational load, it is selected as the similarity test of $R$ and $F$ images, and then refers to 1 to find the best transformation of the two images, so as to maximize the PSNR value of the image after registration:
Self-registration was carried out for the $X$ direction, $Y$ direction and Angle rotation in Fig. 1, and the following PSNR change curve (Fig. 6) was obtained.
As can be seen from the Fig. 6, the PSNR curve has a break point and needs to be corrected.
Fig. 6. PSNR curve
5. Improved M_PSNR method
Theoretically, the value of $\sum _{x=1}^{M}\sum _{y=1}^{N}\left(r\right(x,y)-f(x,y){)}^{2}$ could be 0, in which case PSND$=\mathrm{\infty}$, it is like an image registration with itself; But in the actual situation $R$ and $F$ the two images on time, $\sum _{x=1}^{M}\sum _{y=1}^{N}\left(r\right(x,y)-f(x,y){)}^{2}$ is impossible to be 0.
It is necessary to modify the above Eq. (22) and add a modified value $\mathrm{\Delta}d$ to obtain the improved$$PSNR, which is defined as M_PSNR algorithm. The calculation of M_PSNR method is as follows:
The optimal transformation ${\stackrel{~}{O}}_{tf}$ of the corresponding two images becomes:
According to Eq. (24), the modified value $\mathrm{\Delta}d$ is:
Because of the preconditions:
The norm of the modified value $\mathrm{\Delta}d$ is obtained:
A large number of experiments have shown that:
As a result:
Substitute $\mathrm{\Delta}d$ into Eq. (24), and get:
The following M_PSNR curve graph (Fig. 7) was obtained by self-registration of Fig. 1.
After adding the modified value, the M_PSNR curve is smoother, and the registered image will not show the $\mathrm{\infty}$ break point. When testing the similarity degree of images, the curve of the improved M_PSNR method is relatively smooth, and there is no breakpoint in the image after registration, and the registration speed is greatly improved. Secondly, the algorithm is simple and universal with strong portability.
6. Discussions
This paper is based on the problem of mutual information value in medical image registration. For example, when calculating the maximum gray mutual information $I\left(\alpha \right)$ [9], it involves the solution of some local minima, which requires a large amount of calculation, a tedious process and a lot of time. A method based on$$PCA neural network and M_PSNR model is proposed.
When determining the centroid coordinates of the image, the zero-order moment and the first-order moment are calculated. Then PCA neural network method was used to calculate the rotation Angle of the first principal direction, and the rotation invariance of PCA was used to get the initial value, and then the Angle between the two principal directions, namely the rotation Angle, was obtained. The initial registration value obtained [10] by this method is close to the optimal, which can reduce the search time, avoid getting into the local optimal problem when searching for the optimal rotation, and obtain better registration accuracy.
In order to further reduce the time of image registration, M_PSNR method is required to conduct similarity test for the two images and find the best transformation ${O}_{tf}$ of the two images, so that the M_PSNR value of the image after registration can reach the maximum. At the same time, an improved M_PSNR method was proposed. The calculation amount was about the same, but the modified $\mathrm{\Delta}d$ was added to improve the registration speed.
Fig. 7. M_PSNR curve
7. Conclusions
In conclusion, for the registration methods used for mutual information, the image registration methods used in this paper have more advantages. The larger the image size, the higher the registration effect. PCA neural network model was used to complete the registration of its first principal direction and the centroid of the image, and then the improve PSNR d method was used to test the similarity of $R$ and $F$ images. A rigid registration method based on PCA neural network and M_PSNR was proposed, which was simple and efficient. CT and MRI image registration will provide doctors with more medical information, including diagnosis and treatment information. Accurate registration is helpful for diagnosis and treatment of craniocerebral tumor and other problems [12].
Acknowledgements
This work was supported by the National Science Foundation of China (51674121, 61702184), Tangshan Innovation Team Project (18130209B, 18130210B), the Hebei Key Laboratory of Science and Application, Research and practice project of higher education teaching reform in Hebei Province (2017GJJG111) and Ministry of education production university cooperation education project (201802305012).
References
- Zou Maoyang, Yang Hao, Pan Guanghui, Zhong Yong Research progress and challenges of deep learning in medical image registration. Journal of Biomedical Engineering, Vol. 36, Issue 4, 2019, p. 677-683. [Search CrossRef]
- Ding Min Multi-Mode Medical Image Registration Algorithm Based on Mutual Information Correlation Ratio. Nanjing University of Science and Technology, 2018. [Search CrossRef]
- Ke Xinhua, Ou Dong, Geng Bo, Zeng Fan, Huang Hao, Yu Hongfei Networked management of medical photography. China Medical Education Technology, Vol. 6, 2004, p. 391-392. [Search CrossRef]
- Liang Zhigang, Gu Junhua Improved brainstorm optimization algorithm combined with Powell algorithm for medical image registration. Computer Application, Vol. 38, Issue 9, 2018, p. 2683-2688. [Search CrossRef]
- Zhang Weiguo, Wang Guihua Study and implementation of clinical image recognition of liver ultrasound based on PCA multi-feature fusion. Computer Application and Software, Vol. 31, Issue 4, 2014, p. 239-243. [Search CrossRef]
- Huang Ximou, Wan Lei Quantitative study of cervical vertebra manipulative rotation Angle based on MR model. Journal of Taishan Medical College, Vol. 40, Issue 8, 2019, p. 581-583. [Search CrossRef]
- Tang Yi Ling, Shun Liang Jiang, Shao Ping Xu, Liu Tingyun, Li Chongxi Based on the asymmetric distortion of ocular dominance stereo image quality assessment. Acta Automatica Sinica, 2019, https://doi.org/10.16383/j.aas.c190124. [Search CrossRef]
- Pan Meisen, Tang Jingtian, Yang Xiaoli Medical image registration using PCA and PSNR. Infrared and Laser Engineering, Vol. 40, Issue 2, 2011, p. 355-364. [Search CrossRef]
- Wang Dongdong Research on Medical Image Registration Based on Mutual Information Measure and Multi-Resolution Strategy. Jiangsu University, 2017. [Search CrossRef]
- Casella A., De Falco I., Della Cioppa A., Scafuri U., Tarantino E. Exploiting multi-core and GPU hardware to speed up the registration of range images by means of differential evolution. Journal of Parallel and Distributed Computing, Vol. 133, 2019, p. 307-318. [Publisher]
- Li Kuncheng, Li Yongcheng, Zhao Fuqiang, Zhao Xin Preliminary application of CT and MRI image registration and fusion in craniocerebral tumors. Chinese Medical Imaging Technology, Vol. 9, 2001, p. 823-825. [Search CrossRef]
- Wang Y. C., Wu Z. Q., Shi J. Y., Li Z. M., Shan F., Li Z. Y., Ji J. F. Evaluation of postoperative complications registration status of gastric cancer by medical information: a single center feasibility study. Chinese Journal of Gastrointestinal Surgery, Vol. 22, Issue 8, 2019, p. 729-735. [Search CrossRef]