A new highly anti-interference regularization method for ill-posed problems
Hai Liang Zhuang1 , Hai Bo Chen2
1, 2Department of Modern Mechanics, University of Science and Technology of China, CAS Key Laboratory of Mechanical Behavior and Design of Materials, Hefei, China
Vibroengineering PROCEDIA, Vol. 15, 2017, p. 128-133.
Received 2 November 2017; accepted 10 November 2017; published 1 December 2017
The solution of inverse problems has many applications in mathematical physics. Regularization methods can be applied to obtain the solution of ill-conditioned inverse problems by solving a family of neighboring well-posed problems. Thus, it is significant to investigate the regularization methods to increase the accuracy and efficiency of the solution of inverse problems. In this work, a new regularization filter and the related regularization method based on the singular system theory of compact operator are proposed to solve ill-posed problems. The Cauchy problem of Laplace equation of the first kind is a kind of well-known ill-posed problem. Numerical tests show that the proposed regularization method can solve the Cauchy problems more efficiently under a proper selection of regularization parameters. Numerical results also show that the proposed method is especially effective in solving ill-posed problems with big perturbations.
Keywords: ill-posed problems, regularization filter, regularization algorithm, the convergence of solution.
In engineering technologies, there are a large amount of inverse problems in mathematical physics from different fields, such as seismic exploration, image processing, load identification, and so on. In most cases, the inverse problems of mathematical physics are ill-posed, and a large amount of ill-posed problems can be transformed into the first kind Fredholm integral equations, which can be discretized into a system of linear algebraic equations, and by solving it the solution of the problem of mathematical physics can be obtained. However, the discretized algebraic equations are normally also ill-posed. So, the regularization methods for solving this kind of problems have become the focus in this research field, in which the selection of regularization parameters has gotten much attention.
The research of regularization theory and method for ill-posed problems can be traced back to 1923, when one of the world’s most famous mathematicians Hadamard conducted a study and gave a description of ill-posedness of the Cauchy problem of linear partial differential equations. In the 1940s, Tikhonov an academician of the former Soviet Union started to systematically study the theories and methods of ill-posed problems, and a widely used Tikhonov regularization method was put forward. Then a very classical monograph, Solutions of Ill-Health Problems, was published in the 1970s. Since then many researchers have devoted to the research and put forward different forms of regularization methods. This paper presents a new regularization method after a careful study of the existing regularization methods. So, let us start with the equation that we are going to solve:
Let and be Hilbert Spaces, , where is a bounded continuous linear operator [1, 2]. We discuss the typical mathematical expression of linear inverse problems consisting in the solution of linear operator equation:
Eq. (1) is usually an ill-posed problem, i.e. the solution of problem need not be unique or does not depend continuously on the right-hand side, and the second situation is discussed in most cases, which means that if the data on the right hand side have small perturbations, large deviations of the solution will appear. The data in Eq. (1) has perturbations, the expression of the distorted data can be expressed as:
where is a diagonal matrix with random elements coming from [–1, 1], is the indicator of perturbation.
The core of the article is how to solve the following equation effectively:
and establishing an effective regularization method is the key task.
This article is organized as follows: In Section 2, we propose a new regularization filter and the related new regularization method and prove its rationality. In Section 3, we give an accuracy estimate of the solution and present a method for the selection of regularization parameters for the new regularization method. In Section 4, we test the new regularization method by solving the first kind Fredholm integral equation and demonstrate the stability and validity of the new regularization method.
2. Construction of new regularization method
The existing regularization methods are mostly based on the perturbation theory and the continuation method, and typical representatives are the Tikhonov regularization method, the iterative methods, and other improved methods. In this section, we propose a new filter factor and correspondingly establish a new regularization method. In addition, we prove that the new regularization method is valid from the view of mathematics.
The Tikhonov regularization method is composed of the following filter factor:
where is a regularization parameter, is the singular value of matrix .
The solution of Eq. (3) with Tikhonov regularization method can be expressed as:
We define a new filter factor (, ) as follows:
Theorem 2.1 If (, ) is a filter factor given by Eq. (6), we have [3-5]:
1) , , ;
2) , ;
3) , , ;
4) , , .
Proof obviously, 1) and 2) are valid, so we only need to prove 3) and 4).
3) It is easy to know that and , so we have:
4) We prove this conclusion in two different situations:
According to Theorem 2.1, we are sure that Eq. (6) is a reasonable regularizing filter, and the approximate solution of Eq. (3) can be defined as:
where is defined as follows:
The corresponding formula for solving Eq. (3) is as follows:
where is a vector with the elements the correction factors.
3. Optimum selection of the regularization parameters
In order to obtain a stable and accurate approximate solution of Eq. (3), some regularization methods are usually adopted. The regularized approximate solution has the nature: when , , where the regularization parameter must be properly selected. In this way, the regularization solution can continuously depends on the right hand side data of Eq. (3). In general, if the unknown solutions of ill-posed problems are not confined to a certain range, the regularization solution may converge to the real solution very slowly. In order to get a faster convergence rate of , the approximate solutions of Eq. (3) need to be restricted to some source collections. We define a subspace of :
Thus, we can give an accurate estimate of the solution error by using the following theorem.
Theorem 3.1 Assume that the following conditions are met:
a) , where is a constant;
b) is a compact linear operator and injective one-to-one, ;
c) , the source condition is required.
Then we have:
Proof: According the triangle inequality, we have:
Using the properties of singular system of and Eqs. (3), (4) in theorem 2.1, we obtain:
So, we have the error estimate Eq. (16) and now we need to know how to get the suitable values of its regularization parameters.
For actual engineering problems, the prior information of the solution is generally unavailable. Thus, it is difficult or impossible to determine a suitable value for the regularization method, and the error to the actual measurement is unknown. In this case, the L-curve method is often used to select the proper regularization parameters [6, 7]. If the proper regularization parameters are selected by the L-curve method, it is possible to achieve an optimal asymptotic convergence rate in the iterative calculation. Based on the L-curve method [6, 7], we use the ergodic searching to find the proper regularization parameters, i.e. to ensure that the sum of absolute value of is as small as possible. In other words, we need to try as much as possible combinations of the regularization parameters (, ) to find the smallest value in all those sums of absolute value of and the corresponding combination of regularization parameters (, ) is just the suitable values for the new regularization method.
4. Numerical tests
In order to demonstrate the stability and validity of the new regularization method proposed in this article, we now test the method by solving the first kind Fredholm integral equation. The first kind Fredholm integral equations are usually ill-posed, and when the Fredholm integral equation is discretized into a linear algebraic equation, the matrix equation is also ill-posed. In this case, the direct inversion of the matrix cannot reach the real solution normally and some regularization methods need to be used. Here is an example of the first kind Fredholm integral equation:
If , it is easy to know that is the only solution of Eq. (20).
To solve the numerical solution of the first kind Fredholm integral equation, firstly, we discretize Eq. (20) into a linear algebraic equation. Let be the number of discretization to the independent variable, we name the following numerical equation:
and we can get the approximate equation as follows:
By this means, we turn Eq. (20) into a linear algebraic Eq. (21). For simulating the real problems of mathematical physics, we give some perturbations on the right-hand side of Eq. (1) and get Eq. (3). For this system, the direct solution is unstable and as the number of discretization nodes increases, the error increases. Therefore, it is necessary to apply the regularization algorithm to obtain a better solution.
Now we solve Eq. (3) by the Tikhonov regularization method and the new regularization method, respectively, and the accuracy and anti-interference performance of the two methods are compared under different interference conditions.
We define is the sum of absolute value of . When 0.01 and 60, using the ergodic searching treatment to find the relationship between and the combinations of regularization parameters (, ), the most suitable value of parameters (, ) can be found. The results are shown in Fig. 1. When 0.1 and 60, the results are shown in Fig. 2.
Fig. 1. Relationship between and (, ) with small perturbation 0.01
Fig. 2. Relationship between and (, ) with big perturbation 0.1
When 0.01 and 60, the theoretical solution of Eq. (3) and the solutions of different regularization methods are shown in Fig. 3. While 0.1 and 60, the results are shown in Fig. 4. It can be seen from Figs. 3 and 4 that the new regularization method proposed in this article presents much better results than those by conventional Tikhonov method for both small and big perturbations, which shows the validity and stability of the new method.
Fig. 3. Comparison of numerical results with small perturbation 0.01
Fig. 4. Comparison of numerical results with big perturbation 0.1
To improve the solution accuracy of ill-posed problems, in this paper we propose a new regularization method and prove its rationality from the view of mathematics. Furthermore, we demonstrate the convergence of its solution and present a selection method of the regularization parameters when using the new regularization method. Numerical tests have been made for solving the first kind Fredholm integral equation and comparing the results with the classical Tikhonov regularization method. Numerical results show that the present method can provide much better approximation of true solution, especially in the case of the ill-posed problems with big perturbation. Because of regularization methods are applied to obtain the solution of ill-conditioned inverse problems by solving a family of neighboring well-posed problems, if we want to get good numerical results, the error in practical engineering of measuring progress requires high, otherwise will enlarge the final result of the error, the new regularization method which reduces the need for measurement errors. A disadvantage of the new regularization method is the ergodic searching of its regularization parameters, which needs huge computational cost and should be further optimized in the coming research.
This work is supported by the strategic Priority Research Program of the Chinese Academy of Sciences, Grant No. XDB22040502.
- Tikhonov A. N., Arsenin Yl. V. A Solution of Ill-Posed Problems. Wiley, New York, 1977. [Search CrossRef]
- Landweber L. An iteration formula for Fredholm integral equations of the first kind. Journal of Mathematics, Vol. 73, Issue 3, 1951, p. 615-624. [Search CrossRef]
- Linjun Wang, Xun Han, Jie Liu A new regularization method and application to dynamic load identification problems. Inverse Problems in Science and Engineering, Vol. 19, Issue 6, 2011, p. 765-776. [Search CrossRef]
- Dym C. L., Lang M. A. A modified tikhonov regularization method for solving ill-posed problems – (1) construction of regularization. Chinese Quarterly Journal of Mathematics, Vol. 15, Issue 2, 2000, p. 98-101. [Search CrossRef]
- Billel Neggal, Nadjib Boussetila, Faouzia Rebbani Projected Tikhonov regularization method for Fredholm integral equations of the first kind. Journal of Inequalities and Applications, Vol. 1, 2016, p. 2016-195. [Search CrossRef]
- Hansen P. C. Analysis of discrete ill-posed problems by means of the L-curve. Siam Review, Vol. 34, Issue 4, 1992, p. 561-580. [Search CrossRef]
- Calvetti D. Tikhonov regularization and the L-curve for large discrete ill-posed problems. Journal of Computational and Applied Mathematics, Vol. 123, Issues 1-2, 2000, p. 423-446. [Search CrossRef]
Articles Citing this One
International Journal of Numerical Methods for Heat & Fluid FlowMagda Joachimiak