Noise subspaces subtraction in SVD based on the difference of variance values

Xiaoming Zhang1 , Jian Tang2 , Meijun Zhang3 , Qunce Ji4

1, 2, 3, 4Engineering Institute of Engineering Corps, PLA University of Science and Technology, Nanjing 210007, China

2Corresponding author

Journal of Vibroengineering, Vol. 18, Issue 7, 2016, p. 4852-4861. https://doi.org/10.21595/jve.2016.16745
Received 17 December 2015; received in revised form 22 April 2016; accepted 8 August 2016; published 15 November 2016

Copyright © 2016 JVE International Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Creative Commons License
Table of Contents Download PDF References
Cite this article
Views 902
Reads 594
Downloads 1070
Abstract.

As a matrix decomposition method, Singular Value Decomposition (SVD) is introduced to signal processing such as denoising. Firstly, a polluted signal is constructed in Hankel matrix form, and then through SVD the Hankel matrix is decomposed to two unitary matrices and a diagonal matrix in which a series of singular values are arranged in a descending order. These singular values are considered to be located in a series of subspaces including signal subspaces and noise subspaces. The singular values in these subspaces are different because the signal magnitudes dominate noise magnitudes. Therefore, if these two kinds of subspaces are well separated, an ideal denoised signal could be achieved by reconstruction. This paper improves the traditional SVD denoising which merely does well in processing periodic signals’ subspaces separation. The improved SVD denoising method based on variance value extends SVD denoising to aperiodic signal denoising. The denoising results by improved SVD denoising, traditional SVD denoising, wavelet thresholding and EEMD denoising are compared and the improved SVD denoising method received an excellent numerical experimental effects.

Keywords: singular value decomposition, noise reduction, subspaces separation, improved SVD denoising.

1. Introduction

Noise is an inevitable phenomenon during signal acquisition. signals are polluted and the real working status are blurred in time and frequency domains because of noise. Therefore, denoising is a vital pre-processing step in signals processing.

Low-pass filter is a traditional denoising method but sometimes this filter is limited when a signal contains a sharp edge or an impulse in a short duration because these two kinds of drastic changes are not well caught. Moreover, this denoising method works on the basis that noise covers the high frequency band but the high frequency band of practical signals could also contain the useful components. In order to overcome the shortcomings of the traditional denoising method, some other methods are proposed such as wavelet thresholding and EMD denoising [1-5]. Wavelet thresholding conducts on the assumption that the signal magnitudes dominate the noise magnitudes, therefore the wavelet coefficients of noise can be set to zero when it is smaller than a preset thresholding value. However, the fixed basis function cannot guarantee a perfect match with real signals. Empirical Mode Decomposition(EMD) is a data-driven processing method which was introduced by Huang [6] in 1998 to analysis the non-stationary and non-linear signals. Afterwards EMD was improved by Wu Z. and Huang N. E. [7] by adding a group of Gaussian noise before decomposition and they called the improved method EEMD. EEMD overcomes mode aliasing to a great extent. EEMD is also be used to denoise signals [8-13]. Through EEMD a signal is decomposed to a series of Intrinsic Mode Functions (IMFs) and some of the IMFs belong to useful signal while some other IMFs belong to noise. The first kind of IMFs can be picked to reconstruct denoised signal, but some other problems such as end effect will affect the denoising performance.

Singular Value Decomposition (SVD) [14] is another data-driven signal processing method. When SVD is applied to denoising, firstly A one-dimensional signal can be rearranged to a signal matrix in Hankel matrix form. and then the signal matrix is decomposed to two unitary matrices and a diagonal matrix in which singular values are arranged in a descending order. Each dimension of the diagonal matrix is viewed as a subspace with a singular value in it. Considering that signal magnitude largely exceeds noise magnitude, subspaces with big singular values belong to signal subspaces while the other subspaces belong to noise subspaces. Therefore, a breaking point in singular values will exist. The traditional SVD denoising to determine the break point will be briefly explained in Part 3 [15]. This method is limited to deal with some periodic signal with a few impulses. Sometimes the breaking point is unobvious and different choices of different singular values will result in largely different reconstructed signals and a proper choice will result in a favorable reconstructed signal. However, little attention is put to the choice of proper singular values, researchers focus on the combination of the traditional SVD denoising and other denoising methods such as Savitzky-Golayz (SG) filter denoising and EMD denoising [16-20]. In this paper, an improved SVD denoising method based on the difference of variance values has an excellent performance by comparing with the traditional SVD denoising and other mainstream denoising methods such as wavelet thresholding and EEMD denoising.

2. Matrix construction and SVD algorithm

A polluted signal xn is a one-dimensional vector and its Hankel matrix Hn is constructed as Eq. (1):

(1)
H n = x n ( 1 ) x n ( 2 ) x n ( n ) x n ( 2 ) x n ( 3 ) x n ( n + 1 ) x n ( N - n + 1 ) x n ( N - n + 2 ) x n ( N ) .

In Eq. (1), N is the length of xn and 1<n<N.

There are many choices to form the Hankel matrices if xn has many data points. In order to make the Hankel matrix closer to a square matrix. The row number m and column number n are chosen in Eq. (2) and Eq. (3):

(2)
m = N 2 ,                       N     i s     e v e n , N + 1 2 ,       N     i s     o d d ,
(3)
n = N + 1 - m .

It is supposed that the signal xs is polluted by an additive white Gaussian noise wn:

(4)
x n = x s + w n .

In Eq. (5) the Hn, Hs and Hwn respectively denote the Hankel matrix of xn, xs and wn:

(5)
H n = H s + H w n .

The SVD can be conducted because the Hankel matrices are real. The form of the decomposition is shown as Eq. (6):

(6)
H n = U S V T .

In Eq. (6) Hn is a m×n matrix, U is a m×m orthogonal matrix, S is a m×n diagonal matrix and V is a n×n orthogonal matrix. U and V is respectively called left and right singular vectors. S can also be expressed as Eq. (7):

(7)
S = Σ 0 0 0 .

In Eq. (7) Σ=diag(σ1,σ2,,σr) in which σ1>σ2>>σr>0.

Considering that signal magnitude exceeds noise magnitude and σ are listed in a descending order, the singular values of signal lie in the front dimensions of Σ. Suppose the front s dimensions are signal subspaces. Eq. (6) can be transformed to Eq. (8). In Eq. (8) Σs and Σn respectively denote the singular values of the useful signal and noise:

(8)
H n = U S V T = U s U n Σ s 0 0 Σ n V s T V n T ,

Then:

(9)
H n = U s Σ s V s T + U n Σ n V n T .

Theoretically, there exists a breaking point which divides the two kinds singular values. After these values are separated. The clean signal can be achieved by reconstructing matrix Hs':

(10)
H s ' = U S Σ S V s T .

3. Improved SVD denoising approach to subtract noise subspaces

Briefly, traditional SVD denoising involved eight major steps.

Step (1). Construct xn’s Hankel matrix Hn.

Step (2). Achieve Un, Sn, Vn through SVD.

Step (3). Plot each singular value in Sn with respect to its column index.

Step (4). Find the breaking point in singular value-column plot.

Step (5). If the breaking point is unobvious in the plot, calculated difference values of the two successive singular values.

Step (6). Plot each difference value with respect to its column index and the index begins from 2.

Step (7). Find the top of the difference value-column plot and the top’s column index is the dimension where the breaking point is.

Step (8). Reconstruct denoised signal.

According to many experimental results, if more shocks and sharp edges exist in a signal, the relatively big singular values of the useful signal will distribute to a larger dimension. For example, x0 is a linear signal and x1 is a pulse signal. They are generated on the interval between 0 and 1.024 with 0.001 step length. Both of them are polluted by an addictive Gaussian noise whose mean value is zero and standard deviation is 0.25 The time-domain plots of two polluted signals are shown in Fig. 1(a) and Fig. 2(a). According to Eq. (1). and Eq. (2), two 512×513 Hankel matrices are constructed. Through SVD Two diagonal matrices S0, S1 are achieved. Two groups of singular value-column plots and difference value-column plots are respectively drawn in a same plot which is shown in Fig. 1(b) and Fig. 2(b). Apparently, the distribution of x1’s big singular values covers a larger dimension than x0’s which correspond to aforesaid conclusion. Moreover, the breaking point of singular value curve in Fig. 1(b) is obvious and this point is also confirmed by the top of the difference value curve. However, the breaking point in singular value curve in Fig. 2(b) is not that obvious but we can find it by the top of the difference value curve. Therefore, the difference value can be used to find the breaking point more precisely.

Generally, Signals achieved in practice contain much less sharp edges than x1, thus the big singular values of the useful signal cover only first few dozen dimensions of the matrix S. In this paper the first 100 dimensions are chosen because these dimensions are enough to cover the breaking point. Specifically, we choose the first 1, 2,…, 99, 100 dimension of S respectively to reconstruct 100 analytical signals xa1, xa2,…, xa99, xa100.

Fig. 1. x 0 plots

x0 plots

a)x0 time-domain plot

x0 plots

a)x0 value-index plot

Fig. 2. x 1 plots

x1 plots

a)x1 time-domain plot

x1 plots

b)x1 value-index plot

If a clean signal is polluted by an additive white Gaussian noise, the dispersion in time domain of the signal is much larger than the clean signal. Statistically, variance describes the dispersion of a series of points which is used is this paper. Variance is calculated by Eq. (11):

(11)
s 2 = i = 1 N ( x n ( i ) - x - n ) .

Through Eq. (11) 100 variances s12, s22,…, s992, s1002 of the analytical signals are calculated. Because the breaking point divides the signal subspaces and noise subspaces, the noise composition of analytical signals will drastically change which will lead to a great change of s2. Similarly, difference value is used to describe the change of s2. θi is defined as the difference value of s2. The point where s2 drastically change is the point where θi meets its max value:

(12)
θ i = s i + 1 2 - s i 2 ,       i = 1,2 , , 99 .

99 θ are calculated through Eq. (12). Find the index of the maximum of these θ and the index is the dimension where the breaking point is.

Briefly, the improved SVD denoising involves seven major steps.

Step (1). Construct xn Hankel matrix Hn.

Step (2). Achieve Un, Sn, Vn through SVD.

Step (3). Choose first 1, 2,…, 99, 100 dimension of Sn respectively to reconstruct 100 analytical signals xa1, xa2,…, xa100.

Step (4). Calculate s12, s22,…, s992, s1002 of the analytical signals.

Step (5). Calculate θ1, θ2,…, θ98, θ99.

Step (6). Find the maximum θimax of θi.

Step (7). xaimax is the final denoised signal.

4. Numerical experiments

Four signals are generated on the interval between 0 and 1.024 with 0.001 step length:

(13)
x 2 = s i n 20 π t ,
(14)
x 3 = e - 2 t s i n 20 π t .

x 4 is a chirp signal whose frequency is 10 at zero and 30 at the point of 1024.

x 5 is one period of ECG signals.

Periodic vibration is a universal vibration form. Therefore, periodic signals x2 and x3 are simulated. x2 has an invariable amplitude while x2 has an degenerative amplitude. x3 frequency changes constantly while its amplitude remains invariable. Additionally, x5 is the most commonly used aperiodic signal and also it is used in this paper to evaluated the denoising performance of the improved SVD denoising.

These four signals are all polluted by an addictive Gaussian noise whose mean value is zero and standard deviation is 0.25. Other two mainstream denoising methods wavelet thresholding and EEMD denoising are compared in this paper. the clean signals, polluted signals, denoising signals denoised by the traditional SVD denoising, wavelet thresholding, EEMD denoising and the improved SVD denoising are respectively shown in Fig. 3, Fig. 4, Fig. 5 and Fig. 6.

Fig. 3. Signals of x1

Signals of x1

a) Clean signal

Signals of x1

b) Polluted signal

Signals of x1

c) Traditional SVD denoising

Signals of x1

d) Wavelet thresholding

Signals of x1

e) EEMD denoising

Signals of x1

f) Improved SVD denoising

It can be seen from Fig. 3 and Fig. 4 that the traditional SVD denoising and the improved SVD denoising have a same denoising effect. It is also confirmed by the experimental results that both of the methods choose the first two dimensions as useful signal subspaces. Noise are largely eliminated by the traditional SVD denoising and the improved SVD denoising. However, the other two groups of curves achieved by wavelet thresholding and EEMD denoising are less close to the clean signals.

It can be seen from Fig. 5, the curves achieved by wavelet thresholding, EEMD denoising and improved SVD denoising method are close to the clean signal. The change of the frequency is well caught and the amplitude is relatively steady. In numerical experiment the improved SVD denoising choose the first 16 dimensions while the traditional SVD denoising chose the first 10 dimensions thus some components missed after reconstruction. Therefore, the curve in Fig. 5(c) is anamorphic.

Fig. 4. Signals of x2

Signals of x2

a) Clean signal

Signals of x2

b) Polluted signal

Signals of x2

c) Traditional SVD denoising

Signals of x2

d) Wavelet thresholding

Signals of x2

e) EEMD denoising

Signals of x2

f) Improved SVD denoising

Fig. 5. Signals of x3

Signals of x3

a) Clean signal

Signals of x3

b) Polluted signal

Signals of x3

c) Traditional SVD denoising

Signals of x3

d) Wavelet thresholding

Signals of x3

e) EEMD denoising

Signals of x3

f) Improved SVD denoising

From Fig. 6, the curves achieved by wavelet thresholding, EEMD denoising and improved SVD denoising are also close to the clean signal. Some important turning points and sharp edges are caught. In numerical experiment the improved SVD denoising chose the first 12 singular values while the traditional SVD denoising chose the first 6 singular values thus a lot of important components missed which results a totally different denoised curve in Fig. 6(c).

Fig. 6. Signals of x2

Signals of x2

a) Clean signal

Signals of x2

b) Polluted signal

Signals of x2

c) Traditional SVD denoising

Signals of x2

d) Wavelet thresholding

Signals of x2

e) EEMD denoising

Signals of x2

f) Improved SVD denoising

The parameters Signal Noise Ratio (SNR) and Mean Absolute Error (MAE) are widely used to evaluate the performance of denoising [8]. The final denoising performance of SNR and MAE are listed in Table 1 In which the most optimal parameters are bold.

Table 1. The denoising performance parameters

SNR
MAE
x 2
x 3
x 4
x 5
x 2
x 3
x 4
x 5
Polluted signal
7.37
6.32
6.73
–4.09
0.39
0.20
0.40
0.20
Traditional SVD denoising
55.81
72.13
6.35
3.82
0.03
0.01
0.41
0.10
Wavelet
33.98
19.75
34.42
9.40
0.10
0.10
0.10
0.10
EEMD
23.33
11.47
24.67
5.33
0.18
0.19
0.17
0.20
Improved SVD denoising method
55.81
72.13
28.57
19.76
0.03
0.01
0.13
0.06

Form Table 1, three conclusions are achieved.

I. Noise is harder to be eliminated in aperiodic signals than in periodic signals because the SNRs of aperiodic signals in generally are smaller than periodic signals while the MAEs are bigger.

II. The traditional SVD denoising and the improved SVD denoising have a better ability of processing periodic signals than EEMD denoising and wavelet thresholding. The final SNRs and MAEs of x1 and x2 are same because the same singular values are chosen.

III. From periodic signals to aperiodic signals, the regularity of the chosen signals is decreasing. The improved SVD denoising method gains an edge over other three methods especially when processing the ECG signal because SNRs and MAEs exceed other methods. Therefore, the improved SVD denoising can be judged to have a favorable ability to process aperiodic signals.

5. Hydraulic signal denoising

A hydraulic signal is used to evaluate the denoising performance of the improved SVD denoising method proposed in the paper, the signal is texted in a hydraulic complex test-bed. The test-bed and the data-acquisition device are shown in Fig. 7. In Fig. 7(b), on the operation panel there are different valves meant to realize different working status. Valve A is meant to realize the leakage status, valve B is a speed control valve, valve C is meant to realize cavitation status and valve D is meant to realize blocking status. In Fig. 7(c), the data-acquisition software acquires data through displacement sensors which can be absorbed on measured objects tightly by magnetic force.

Fig. 7. Figures of text-bed and data acquisition device

Figures of text-bed and data acquisition device

a) Overall figure of test-bed

Figures of text-bed and data acquisition device

b) Valves distribution

Figures of text-bed and data acquisition device

c) Data-acquisition device

x 6 is a signal of 1024 points subtracted form a long signal achieved in an integrated hydraulic test stand. The signal is denoised by the improved SVD denoising. In order to make the curves easily identified, only the first 100 data points are chosen to be plotted. The time-domain plot of the original signal and the denoised signal are shown in Fig. 8.

Fig. 8. Signals of x6

Signals of x6

a) Original signal

Signals of x6

b) Denoised signal

From the curve in Fig. 8, the periodicity of the denoised signal can be easily identified. Although the denoising performance of tested signals cannot be evaluated by SNR and MAE, the feature of the original signal is more obvious after denoising. It is enough to conclude that the improved SVD denoising is of practical use.

6. Conclusions

This paper improved the traditional SVD denoising which is merely good at processing periodic signals. While the improved SVD denoising based on the difference of variance value further extends SVD denoising to aperiodic signals and receives a good effect by comparing with other mainstream denoising methods. Considering that aperiodic signals are more common in actual occasions, the improved SVD denoising has some application prospect. The important step of SVD denoising is noise subspace subtraction. This step is based on the assumption that each subspace only contains useful signals or noise. Sometimes especially in the high-frequency band noise and useful signal will co-exist. Our further work will focus on the removal of the noise in each subspace.

References

  1. Donoho D. L., Johnstone I. M. Ideal spatial adaptation via wavelet shrinkage. Biometrica, Vol. 81, 1994, p. 425-455. [Publisher]
  2. Donoho D. L. De-noising by soft-thresholding. IEEE Transactions on Information Theory, Vol. 41, Issue 3, 1995, p. 613-627. [Publisher]
  3. Teolis A. Computational Signal Processing with Wavelets. Springer Science and Business Media, 2012. [Search CrossRef]
  4. Goharrizi A. Y., Sepehri N. A wavelet-based approach to internal seal damage diagnosis in hydraulic actuators. IEEE Transactions on Industrial Electronics, Vol. 57, Issue 5, 2010, p. 1755-1763. [Publisher]
  5. Vetterl M., Chang S. G., Yu B. Adaptive wavelet thresholding for image denoising and compression. IEEE Signal Processing Letters, Vol. 5, Issue 10, 1998, p. 265-267. [Search CrossRef]
  6. Huang N. E., Shen Z., Long S. R., et al. The Empirical Mode Decomposition and the Hilbert Spectrum for Noncolumnar and Non-Stationary Time Series Analysis. The Royal Society, 1998. [Search CrossRef]
  7. Wu Z., Huang N. E. Ensemble empirical mode decomposition: a noise-assisted data analysis method. Advances in Adaptive Data Analysis, Vol. 1, Issue 1, 2009, p. 1-41. [Publisher]
  8. Boudraa A. O., Cexus J. C. EMD-based signal filtering. IEEE Transactions on Instrumentation and Measurement, Vol. 56, Issue 6, 2007, p. 2196-2202. [Publisher]
  9. Boudraa A. O., Cexus J. C. EMD-based signal noise reduction. International Journal of Signal Processing, Vol. 1, Issue 1, 2004, p. 33-37. [Search CrossRef]
  10. Datig M., Schlurmann T. Performance and limitations of the Hilbert-Huang transformation (HHT) with an application to irregular water waves. Ocean Engineering, Vol. 31, Issue 14, 2004, p. 1783-1834. [Publisher]
  11. Kabir M. A., Shahnaz C. Denoising of ECG signals based on noise reduction algorithms in EMD and wavelet domains. Biomedical Signal Processing and Control, Vol. 7, Issue 5, 2012, p. 481-489. [Publisher]
  12. Rilling G., Flandrin P., Goncalves P. Empirical Mode Decomposition and Its Algorithms. NSIP-03, Grado (I), 2003. [Search CrossRef]
  13. Žvokelj M., Zupan S., Prebil I. Multivariate and multiscale monitoring of large-size low-speed bearings using ensemble empirical mode decomposition method combined with principal component analysis. Mechanical Systems and Signal Processing, Vol. 24, Issue 4, 2010, p. 1049-1067. [Publisher]
  14. Businger P. A., Golub G. H. Algorithm 358: Singular value decomposition of a complex matrix. Journal of the Association for Computing Machinery, Vol. 12, 1969, p. 564-565. [Publisher]
  15. Zhao X. Z., Ye B. Y., Chen T. J. Difference spectrum theory of singular value and its application to the fault diagnosis of headstock of lathe. Journal of Mechanical Engineering, Vol. 46, Issue 1, 2010, p. 100-108. [Publisher]
  16. Hassanpour H., Zehtabian A., Sadati S. Time domain signal enhancement based on an optimized singular vector denoising algorithm. Digital Signal Processing, Vol. 22, Issue 5, 2012, p. 786-794. [Publisher]
  17. Oropeza V., Sacchi M. Simultaneous seismic data denoising and reconstruction via multichannel singular spectrum analysis. Geophysics, Vol. 76, Issue 3, 2011, p. 25-32. [Publisher]
  18. Hanssanpour H., Mesbah M., Boashash B. Time-frequency feature extraction of newborn EEG seizure using SVD-based techniques. EURASIP Journal on Advances in Signal Processing, Vol. 16, 2004, p. 2544-2554. [Publisher]
  19. Hassanpour H. A time-frequency approach for noise reduction. Digital Signal Processing, Vol. 18, 2008, p. 728-738. [Publisher]
  20. Muruganatham B., Sanjith M. A., Krishnakumar B. Roller element bearing fault diagnosis using singular spectrum analysis. Mechanical Systems and Signal Processing, Vol. 34, 2013, p. 218-230. [Publisher]

Cited By

Sensors
Yin, Xu, Sheng, Shen
2019