Behavioral models of nonlinear filters based on discrete time cellular neural networks

The nonlinear dynamic system modeling based on the input/output relationship results from solving the approximation problem. One can distinguish two large classes: polynomials and neural networks. The different types of neural networks draw attention. The discrete time feedforward cellular neural network is suggested for filtering non-Gaussian noise, as well as the example of nonlinear filters modeling to cancel the impulse noise is represented.


Introduction
The behavioral models of nonlinear dynamic systems are constructed with using the sets of input and output signals as well as taking into account the information (knowledge) about these systems.Depending on the volume of a priori assigned knowledge, three levels are distinguished, namely, "white, grey and black box" [1,2].The less volume of the information about a dynamic system is known, the darker an investigated object is getting.The "black box" level is established in the absence of any information about a system.
Let's highlight the "black box" approach.The reasons of turning to the "black box" methodology are the following [3]: -the functional complexity of dynamic systems, -the high level of the system integration, -lack of the knowledge of the system parameters and characteristics, -new high requirements to the parameters and characteristics of precision technique.Some technical tasks can be formulated as problems, which are solved within the framework of the input/output relationship.Basing on the dynamic system description, the identification, modeling and synthesis can be characterized as follows.
The identification is the process of constructing a nonlinear operator (the determination of the model parameters by solving the optimization task) using the known set of input and output signals.The test input signals can be both deterministic and random ones.
The mathematical modeling is the using of the mathematical model, obtained in the identification process, for calculating nonlinear system output signals.
The synthesis is the creation of hardware or software implemented device with desirable goal, for example, for compensation, filtration, detection and so on [4,5].
The stages of the "black box" modeling are the following: -forming the sets of input and output signals of a modeled object, -selecting the model type and its complexity, -defining the model parameters on solving the approximation task with assigned error, -validating a designed model.The relationship between the input and output signal sets is described by the equation: The operator in Eq. ( 1) approximates the output on the basis of the previous output observation and the input .According to the "black box" approach, the unique input/output relationship is established.
The property of uniqueness means that there is no situation when any input signal from the corresponding set is mapped into two or more output signals [1][2][3].

Turning to neural models
The development of systems with very large-scale integration and widely practiced computer art caused immense interest in neural networks and substantial progress in their investigation in recent years.
On synthesizing neural networks, the complicated matter of the knowledge representation in the network is being solved.A priori information and invariants must be taken into account in order to simplify the architecture and learning process of the network.This problem plays the special part since the proper network configuration facilitates its specialization that is very important in view of the following reasons [6,7]: -the neural network having specialized structure usually comprises less number of free parameters, which are to be adjusted than fully connected network does.Consequently, less amount of data is required for specialized network learning.Moreover, it takes less time, and such a network possesses much better generalization power; -specialized networks are characterized by more network capacity; -the implementation cost of specialized neural networks decreases because their size is significantly less than that of fully connected networks.
Nowadays, there is no clear solution of the problem how to construct specialized neural networks taking into account a priori information.Nevertheless, there are many forms of neural networks already developed, among which an acceptable structure (or structures) can be selected after estimation of their advantages and drawbacks and used for solving various research problems connected with maths, physics, engineering, neurobiology, psychology.In the course of developing the neural network theory, the sciences mentioned above will raise their theoretical and applied potentialities as well [6,7].

Example of nonlinear filters modeling to cancel non-Gaussian noise
The problem of non-Gaussian noise filter synthesis is often effectively solved within the framework of the "black box" approach.According to this approach, the mathematical filter model describes the relationship between the sets of input and output signals.The model parameters are determined by solving the approximation problem in the mean-square norm.
Nonlinear filters are synthesized on the class of bit-map (dot element) half-tone images at the resolution measured by 256 gray levels, i.e., image is the matrix of integers (elements of brightness, pixels) in the interval [0; 255].The pixel format is unit8.The impulse noise model is assigned as "salt and pepper" [8].
Four types of nonlinear filter are applied to cancel the impulse noise in distorted images.Every filter is the cascade connection of the median filter with the 3x3 square aperture and a nonlinear unit in the form of either a neural network or polynomial.
The first type of the filter is suggested to build as the cascade connection of the median filter [8] and the discrete-time feedforward cellular neural network with the unity gain piecewise linear saturation function.Hereinafter, this filter is referred to as the combined discrete-time cellular neural network (CDTCNN).
The dynamics of a cell in the feedforward cellular neural network is obtained by the following differential equation [9]: One can see from Table 1 and Fig. 1, the offed CDTCNN yields higher filtration precision, than the CTLPN, the CVF and the MF.It should be observed that the CDTCNN, the CTLPN and the CVF provide different accuracy at the nearly equal complexity of these filters (56 parameters of the CDTCNN and the CTLPN, 54 parameters of the CVF).
The use of the hyperbolic tangent activation functions in the CTLPN negatively affects the image quality (white color turns to gray one, as well as there is a bit ripple i.e. image loses its smoothness (Fig. 1(d)).Indeed, at an equal probability of the impulse noise (for instance, white and black dots on images) occurrence, the filtration with different gains at low and high amplitudes of signals (in case of the hyperbolic tangent) is not expedient.
In practice, the CDTCNN is more preferable in comparison with the CTLPN since its hardware implementation is simple due to using the piecewise linear saturation functions.

Conclusions
The mathematical modeling of nonlinear dynamic systems are frequently carried out based on the input/output system relationship by means of neural networks.
The combined discrete time feedforward cellular neural network is proposed for the impulse noise filtration on half-tone images.This neural network with the piecewise linear saturation functions carries out more accurate restoration of images in comparison with the combined two-layer perceptron network comprising the hyperbolic tangent activation functions, the combined Volterra filter and the median filter.
It should be emphasized that the hardware implementation of the cellular neural network is simpler in comparison with the two-layer perceptron network since the piecewise linear saturation functions used in the cellular network are simpler than the hyperbolic tangent activation functions included in the perceptron network.

Fig. 1 .
Output signals of nonlinear filters

Table 1 .
Mean-square errors of filtration