Inverse Problem with Respect to Domain and Artificial Neural Network Algorithm for the Solution

Abstract

We consider the inverse problem with respect to domain. We suggested a new approach for reducing the inverse problem for a domain to an equivalent problem in a variational setting and gave an effective solution algorithm for solving such problems. In order to solve boundary problem, the artificial neural network is used in each step of the iteration.

1. Introduction

A wide class of practical problems are reduced to the inverse problem with respect to domain. As an example, we can show problems of elasticity theory, diffusion problems, the problems arising in hydrodynamics [1–6], and so forth. The papers concerning inverse problems usually deal with inverse problems for an unknown function (coefficients and functions occurring in the boundary and initial conditions). But in our case, a domain is sought and the investigation of the considered problems is related with some strong difficulties. In order to avoid these difficulties and to investigate such problems, firstly, the considered inverse problem is reduced to the variational statement. As the obtained variational problem is a domain-dependent variational problem, the investigation of such problems is also related to some difficulties. Here, we give an effective algorithm for solving such problems.

2. Statement of the Problem and Main Result

Let 𝐷 be an 𝑟-dimensional domain, that is, 𝐷⊂𝑅^𝑟 and 𝑥=(𝑥₁,𝑥₂,…,𝑥_𝑟)∈𝐷.

Denote by 𝑆_𝐷 the boundary of the domain 𝐷,𝑆_𝐷=𝜕𝐷. Assume that the boundary 𝑆_𝐷 is in the space 𝐶². Let us consider the following inverse problem: −Δ𝑢+𝑢𝑎(𝑥()𝑥=)𝑢0=,𝑓𝑥(𝑥∈)𝑆,_𝐷𝑥∈𝐷,(2.1)𝜕𝑢(𝑥),(2.2)𝜕𝑛=0,𝑥∈𝑆^𝐷,(2.3) where the functions 𝑎 are 𝑓 and continuously differentiable functions in 𝑅^𝑟 and 𝑎(𝑥)>0. Denote by 𝐾 the set of convex domains set with a boundary from 𝐶².

Our goal is to find a pair (𝐷,𝑢)∈𝐾×𝐶²(𝐷) such that the function 𝑢=𝑢(𝑥) satisfies (2.1) and boundary conditions (2.2), (2.3) in the domain 𝐷. As it is seen, condition (2.2) is a Dirichlet condition, and condition (2.3) is a Neumann condition. For solving inverse problem (2.1)–(2.3), at first, we consider the following unknown domain variational problem:𝐽(𝐷,𝑢)=_𝐷𝐹𝑥,𝑢(𝑥)𝑢,𝑢(_𝑥𝑥()𝑥=)0𝑑,𝑥⟶𝑥∈𝑆m_𝐷in,𝐷∈𝐾,𝑢∈𝐶²(𝐷),(2.4).(2.5) We will assume that the function 𝐹(𝑥,𝑢,𝑝) is a continuously differentiable function of its own variables in 𝐷×𝑅×𝑅^𝑟.

If boundary condition is satisfied for 𝐷∈𝐾,𝑢∈𝐶²(𝐷), the pair (𝐷,𝑢) is said to be a possible pair. Denote by 𝑀 all possible pairs set. The pair (𝐷^∗,𝑢^∗)∈𝑀 is called an optimal pair if it gives a minimum to functional (2.4) in the set 𝑀.

Give the following theorem obtained in [7].

Theorem 2.1. Let the pair (𝐷^∗,𝑢^∗)∈𝑀 be an optimal pair for variational problem (2.4), (2.5). Then the function 𝑢^∗=𝑢^∗(𝑥) is a solution of the following Euler equation in the domain 𝐷^∗: 𝐹_𝑢𝑥,𝑢(𝑥),𝑢_𝑥(𝑥)−^𝑛_𝑖=1𝑑𝑑𝑥_𝑖𝐹_𝑢𝑥𝑖𝑥,𝑢(𝑥),𝑢_𝑥(𝑥)=0,𝑥∈𝐷^∗,(2.6) and moreover, in the boundary 𝑆_𝐷^∗, the condition 𝐹𝑥,𝑢^∗(𝑥),𝑢^∗𝑥(𝑥)−^𝑛_𝑖=1𝑢^∗𝑥_𝑖(𝑥)𝐹_𝑢𝑥𝑖𝑥,𝑢^∗(𝑥),𝑢^∗𝑥(𝑥)=0,𝑥∈𝑆_𝐷^∗,(2.7) is satisfied.

As it is seen, this theorem is proved for convex domains. But one can obtain the similar result for doubly connected domain 𝐷 with internal and external boundaries 𝑆₁ and 𝑆₂. For that, we must use the expansion_𝐷𝐹𝑑𝑥=_𝐷2𝐹𝑑𝑥−_𝐷1𝐹𝑑𝑥,(2.8) where 𝐷₁ and 𝐷₂ are convex domains bounded by the boundaries 𝑆₁ and 𝑆₂.

Now, take the function 𝐹(𝑥,𝑢,𝑝) in the following form 𝐹𝑥,𝑢,𝑢_𝑥=||𝑢_𝑥||²+𝑎(𝑥)𝑢²−2𝑓(𝑥)𝑢,(2.9) where||𝑢_𝑥||²=||𝑢_𝑥1||²+||𝑢_𝑥2||²+⋯+||𝑢_𝑥𝑟||².(2.10) As the functions𝑎 and 𝑓 are continuously differentiable functions in 𝑅^𝑟, the function 𝐹(𝑥,𝑢,𝑝) is a continuously differentiable function of its own variablies in𝐷×𝑅×𝑅^𝑟. Apply the theorem mentioned above to this function. It is clear that 𝐹_𝑢=2𝑎(𝑥)𝑢−2𝑓(𝑥)𝑢,𝐹_𝑢𝑥𝑖=2𝑢_𝑥𝑖.(2.11) Hence,𝑑𝑑𝑥_𝑖𝐹_𝑢𝑥𝑖=2Δ𝑢.(2.12) So, from condition (2.6), we get that the function 𝑢^∗=𝑢^∗(𝑥) satisfies (2.1) in the domain 𝐷^∗. From condition (2.7), we get the following boundary condition||𝑢^∗𝑥||²+𝑎𝑢^∗2−2𝑓(𝑥)𝑢^∗−2||𝑢^∗𝑥||²=0,𝑥∈𝑆_𝐷.(2.13) If we take into account the condition 𝑢^∗(𝑥)=0,𝑥∈𝑆_𝐷^∗, we get ||𝑢^∗𝑥||=0,𝑥∈𝑆_𝐷∗,(2.14) or𝑢^∗𝑥(𝑥)=0,𝑥∈𝑆_𝐷^∗.(2.15) From the equation of the derivative with respect to the normal 𝜕𝑢(𝑥)/𝜕𝑛, we get that function (2.3) satisfies the boundary condition (2.3) as well. Thus, we proved the following theorem.

Theorem 2.2. Let the pair (𝐷^∗,𝑢^∗)∈𝑀 be an optimal pair for problem (2.4), (2.5). Then it is a solution of problem (2.1)–(2.3) as well. This theorem shows that if instead of the inverse problem (2.1)–(2.3) we take the function 𝐹(𝑥,𝑢,𝑝) in the form (2.9), we can investigate the variational problem (2.4), (2.5). Notice that the inverse of this fact is not true in general. Though the functional 𝐽(𝐷,𝑢) is convex with respect to the functional 𝑢, this functional is not convex with respect to 𝐷 in general. Using the obtained results to solving problem (2.4), (2.5), we give the following method.

Let the system of the functions {𝜑_𝑘(𝑥)},𝑘=1,2,… form a basis in the space 𝐶²(𝐷). Then function 𝑢=𝑢(𝑥) may be expanded by this basis𝑢=^∞_𝑖=1𝛼_𝑘𝜑_𝑘(𝑥).(2.16) We take this into account in problem (2.4), (2.5) and get𝐼(𝐷,𝛼)=_𝐷Φ(x,𝛼)dx⟶min,𝐷∈𝐾,𝛼=𝛼₁,𝛼₂,𝛼₃,…,(2.17)_∞_𝑖=1𝛼_𝑘𝜑_𝑘(𝑥)=0,𝑥∈𝑆_𝐷,(2.18) whereΦ(𝑥,𝛼)=𝐹𝑥,^∞_𝑖=1𝛼_𝑘𝜑_𝑘(𝑥),^∞_𝑖=1𝛼_𝑘𝜕𝜑_𝑘(𝑥)𝜕𝑥.(2.19) In our case, as𝐹(𝑥,𝑢,𝑝) is in the form (2.9),Φ(𝑥,𝛼)=|||||^∞_𝑘=1𝛼_𝑘𝜕𝜑_𝑘𝜕𝑥|||||²+𝑎(𝑥)|||||^∞_𝑖=1𝛼_𝑘𝜑_𝑘(𝑥)|||||²−2𝑓(𝑥)^∞_𝑖=1𝛼_𝑘𝜑_𝑘(𝑥).(2.20) For solving problem (2.17), (2.18), calculate the first variation of functional (2.17). It is clear that 𝜕𝐼𝜕𝛼=_𝐷𝜕Φ(𝑥,𝛼)𝜕𝛼𝛿𝛼𝑑𝑥.(2.21) Calculate the first variation of the functional 𝐼(𝐷,𝛼) with respect to the domain 𝐷. In [5, 7], the functional of the form𝐺(𝐷)=_𝐷𝑔(𝑥)𝑑𝑥(2.22) is considered, and, for its first variation, the formula 𝛿𝐺(𝐷)=_𝑆𝐷𝑔(𝑥)𝛿𝑃_𝐷(𝑛(𝑥))𝑑𝑠(2.23) is obtained. Here, the function 𝑔(x) is a continuously differentiable function in 𝑅^𝑟,𝑛(𝑥) is an external normal to the surface 𝑆_𝐷 at the point 𝑥 and 𝑃_𝐷(𝑥) is a support function of the domain 𝐷 and is determined as follows:𝑃_𝐷(𝑥)=sup_𝑙∈𝐷(𝑙,𝑥),𝑥∈𝑅^𝑟.(2.24) We take into account this formula and get 𝛿𝐼(𝐷,𝛼)=_𝑆𝐷Φ(𝑥,𝛼)𝛿𝑃_𝐷(𝑛(𝑥))𝑑𝑠+_𝐷𝜕Φ(𝑥,𝛼)𝜕𝛼𝛿𝛼𝑑𝑥.(2.25)

It is seen from this formula that the numbers 𝛼₁,𝛼₂,𝛼₃,… are found from the system of equations _𝐷𝜕Φ(x,𝛼)𝜕𝛼𝑑𝑥=0.(2.26) In our case, as Φ(𝑥,𝛼) is in the form (2.20), the system of (2.26) will be a system of linear equations.

The set 𝐾 may satisfy some additional restrictions as well. For example, the volume 𝐾 may be the mentioned domains set, and domains set with the given area of surface. In another case, the domains set 𝐾 may be given as 𝐷₀⊂𝐷⊂𝐷₁ as well, where 𝐷₀,𝐷₁, and 𝑅^𝑟 are the given domains. In practical problems, the set 𝐾 may be given in the form of the integral restrictions_𝐷𝑔(𝑥)𝑑𝑥=𝑐(2.27) or_𝐷𝑔(𝑥)𝑑𝑥≤𝑐.(2.28) In general, assume that there is a domain 𝐺⊂𝑅^𝑟 such that for arbitrary 𝐷∈𝐾 contained in the set 𝐾,𝐷⊂𝐺. We can state this condition in a simpler form as follows.

We know that the optimal domain is contained in a certain domain 𝐺. The obtained relations (2.25), (2.26) enable to solve problem (2.17), (2.18) approximately. For that we give the following algorithm.

3. Algorithm for Numerical Solution

Based on the provided procedure, for numerical solution of the problem (2.4)-(2.5), the following methods are proposed.

Step 1. Take arbitrary domain 𝐷⁽⁰⁾∈𝐾 and the basis functions 𝜑^𝑘(0)(𝑥),𝑘=1,2,….(3.1)

Step 2. Solving the system of equations _𝐷(0)𝜕Φ(𝑥,𝛼)𝜕𝛼𝑑𝑥=0,(3.2) we find the convergence 𝛼⁽⁰⁾=(𝛼⁰¹,𝛼⁰²,𝛼⁰³,…).

Step 3. Minimizing the linear functional _𝑆𝐷(0)Φ𝑥,𝛼⁽⁰⁾𝑃_𝐷(𝑥)𝑑𝑠⟶min,𝐷∈𝐾,(3.3) we find the convex function 𝑃(𝑥).

Step 4. The intermediate domain 𝐷⁽⁰⁾ is found as a subdifferential of the function 𝑃(𝑥) at the point 𝑥=0 [8]. In other words, 𝐷⁽⁰⁾=𝜕𝑃(0)={𝑙∈𝑅^𝑛;𝑃(𝑥)≥(𝑙,𝑥),∀𝑥∈𝑅^𝑛}.(3.4)

Step 5. A new domain 𝐷⁽¹⁾ is found as follows: 𝐷⁽¹⁾=(1−𝜇)𝐷⁽⁰⁾+𝜇𝐷⁽⁰⁾,0<𝜇<1.(3.5) Here, the domain 𝜇 may be chosen in different ways [7, 9, 10].

If a new found domain 𝐷⁽¹⁾ satisfies definite exactness conditions, the iteration process is completed. On the contrary, for a new domain 𝐷⁽¹⁾, the iteration begins from the first step. The exactness condition may be given in different ways for example,||𝐼𝐷^(𝑘+1),𝛼^(𝑘+1)−𝐼𝐷^(𝑘),𝛼^(𝑘)||<𝜀.(3.6) Here, 𝜀>0 is said to be accuracy order of the method. Now, give some rules for choosing the quantity 𝜇_𝑘.(1)In general, the numbers 𝜇_𝑘 may be chosen from the following condition:𝑓_𝑘(𝜇)=𝐼1−𝜇_𝑘𝐷^(𝑘)+𝜇_𝑘𝐷^(𝑘),𝛼_𝑘⟶min,𝜇∈[0,1].(3.7)(2)The quantity 𝜇_𝑘 may be given as a sequence satisfying the following conditions: 0≤𝜇_𝑘≤1,lim_𝑘→∞𝜇_𝑘=0,^∞_𝑘=0𝜇_𝑘=∞,(3.8) for example,𝜇_𝑘=1𝑘+1,𝑘=1,2,3,….(3.9)(3)The another method is to take 𝜇_𝑘=1 and verity that the value of the functional decreases. If the value of the functional does not decrease, then the value of 𝜇_𝑘 decreases twice. In order to solve boundary problem (2.1), (2.2), we will use the artificial neural network in each step of the iteration.

4. Application of Neural Networks to Solving the Boundary Problem

Before passing to solution of differential equations by means of neural networks, notice the cause of this necessity. It is known that there are many methods and approaches for solving differential equations and related boundary value problems. But there exist such boundary value problems that application of the known methods to them does not enable to find the solution with high accuracy because of many reasons. This may be connected with complexity of geometric structure of the domain where these differential equations are given and with strong nonlinearity of the problem. One of the reasons is that there are a great plenty of partition points, and this may strongly influence on error. At the same time, the solution of differential equation may require numerous iterations.

In addition, application of a neural network to the solution of differential equations is stipulated also by two reasons. The first reason is that logical sequence scheme corresponds to the logical scheme in the solution of differential equations [11–14]. In other words, the solution scheme of differential equations enables to create the appropriate structure of neural networks. The second reason is that neural networks can accurately approximate the function [15, 16].

Let 𝐷⊂𝑅² and 𝑆=𝜕𝐷. Consider the following two-dimensional Poisson equation:𝑢Δ(𝑢𝑥)==𝑓𝑔((𝑥𝑥)),,𝑥𝑥∈∈𝐷𝑆,.(4.1) Find the solution of this problem as follows: 𝑢(𝑥)=^𝑛_𝑖=1𝑤_𝑖𝜑_𝑖(𝑥),(4.2) where the functions 𝜑_𝑖(𝑥) are usually chosen as radial-basis functions. Writing expansion (4.2) in (4.1), we get^𝑛_𝑖=1𝑤_𝑖Δ𝜑_𝑖(𝑥)=𝑓(𝑥),𝑥∈𝐷,_𝑛_𝑖=1𝑤_𝑖𝜑_𝑖(𝑥)=𝑔(𝑥),𝑥∈𝑆.(4.3) The weight coefficients 𝑤_𝑖,𝑖=1,𝑛 are found from the minimality condition of the following function:𝐽(𝑤)=_𝐷|||||_𝑛_𝑖=1𝑤_𝑖Δ𝜑_𝑖(𝑥)−𝑓(𝑥)|||||²𝑑𝑥+_𝑆|||||_𝑛_𝑖=1𝑤_𝑖𝜑_𝑖(𝑥)−𝑔(𝑥)|||||²𝑑𝑠⟶min.(4.4) But, in great majority of cases, the functions𝐽(𝑤)=^𝑀1_𝑘=1|||||_𝑛_𝑖=1𝑤_𝑖Δ𝜑_𝑖𝑥_𝑘−𝑓𝑥_𝑘|||||²+^𝑀2_𝑘=1|||||_𝑛_𝑖=1𝑤_𝑖𝜑_𝑖𝑠_𝑘−𝑔𝑠_𝑘|||||²⟶min(4.5) are considered using the uniformity of the equation in the values at discrete points. In order to find the minimum of this function, we can use the gradient method. The iteration may be constructed to find the weight coefficients 𝑤_𝑖,𝑖=1,𝑛.

Now, in this way, we will try to solve a boundary value problem stated for partial equations by means of a neural network. We will consider problem (4.1). If we write the solution of this problem by means of the Green function, we will see that the solution is linearly and continuously dependent on the right hand side of the boundary function. As it was noted, the neural networks allow to approximate the function with any accuracy [15, 16]. Replace the domain 𝐷⊂𝑅² by a regular discrete network by means of a small step ℎ. Let the value of the function 𝑓(𝑥) at the internal nodal points be 𝑓_𝑖𝑗. Let 𝐼₁ be an indices set and denote the data at the boundary point by 𝑔_𝑝𝑞, the indices set by 𝐼₂. Our goal is to construct a neural network by changing the functions 𝑓(𝑥),𝑔(𝑥). To get input and output data, take some 𝑢₁(𝑥),𝑢₂(𝑥),…,𝑢_𝑀(𝑥) functions. For these functions to be the solutions of (2.3), the function 𝑓(𝑥) should be chosen as the functions 𝑓⁽¹⁾(𝑥),𝑓⁽²⁾(𝑥),…,𝑓^(𝑀)(𝑥), respectively. For satisfying the boundary condition, the function𝑓(𝑥) must be defined as 𝑔⁽¹⁾(𝑥),𝑔⁽²⁾(𝑥),…,𝑔^(𝑀)(𝑥) in the boundary. In other words, if, in problem (2.3), (2.4), we take 𝑓(𝑥)=𝑓_𝑘(𝑥) and𝑔(𝑥)=𝑔_𝑘(𝑥), we get the appropriate solution𝑢(𝑥)=𝑢_𝑘(𝑥). So we must construct such a neural network that it could associate the 𝑀 number of inputs 𝐻_𝑘=𝑓^{(𝑘)𝑖𝑗},(𝑖,𝑗)∈𝐼₁,𝑔^{(𝑘)𝑝𝑞},(𝑝,𝑞)∈𝐼₂(4.6) to the 𝑀 number of outputs 𝑈_𝑘=𝑢^{(𝑘)𝑖𝑗},(𝑖,𝑗)∈𝐼₁.(4.7) Here, 𝑢^{(𝑘)𝑖𝑗} is the value of the chosen solution 𝑢_𝑘(𝑥) at the nodal points (𝑖,𝑗)∈𝐼₁. The constructed neural network enables to find an approximate solution of problem (2.3), (2.4) for arbitrarily given 𝑓(𝑥),𝑔(𝑥). For attaining it, we must insert the set of𝐻=𝑓_𝑖𝑗,(𝑖,𝑗)∈𝐼₁,𝑔_𝑝𝑞,(𝑝,𝑞)∈𝐼₂(4.8) as an input variable and, in this case, the set of 𝑈=𝑢_𝑖𝑗,(𝑖,𝑗)∈𝐼₁(4.9) as an output variable to the neural network. Then, the neural network will be an approximate solution of problem (4.1).

5. Model Example

Now let us consider the following example.

Consider the following problem:Δ𝑢𝑢(𝑥=)=−20,,𝑥𝑥∈∈𝐷𝑆,,(5.1) where 𝐷=𝑥=𝑥₁,𝑥₂∈𝑅²∶𝑥²¹+𝑥²²≤1.(5.2) The used neural network is nonliner “cascade feed forward—distributed time delay—backpropagation” with levenberg-marqwardt algorithm (train LM) that needs 3 layers (Figure 1).

Figure 1 

1st layer is Input layer that contains 12 neuron and tansig transfer functions. 2nd layer is Hidden layer that contains 18 neuron and tansig transfer functions. 3rd layer is Output layer that contains 3 neuron and purelin transfer functions.

Backpropagation is the generalization of the Widrow-Hoff learning rule to multiple-layer networks and nonlinear differentiable transfer functions. Input vectors and the corresponding target vectors are used to train a network until it can approximate a function, associate input vectors with specific output vectors, or classify input vectors in an appropriate way as defined.

The relationship between I/O, weight, and biases is shown as (Figure 2), include a weight connection from the input to each layer, and from each layer to the successive layers.

Figure 2 

The additional connections might improve the speed at which the network learns the desired relationship. Each layer has a weight matrix W, a bias vector b, and an output vector a. Networks with wight, biases, a sigmoid layer, and a linear output layer are capable of approximating any function with a finite number of discontinuities.

Now we can start creating network through command structure:NET=newcf (minmax(P),[12, 18, 3], {“tansig”, “tansig”, “purelin”},“trainlm”);NET=init (NET);NET.trainparam.show=50;NET.trainparam.lr=0.05;NET.trainparam.lr_inc=1.05;NET.trainparam.mc=0.9;NET.trainparam.epochs=300;NET.trainparam.goal=1𝑒−5;[NET,tr]=train (NET,P,T); Network = Neural Network object:  architecture:   numInputs: 1  numLayers: 3  biasConnect: [1; 1; 1]  inputConnect: [1; 1; 1]  layerConnect: [0 0 0; 1 0 0; 1 1 0]  outputConnect: [0 0 1]  numOutputs: 1 (read-only)  numInputDelays: 0 (read-only)  numLayerDelays: 0 (read-only)   subobject structures:   inputs: {1×1 cell} of inputs  layers: {3×1 cell} of layers  outputs: {1×3 cell} containing 1 output  biases: {3×1 cell} containing 3 biases  inputWeights: {3×1 cell} containing 3 input weights  layerWeights: {3×3 cell} containing 3 layer weights  functions:   adaptFcn: “trains”  divideFcn: “dividerand”  gradientFcn: “gdefaults”  initFcn: “initlay”  performFcn: “mse”  plotFcns: {“plotperform”,“plottrainstate”,“plotregression”}  trainFcn: “trainlm”  parameters:   adaptParam:  .passes  divideParam:  .trainRatio,  .valRatio,  .testRatio  gradientParam: (none)  initParam: (none)  performParam: (none)  trainParam:  .show,  .showWindow,  .showCommandLine,  .epochs, .time,  .goal,  .max_fail,  .mem_reduc, .min_grad,  .mu,  .mu_dec,  .mu_inc,  weight and bias values:   IW: {3×1 cell} containing 3 input weight matrices  LW: {3×3 cell} containing 3 layer weight matrices  b: {3×1 cell} containing 3 bias vectors.

After initializing and adjusting the train and weight parameter, the final created form of the artificial neural network is shown in Figure 3.

Figure 3 

After the training network with 𝑃=25 input and output data, Figure 4 shows the result. The result here isn’t reasonable, because the test set error and the validation set error do not have similar characteristics, and it does appear that any significant overfitting has occurred.

Figure 4 

Validation performance.

Figure 5 shows the net results to perform some analysis of the network response. In this case, there are four outputs, so there are four regressions.

(a)

(b)

(c)

(d)

Figure 5 

Regression performance.

Solution of the considered model example is 𝑢(𝑥)=(1/2)(1−𝑥²¹+𝑥²²).

We found an approximate solution of this problem for 𝑃=25 by using a neural network. These results are shown in Figure 6.

Figure 6 

For reaching the high accuracy, we should train the network with more data. We retrain the net with 𝑃=50 data again. The results are shown in Figures 7 and 8.

Figure 7 

Validation performance.

(a)

(b)

(c)

(d)

Figure 8 

Regression performance.

The artificial neural network’s error is reasonable, because the test set error and the validation set error have similar characteristics, and it does not appear that any significant overfitting has occurred.

We found an approximate solution for 𝑃=50 again. These results are shown in Figure 9.

Figure 9 

For improving the high accuracy of approximate solution, the network is capable to train with more data.