International Journal of Biomedical Imaging

Volume 2019, Article ID 9435163, 14 pages

https://doi.org/10.1155/2019/9435163

## An Optical Flow-Based Approach for Minimally Divergent Velocimetry Data Interpolation

^{1}School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, 85281, USA^{2}Department of Computer Science, Technical University of Munich, Munich, 80333, Germany^{3}School of Biological and Health Systems Engineering, Arizona State University, Tempe, 85281, USA^{4}School of Arts, Media and Engineering, Arizona State University, Tempe, 85281, USA

Correspondence should be addressed to Berkay Kanberoglu; ude.usa@orebnakb

Received 18 July 2018; Revised 4 December 2018; Accepted 10 December 2018; Published 3 February 2019

Academic Editor: A. K. Louis

Copyright © 2019 Berkay Kanberoglu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

Three-dimensional (3D) biomedical image sets are often acquired with in-plane pixel spacings that are far less than the out-of-plane spacings between images. The resultant anisotropy, which can be detrimental in many applications, can be decreased using image interpolation. Optical flow and/or other registration-based interpolators have proven useful in such interpolation roles in the past. When acquired images are comprised of signals that describe the flow velocity of fluids, additional information is available to guide the interpolation process. In this paper, we present an optical-flow based framework for image interpolation that also minimizes resultant divergence in the interpolated data.

#### 1. Introduction

Image interpolation is a fundamental problem encountered in many fields [1–9]. There are countless scenarios wherein images are acquired at resolutions that are suboptimal for the needs of specific applications. For example, biomedical images spanning a three-dimensional (3D) volume are often acquired with in-plane pixel spacings far less than the out-of-plane spacings between images. This can be the case with clinical images (e.g., from computed tomography (CT) and/or magnetic resonance (MR) imaging) as well as in vitro images acquired with modalities such as particle image velocimetry (PIV) [10–17]. In cases where motion estimation and registration are parts of an interpolation framework, hardware based approaches can offer solutions as well [18–24].

However, when acquired images are comprised of signals that describe the flow velocity of fluids, additional information is available to guide the interpolation process. Specifically, the flows of an incompressible fluid into and out of an interrogation volume must be equal according to conservation of mass [25]. Quantifying the deviation from zero net flow that is entering (or alternatively leaving) an interrogation volume (i.e., divergence) thus provides a means to direct interpolation in such a way as to reconstruct more physically accurate data.

Optical flow and/or other registration-based interpolators have proven useful in interpolating velocimetry data in the past [26–37]. Particle Image Velocimetry (PIV) is a technique that measures a velocity field in a fluid volume with the help of tracer particles in the fluid and specialized cameras [38, 39]. The default technique to determine the velocity field from the raw PIV data is a correlation analysis between two frames that were acquired by the cameras [40]. This technique can be extended to 3D as well. Optical flow-based approaches have been widely used in computer vision [41–44], and they have been appealing to researchers because of the flexibility of variational approaches. Regularizers can be used for different constraints in the energy functional to be minimized. In the conventional optical flow method there are two constraints, brightness and smoothness [45]. Optical flow-based methods have been promising in the area of fluid flow estimation in PIV [46–51]. For example, in [47], incompressibility of the flow is added as a constraint in the optical flow minimization problem. In [48], the vorticity transport equation, which describes the evolution of the fluid’s vorticity over time, is used in physically consistent spatio-temporal regularization to estimate fluid motion.

Divergence and curl (vorticity) have been used in estimating optical flow previously [52–55]. In [52], the smoothness constraint is decomposed into two parts, divergence and vorticity, in this way, the smoothness properties of the optical flow can be tuned. In [56], both incompressibility and divergence-free constraints are used in the ill-posed minimization problem to calculate a 3D velocity field from 3D Cine CT images. In [54], a second-order div-curl spline smoothness condition is employed in order to compute a 3D motion field. In [55], a data term based on the continuity equation of fluid mechanics [25] and a second-order div-curl regularizer are employed to calculate fluid flow.

Here we present an optical-flow based framework for image interpolation that also minimizes resultant divergence in the interpolated data. That is, the divergence constraint attempts to minimize divergence in interpolated velocimetry data, not the divergence of the optical flow field. To our knowledge, using divergence in this way as a constraint in an optical-flow framework for image interpolation has not been investigated prior to the preliminary work presented in [57]. The method is applied to PIV, computational fluid dynamics (CFD), and analytical data and results indicate that the trade-off between minimizing errors in velocity magnitude values and errors in divergence can be managed such that both are decreased below levels observed for standard truncated sinc function-based interpolators, as well as pure optical flow-based interpolators. The proposed method thus has potential to provide an improved basis for interpolating velocimetry data in applications where isotropic flow velocity volumes are desirable, but out-of-plane data (i.e., data in different images spanning a 3D volume) cannot be resolved as highly as in-plane data.

The remainder of this paper is structured as follows. In Section 2, a definition of the term optical flow will be given and a canonical optical flow method will be briefly described. This will provide a basis for the following sections as most of the work described in this paper has been built on the described method. In Section 2, an optical flow-based framework for interpolating minimally divergent velocimetry data is described. The new method uses flow velocity data to guide the interpolation toward lesser divergence in the interpolated data. In Section 3, performance of the proposed technique is presented with experiments and simulations on real and analytical data. The results and performance of the proposed method are discussed and concluded in Section 4.

#### 2. Methods

##### 2.1. Optical Flow

Optical flow is the apparent motion of objects in image sequences that results from relative motion between the objects and the imaging perspective. In one canonical optical flow paper [45], two kinds of constraints are introduced in order to estimate the optical flow: the* smoothness constraint* and the* brightness constancy constraint*. In this section, we give a brief overview of the original optical flow algorithm (Horn-Schunck method) and the modified algorithm that was used in this project.

Optical flow methods estimate the motion between two consecutive image frames that were acquired at times and . A flow vector for every pixel is calculated. The vectors represent approximations of image motion that are based in large part on local spatial derivatives. Since the flow velocity has two components, two constraints are needed to solve for it.

###### 2.1.1. The Brightness Constancy Constraint

The brightness constancy constraint assumes that the brightness of a small area in the image remains constant as the area moves from image to image. Image brightness at the point in the image at time is denoted here as . If the point moves by and in time , then according to the brightness constancy constraintThis can also be stated asIf we expand the left side of (2) with a Taylor series expansion, thenwhere the ellipsis denotes higher order terms in the expansion. After canceling from both sides of the equationWe can divide this equation by , which leads toSubstitutingthe brightness constraint can be written in a more compact form:where , , and . In this form and represent the image velocity components and represents the brightness gradients.

###### 2.1.2. The Smoothness Constraint

Fortunately, points from an object that is imaged in temporally adjacent frames usually have similar velocities, which results in a smooth velocity field. Leveraging this property, we can express a reasonable smoothness constraint by minimizing the sums of squares of the Laplacians of the velocity components and . The Laplacians are

###### 2.1.3. Minimization

Optical flow assumes constant brightness and smooth velocity over the whole image. The two constraints described above are used to formulate an energy functional to be minimized:Using variational calculus, the Euler-Lagrange equations can be determined for this problem. Those equations need to be solved for each pixel in the image. Iterative methods are suitable to solve the equations since it can be very costly to solve them simultaneously. The iterative equations that minimize (9) arewhere denotes the iteration number and and denote neighborhood averages of and . More detailed information on the method can be found in [45].

##### 2.2. Optical Flow with Divergence Constraint

###### 2.2.1. Continuity Equation

According to the continuity equation in fluid dynamics, the rate of mass entering a system is equal to the rate of the mass leaving the system [25]. The differential form of the equation iswhere is the fluid density, is time, and is the velocity vector field. In the case of incompressible flow, becomes constant and the continuity equation takes the form:This means that the divergence of the velocity field is zero in the case of incompressible flow. Figure 1 shows the change in flow velocity of a voxel.