Complexity

Volume 2018 (2018), Article ID 3618621, 11 pages

https://doi.org/10.1155/2018/3618621

## Protocomputing Architecture over a Digital Medium Aiming at Real-Time Video Processing

^{1}Graduate School of Information Science and Technology (IST), Hokkaido University, Kita 14, Nishi 9, Kita-ku, Sapporo, Hokkaido, Japan^{2}Microelectronic Systems Laboratory, Swiss Federal Institute of Technology (EPFL), 1015 Lausanne, Switzerland

Correspondence should be addressed to Aoi Tanibata

Received 26 October 2017; Accepted 9 January 2018; Published 15 February 2018

Academic Editor: Masashi Aono

Copyright © 2018 Aoi Tanibata et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

A protocomputational architecture is presented that implements the ancient reaction-diffusion model as a microelectronic hardware. A digital medium is selected for the physical mapping of the protoarchitecture as a way to benefit from reliable advanced integrated circuits fabrication technologies. The extraction of dense motion vector fields from textureless objects in a video sequence is selected as a realistic application. Real-time video processing results at 30 fps are achieved using an FPGA physical implementation of the proposed protoarchitecture.

#### 1. Introduction

Modern von Neumann architecture based processors systematically carry over extensive processing tasks governed by the consecutive execution of individual instructions that implement algorithms. The fact is widely accepted that these architectures are suitable to efficiently process a massive amount of simple algebraic operations with limited memory exchange, while their efficiency dramatically decreases at processing tasks that are natural and quick to mammals such as face and object recognition, feature, and saliency extraction. Artificial neural networks [1] and, more recently, machine learning models and algorithms have emerged in response to solving such conceptually complex problems, for example, [2].

Nevertheless, computation has existed in nature prior to the existence of any physical support to engineered and human computing, for example, mathematical structured thinking, geometrical support to calculations, and mechanical and electronic computing systems. The operation and behavior of neurons, the topology of neural networks, and more generally operations of the brain have been a source of inspiration yielding disciplines including computational neuroscience, neuromorphic computing, and engineering [3]. Protocomputing is emerging as a novel research discipline accepting two fundamental research hypotheses; namely, (a) computing is not limited to excitable tissues and mammals and is also processed by simple organisms as well as liquids and materials in the form of biochemical and physical processes and reactions often forming nonlinear dynamical systems with complex behavior and (b) the very fundamental nature of this processing may hold its properties from the origin of life and of the universe.

Still in its infancy, the protocomputing discipline is mostly addressed at the level of theoretical developments, modeling, and algorithmic studies [4]. Decision-solving methodologies and algorithms have been presented to efficiently solve complex optimization problems exploiting specific feature of physical phenomena [5]. Wet-laboratory experiments lend themselves particularly well to an experimental perspective, specifically considering the chemical and biology related approaches. The sophisticated spatiotemporal oscillatory dynamics developed by the primitive single-cell amoeba in solving complex environmental adaptation is used as an inspiration to solving NP-hard problems [6]. The usage of slime mould to constructing various sensing and computational building blocks is discussed in detail in [7]. One research direction in synthetic biology consists of ancestral sequence reconstruction to gain understanding in mechanisms of life and its origins [8]. Photonic medium is applied to experimentally solving the multiarmed bandit problem exploiting the ultrafast chaotic oscillatory dynamics of lasers [9]. Astronomy and bioastronomy also seek to understand computing of the origin of life.

From an electronic viewpoint, protocomputation may be considered from a classical device-level perspective, that is, as a discipline aiming at performing unconventional computation from benefiting of specific electrical characteristics of novel, post-CMOS devices and thus positioning itself in overlap to advanced switching devices research and material science, for example, single-electron transistors, graphene and quantum devices, and molecular electronics. In general, exploiting such devices requires their circuit-level hybrid usage with CMOS technology. Hence, classical analog MOSFET design is used to amplify excessively low current levels and create functionality that can successfully be interfaced to other circuits and to the outside world.

The majority of the demonstrated protocomputing device-level experiments may be considered as proof-of-concept aiming at understanding the operation and characteristics of the computing media. Prototypes supporting real-world experiments face scaling issues relating to dynamically controlling and adapting the computing media. In this work, we propose exploiting the stable and well-understood digital technology as the computing media. The protocomputing system is implemented as a protoarchitecture that is mapped into the digital media. The proposed protocomputing system consists of a reaction-diffusion process that is observed in fundamental chemical and biological condition developments. A cellular automata is used as the method of implementing the algorithm [10, 11], which is mapped into a digital protoarchitecture. As a merit of the proposed approach, a real application example of video processing is demonstrated in real-time. Abnormal behavior, saliency detection, and attention attraction are part of the reflex cognitive processing of mammals. Motion detection is a prerequisite to these tasks. Motion detection under conditions where the moving object has no texture is used as the application exploiting the protocomputation architecture to achieve real-time.

Section 2.1 details the problem and the selected cellular automata implementation of the reaction-diffusion algorithm. The original RD algorithm is adapted to support its flexible parallel operation as a solution to enable real-time operation over a wide range or size of the input space. Sections 2.2–2.4 present the fundamental protoarchitecture and its implementation into a digital computation medium. The control of the architecture and its real-time scheduling are presented. Section 3.1 presents implementation results and demonstrates the real-time capability of the proposed protoarchitecture over a real-world application. Finally, a methodology that enables scaling the input space (input image size) and that dictates the correct level of parallelism to reach real-time operation is presented in Section 3.2. Section 4 presents the conclusions.

#### 2. Materials and Methods

In the following, a method based on the ancestral reaction-diffusion algorithm is proposed to a real-life video example consisting of creating texture into textureless objects to the aim of enabling the real-time motion vector extraction. A fundamental condition to achieving real-time operation consists in the specific development of a protoarchitecture and its mapping into a digital medium.

Initially developed within the context of video compression techniques [12], motion vector estimation has found new application fields with the advent of modern ubiquitous consumer electronic products and smart vision sensors, with examples including target tracking [13], hand gesture user interface [14], image stabilization [15], surveillance, event analysis and automatic anomaly detection from monitoring cameras, depth map estimation, and 3D vision, which reveal a dynamic research activity over the recent years [16]. Classical motion vector extraction algorithms using block matching [17] are known to deliver accurate results in highly textured regions of images, while they perform poorly in low textured regions. The recent usage of block matching in smart machine-vision applications poses increased constraints on the necessity of a correct motion vector field extraction in real-time. The generated field of motion vectors is used in further algorithmic processing and thus must satisfy severe criteria in terms of spatial homogeneity [18]. For example, postprocessing tasks such as classification may require an accurate density of the motion vector field in order to decrease their error rate. Edges tracking and particle filters have been proposed as a solution to detect and track textureless objects, resulting in computationally complex algorithms, for example, [19], hence with limited practical usage in energy-constrained portable systems.

##### 2.1. Cellular-Automaton Generating Spatial Patterns Aiming at the Motion Vector Estimation of Textureless Objects

Reaction-diffusion (RD) algorithms have initially been introduced to model the concentration dynamics of different chemical species placed in presence of each other within a single container [20]. Diffusion processes represent a fundamental natural phenomena underlying the macroscopic effect observed as a result of numerous irregular microscopic motion of individual particles that spread out as a result of the motion of each. The particles may consist of chemicals, cell or bacteria, or even larger species (animals) [21]. Reaction describes the conversion of one involved particle species into another, as a result of the diffusion of two or more substances.

In its original expression, the RD model is defined over a continuous spatial domain. The dynamics of RD considering the activators and inhibitors over independent spatial fields is adapted following the procedure that is presented in [22]. Diffusion of activators and inhibitors on 1D space is described by the following diffusion equation: where represents the diffusion coefficient of the activators or inhibitors, the space, and the spatial concentration at time . The general solution is given by where represents the initial concentration at and represents the cycle index, further used in (4) to (6). In [22], two independent diffusion equations for activators and inhibitors were introduced, where (i) diffusion coefficient of inhibitors was set at much larger value than that of activators and (ii) the same initial concentrations were set to the two diffusion equations. Here we introduce a novel method that uses one diffusion equation only to describe the original model equations. First, diffusion of activators is performed during time with initial concentration , and the result is obtained as . Then the diffusion is further continued for additional time , and the result is described by . It should be noticed that and represent results of diffusions of activators and inhibitors, respectively, because is obtained by diffusion of during , which is equivalent to the two-diffusion-equation system under the same initial concentration with . Second, as in [22], differential concentration of activators and inhibitors is amplified by the sigmoid function and is set to the subsequent initial condition as where represents the gain of the sigmoid function and represents the offset value introduced in [22]. When , , and hence, by assuming and redefining as , (3) is simplified as By using , subsequent is obtained by and the generalized update equation is

Under specific parameters sets, the discrete version of RD algorithms generates repeatable, stable spatial patterns [23, 24] consisting of stripes or spots from an initial image and applying an iterative processing. Henceforth, this property is used to create texture in a video scene and in particular into textureless moving objects. Assuming that the interframe movement is sufficiently small, then the texture follows the movement of the textureless object, enabling the detection of the movement of texture, rather than object edges only, as a limitation of classical algorithms. An algorithm that implements the aforementioned principles has been presented in [25]. The RD process is defined in its fundamental form over a continuous spatial domain. Following the procedure described in [22], the dynamics of the RD process is adapted to support the diffusion of activators and inhibitor in independent discrete fields, which are eventually convoluted into a 2D array of cells.

Two-dimensional RD over images and video sequences is applied from a combination of one-dimensional processing, following the procedure presented in [25] and which delivers patterns that are stable, even in a noisy environment. The theoretical approaches governing the derivation of the reaction-diffusion equation in a continuous time and spatial domains are modified to support the circuit-level approach as expressed inwhere is a natural number representing the pixel intensity, indexes the pixel in the row of an image, and represents the number of diffusion processes, or diffusion steps. A discrete update consists of a number of diffusion steps iteratively applied from the initial row of pixel intensities. The subsequent reaction consists of subtraction and amplification by a nonlinear logistic function. Several updates are required to generate stable patterns.

This study extends the earlier theoretical study to the implementation of the reaction-diffusion pattern generation algorithms as a protoarchitecture that is mapped into a digital medium. A field-programmable gate array (FPGA) is selected for the physical implementation enabling real-time operation of the protocomputation based application.

##### 2.2. One-Dimensional Reaction-Diffusion Protoarchitecture Aiming at a Digital Media Implementation

A data-flow and a block diagram representing the one-dimensional pattern generation process are shown in Figure 1. The ideal example of a step input is presented in Figure 1(a), which is processed by the system until reaching a stable state after update number 10 as a spatial wave (one-dimensional pattern). The progress of the step input along the first update is shown, evidencing the edge smoothing obtained by the diffusion, the subtraction of the diffused state from the input step, and the result of nonlinear amplification. The state-diagram of the RD system is presented in Figure 1(b), showing the two fundamental states, namely, diffusion and reaction consisting of subtraction and amplification, as well as the iterative data passing process that is required and represented as arrows. The state-diagram of the system including the filter is presented in Figure 1(c), where the lower part of the state-diagram represents the RD process that is followed most of the time, and the upper part of the state-diagram represents the Filtering process that is followed repeatedly. Recognizing that the Filtering process consists of a regular diffusion step, the integrated state-diagram of Figure 1(d) can be derived, which evidences the possible use of identical resources for diffusion and filtering. Controlling and canceling collision of spatial waves is obtained by the action of the spatial Filtering process that is applied after the first update, and then repeatedly after a fixed number of updates, for example, Figure 1(a), in the Filter process (red box). The filter consists of one step of diffusion (blurring), which is not followed by the reaction process (subtraction and amplification). In addition, the spatial filter controls potential effects of noise which is diffused such that its effect is not amplified, resulting into damping out its propagation. In practical terms, the filtering repetition frequency, the number of diffusion steps, and the maximal gain of the sigmoid function factors are determined empirically; for example, in our application case, one diffusion filtering step is applied every four updates while the gain of the sigmoid function is equal to five.