Abstract
Cell behavior analysis is a fundamental process in cell biology to obtain the correlation between many diseases and abnormal cell behavior. Moreover, accurate number estimation plays an important role for the construction of cell lineage trees. In this paper, a novel Gaussian ant colony algorithm, for clustering or spatial overlap cell state and number estimator, simultaneously, is proposed. We have introduced a novel definition of the Gaussian ant system borrowed from the concept of the multi-Bernoulli random finite set (RFS) in the way that it encourages ants searching for cell regions effectively. The existence probability of ant colonies is considered for the number and state estimation of cells. Through experiments on two real cell sequences, it is confirmed that our proposed algorithm could automatically track clustering cells in various scenarios and has enabled superior performance compared with other state-of-the-art approaches.
1. Introduction
Cell tracking has become a focus in the bioengineering community, as it provides a powerful tool for cell tracking in understanding of cellular phenomena. For example, to analyze the cell cycle, to track the cell and extract the motion features accurately, to consider cells undergoing division accelerated the wound healing process, and to understand how the drug effects on cells. Image analysis methods are needed to extract quantitative information from these vast and complex datasets. There are numerous approaches that address the cell tracking problem in general. However, with the increasing of big datasets, manual cell tracking approaches often incur tracking error and becoming time-consuming and tedious. Thus, cell tracking with an automatic method is essential.
Unlike tracking cars or people in videos, cellular image sequences usually vary number or shapes of cells in the field of view (FOV). Also, because cells are grouped together or spatially overlapped, it is difficult to distinguish them separately. In addition, the underlying cellular activities may be diverse, including cell division, cell death, large deformation, and leaving/entering the FOV. Taken together, the multicells’ tracking becomes a very difficult task.
To cope with these difficulties, sophisticated tracking approaches have been extensively studied and many effective approaches have been developed recently [1, 2]. These approaches generally adapt some good established algorithmic framework, such as segmentation and association methods, model evolution methods, and stochastic filtering methods. However, most works typically utilize different cell tracking methods according to variable cell types and tracking requirements.
Ant colony optimization (ACO) was inspired by the idea of collective intelligence and was introduced by Dorigo [3]. It works on the principle that an ant while moving leaves pheromones on its path, which is used as a guide to be followed by other ants. In recent years, ACO algorithm has been successfully applied to solve real optimization problems. These problems include visual tracking, the traveling salesman problem (TSP) [4–6], vehicle routing [7], clustering [8], and image processing [9, 10].
In clustering cell environment, not only the states of the cells vary with time but also the number of cells also changes due to cells appearing and disappearing, but this aspect has been less discussed before in the literature. The goal of this paper is to design a tracking framework that addresses the above challenges. Ant colonies move randomly but purposely towards different cell regions. The ant colonies and pheromone-resulting distribution can be used to extract the cells state and estimate the cells number. And then, we will optimize the ant work mode to improve the algorithm accuracy through analyzing the issue about the effect of heuristic information, pheromone, and pheromone gradient information. Also, prediction strategy is utilized to reduce the processing time of ACO. In essence, the contributions of this paper can be summarized as follows:(1)An efficient Gaussian ACO method borrowed from the concept of the multi-Bernoulli random finite set (RFS) for clustered cells state and number estimator simultaneously is proposed(2)An ant work mode based on the pheromone gradient information with the exponential form is presented(3)The existence probability of ant colony is established for effectively estimating the state and number of cells
The rest of the paper is organized as follows. Section 2 gives an overview on approaches in the field of cell tracking. The necessary background on ACO algorithm is briefly described in Section 3. Section 4 details the cell-tracking methodology based on Gaussian ant colony. We evaluate the results of our algorithm in Section 5. Finally, a brief summary is given in Section 6.
2. Related Works
Cell-tracking methodologies can be divided into three general categories, i.e., detection, model evolution, and stochastic filter method.
In the detection method, cell tracking is implemented by detection of the objects in each frame and then matching cells between consecutive frames. Nasrul et al. [11] used radii limits for the cells during training data to reduce the Hough transform computation cost. However, the technique failed to work in case of occlusions, low contrast, and noise. Zhang et al. [12] proposed a novel system for adhesive cell detection and tracking by processing a sequence of images with nonrigid objects changing position and shape. However, this method required very good detection in almost all frames because low performance in cell detection may heavily affect the tracking results. In [13], a robust segmentation and tracking system for the plant cells was proposed by exploiting the cells’ spatiotemporal contextual information. Massoudi et al. [14] proposed an automatic cell tracking algorithm that did not rely on a perfect segmentation module. This algorithm can track cells that enter or leave the FOV and can also handle cell division. However, when occlusion happens at a node in the graph, the algorithm cannot distinguish between occluded cells. In sum, these techniques have limitations. The accumulation of tracking errors due to incorrect segmentation may result in the tracker losing a significant number of cells.
Model evolution approach integrates the stages of segmentation and tracking into a joint model. Mean shift [15, 16], active contours [17, 18], and level set [19, 20] belong to this approach. He et al. [21] presented an active-contour-based segmentation method which used an improved region-scalable fitting (RSF) model and was robust against initialization and noise; however it still suffered from the problem of arbitrary initialization and increased computational cost. Zimmer et al. [22] presented a parametric active contour modeling for segmentation and tracking of cell from in vitro video microscopy. However, manual initialization was required for the first frame and is unable to handle objects entering the scene later in the sequence. Li et al. [23] proposed a new distance regularized level set evolution formulation that evolves as the gradient flow minimizing an energy functional having a distance-based regularization term and external energy. In summary, these methods often have the advantage of obtaining segmentation from the model but usually have high computational cost and may require special methods of dealing with changes of object topology.
Stochastic filter methods have powerful function when objects’ motion can be modeled in a Bayesian framework. Examples of this category include particle filters [1, 24, 25], Kalman filters [26, 27], and multi-Bernoulli filtering [28, 29]. In [26], a Kalman filter-based local graph matching method was proposed to track the plant cells. Vishwanath et al. [30] presented an auxiliary particle filtering algorithm with dynamic variance adaptation of the posterior distribution to account for nonlinear movements. However, it has low performance when there were other cells in the close vicinity of the object of interest. Yang et al. [31] presented a spatially constrained particle filtering method for tracking the movement of axonal neurofilaments. However, it was not good at overlap cells’ tracking.
As mentioned above, most algorithms focus on specified cell data with special morphological characteristics and need reliable cell segmentation, which hampers the generality of the methods. To reduce computational expense and improve the effective and accuracy with different cell types, we proposed a novel ant-based approach with the cardinality estimator for multiple cells’ tracking.
3. Generic Ant Colony Systems for Cell Tracking
The ant colony optimization method is inspired by the fact that ants are able to find the shortest route between their nest and a food source. In the ant-based algorithm for cell tracking, searching for interest region (cells) is looked upon as an ant colony foraging process [32]. The approach is based on a number of artificial ants, moving on the pixels of the image driven by the image’s intensity values and pheromone amount to establish a pheromone matrix. This matrix represents the cell information at each pixel location of the image.
For one ant , placed at pixel , the probability of choosing the next pixel is described bywhere denotes the set of unvisited pixels of ant , represents pheromone on edge , is the heuristic function, where is the distance between pixels and , and and denote the weight factors to determine the relative influence of pheromone and the heuristic function on the decision of ant .
After all ants have finished their tours, update the pheromone level according towhere is the pheromone persistence coefficient. The term is the pheromone amount that deposited by ant on the edge .
The above process is repeated multiple times until certain termination conditions are reached. The following sections describe our proposed ACO algorithm, followed by details of the modifications made to attempt to improve its performance in a complexity environment.
4. Methods
4.1. Algorithm Description
In this paper, we present a Gaussian ant colony algorithm for clustering cells to reduce the computation time and improve the accuracy. First, ant colonies are pioneers and aim to find the areas where cells might exist based on existing cells’ states in the prior frame. The existence probability function of ant colony corresponding to new emerging cell is designed as a Boolean function which is specifically defined to maximize cell region coverage. It obtains one if the gray mean value of the pixels covered by the ant colony is big enough, otherwise it returns zero. And then, an ant work mode based on the pheromone gradient information with exponential form is presented, which is sensitive to small change of heuristics, the pheromone intensity, and pheromone gradient information. Finally, the existence probability of ant colonies is proposed, which can faithfully reflect the intensity field of pheromone diffusion and strengthen the collaboration and communication among ants. A large number of experiments on cell image sequence show that the proposed algorithm can not only improve the estimation accuracy but also greatly enhance convergence speed. The framework is shown in Figure 1.
Before the technical details are provided, the notation is listed in Table 1.
4.2. Gaussian Ant Colony Algorithm for Clustered Cell Tracking
Due to the random search, ACO generally needs hundreds of iterations, and each iteration is time-consuming. In fact, high computational complexity of cell state extraction algorithms has always been a tremendous challenge, especially for low signal-noise ratio image sequences. To solve these problems, the Gaussian ant colony algorithm is employed, in which ant colonies are transmitted and predicted from frame to the potential region where cells might be located at frame .
4.2.1. Stochastic Models of the Multicell State
In the image sequence, as a cell moves from the current frame to the next frame, it may survive or disappear, and new cells may appear into FOV in the current frame. So, cells are random and the number of cells is dynamic. As described in [33], stochastic models of the multicell state are established with the time evolution.
Suppose that, at frame , there are cells. is the multicell state set. will be propagated to the next time k accordingly. is the probability that the cell state continues to exist and is the probability of death and denotes the transition probability at time . It is important to note that in stands for surviving cell. The multicell state at frame can be represented as the surviving and new emerging objects:where denotes the surviving cells state set, denotes the new emerging cells state set in the FOV at frame , and stands for new appearing cell. In our concept, new appearing cells include both incoming and new born cells.
4.2.2. Gaussian Ant Colony
In the ACO framework, we assume that a cell corresponds to an ant colony. So, cell’s behavior can be represented by ant colony. In order to improve the search efficiency and increase computing speed of ACO algorithm, the initial position of ants is considered. As the cells move very slowly, the move distances between frames are slight. So, within this work, the position of the ants in the current frame is propagated according to the movement of the cell. This section describes how an ant colony moves and survives to the next frame.
Suppose is the state of ant , which represents the 2D position and corresponding velocities , respectively. The cell’s existence probability approximately equals to that of corresponding ant colonies.
The motivation is based on the fact that the multi-Bernoulli parameter set describing the multicell state and ACO is an effective tool that is able to synthetically describe the randomness of cells. To model the evolving process of ant colony from the next frame to the current frame, the concept of the generic Gaussian implementation of the cardinality balanced multitarget multi-Bernoulli (CBMeMBe) [33] is borrowed to generate Gaussian ant colony.
Suppose that the cell states at frame are approximately represented as the multi-Bernoulli parameter set corresponding to the ant colonies:where is the number of surviving cells, is interpreted as the existence probability of the th cell, and is probability density comprised of the Gaussian form:where and is mean and covariance of the th term, which is fitted by subant colony at frame . denotes a Gaussian function with mean and covariance .
The cell state at time is described by the combination of multi-Bernoulli parameters’ sets for surviving cells and new appearing cells:where the first term is the parameter set of surviving cells, is denoted as the existence probability of cell, and is probability density, which are approximately equivalent to that of corresponding ant colonies:
Assume that there are ants (, ) included in the subcolony at frame , and is the number of ants; then, the predicted ant state is given bywhere and are the state of ant at frame and , respectively, is the ant state transition matrix, and is the process noise covariance. So, we have
To represent the new cells’ appearance in the current frame, additional ants should be added. The second term is the parameter set of new appearing cells. is the number of new appearing cells. denotes the existence probability of the ant colonies, and probability density is given bywhere denote the means and covariance of the th cell. New appearing ant colony is generated following this Gaussian distribution. In general, in order to represent a division cell, ant colonies of different orientations are generated.
The existence probability of new appearance ant colony is designed with a Boolean function which is specifically defined to maximize cell region coverage. In general, the intensity average of objects is different from that of background. For a gray cell image, calculate the intensity average value of pixels’ set where ant colony located:where denotes a normalized image gray intensity at pixel , , and represents a pixel set each located by ants.
Boolean function obtains one if the gray mean value of the pixels covered by the ant colony is big enough, otherwise it returns zero. It is given bywhere is threshold.
In order to describe above process clearly, the framework is shown in Figure 2. There are 5 cells in frame . 4 cells survive and one cell (gray color) is left out in the next frame . At same time, two cells (blue color) appear in frame . Gaussian ant colonies in frame are generated based on prior information of frame .
4.2.3. Transition Probability and Pheromone Update
After ant’s distribution is given, these ant start to move forward to search and determine the number of cells and potential region where each cell is located.
During cell image sequencing, if cells are clustered together, it is difficult to distinguish them separately. It leads to the fact that, for describing cells during their activity, the development of additional ant mode for their separation is required. Considering pheromone field changes dramatically at the edge of cell, therefore, the pheromone gradient information is used in the ant decision-making model.
In each cycle of iteration, an arbitrary ant starts from pixel and selects one of the neighboring pixels following according to transition probability function :where parameters , , and are adjustment factors which control the relative importance of their individual terms, denotes the neighbors of pixel , denotes the heuristic function of pixel , is the pheromone level on pixel at the th iteration, and denotes the pheromone gradient information of pixel .
Transition probability function with the exponential form is sensitive to small change of the pheromone intensity, pheromone gradient information items, and heuristic function.
The pheromone gradient information is given bywhere denotes pheromone level on pixel with the coordinate at the th iteration.
The heuristic function is another key issue in selecting which pixels the ants will visit in the neighbor of their positions. Assume that ant is able to acquire knowledge of neighboring environment. The intensity difference between two pixels in a blob can be used as a measure to evaluate whether a pixel belongs to a cell or not.
As shown in Figure 3, ant locates at pixel , which is the center of blue color blob ( pixels), and the corresponding heuristic function is calculated bywhere is the maximum absolute value of the intensity difference between two pixels, is the maximum intensity value of the image, which is equivalent to a normalization factor, and is the intensity value on pixel with the coordinate in image.
Generally, the more the number of ants clustered in the local area, the larger the pheromone will be. Thus, it can attract more and more ants to move closer to its neighborhood. On the contrary, the pheromone, as a chemistry substance, will evaporate as time goes on, reducing its attractive strength. When ant moves to pixel , it will deposit a certain amount of pheromone, and the pheromone update function is given bywhere represents the pheromone trail of the pixel at the th iteration and is the pheromone attenuation coefficient. The bigger the is, the stronger the forgetting effect is. A constant pheromone amount is released on pixel by ant . represents the total pheromone increment that ants deposited when the th iteration ends.
In addition, the pheromone will propagate to its neighboring as time goes on. A cone model is employed to simulate the pheromone diffusion. In general, the closer the distance between the pixel and info fountain center, the stronger the field force of the intensity field. More precisely, the pheromone influence of an info fountain on other nodes will gradually reduce as the distance between pixels becomes long. Assume the center dot is denoted by the pixel , is the radius of diffusion area in the image, the pixel locates in the intensity field of the info fountain, and is the distance between pixel and the center dot in the image. Let be the pheromone amount on pixel laid by the ant colony at the th iteration. We can give the pheromone diffusion model based on the real distance between two pixels in the image, as shown in Figure 4:where denotes the set of neighbors of pixel and denotes the number of elements in set .
Both the pheromone evaporation and pheromone propagation are considered simultaneously, and the evolution of pheromone is rewritten as
4.2.4. Existence Probability of Gaussian Ant Colony
The resulting pheromone field is very important for existence probability of ant colony and state extract. When a given number of iterative runs are reached, the existing probability of ant colony can be described as in [34]. If, at frame , the number of ant colonies is , then the existence probability of ant colony is given bywhere denote the pheromone values in the position of ant located with state , is the heuristic information of pixel which ant located, and is the number of ants in th ant colony in frame .
The existence probability indicates how likely the th ant colony denotes a true cell. So, the subcolonies, whose existence probabilities are larger than a threshold (set at 0.5 in our experiments), are chosen to estimate the number and state of cells.
To reduce the false alarms caused by noise and clutter, the “pruning” and “merge” procedure described in [35] is also directly applied for our proposed algorithm. The basic idea is to discard subant colonies with negligible existence probability and merge ant colonies that are close together. Finally, trajectories of multiple cells are obtained by data association strategy between consecutive frames.
5. Simulation Results
The proposed method is tested on low-contrast cell image datasets. We present two image sequences to evaluate the tracking performance. The identified positions of the cells are overlaid on the original images to show the tracking results. All experiments are conducted in MATLAB R2016a and on i5 2.4 GHz PC.
5.1. Parameter Settings
Suppose each cell evolves in a general linear model. So, ant state prediction in frame is obtained with and , where is the sampling interval.
We tested different sets of parameters and then used the one with the best tracking performance for two image sequences. For simplicity, other parameters are illustrated in Table 2.
5.2. Results
To give an objective evaluation, the performance of our proposed cell-tracking approach is investigated, and two criteria are known as and measures and are used and given by [36]where , , , and are the number of true positive, true negative, false positive, and false negative, respectively.
5.2.1. The Tracking Result of Sequence 1
Figure 5 shows the tracking results in noisy scenarios, and some of them are clustering and occluding with each other. For example, Figure 5(c) shows how our proposed algorithm tracks’ clustering cells (cell 2, cell 5, cell 11, and cell 12), as shown zone “A” in frame 1 and zone “B” in frame 2.
(a)
(b)
(c)
Figures 6 and 7 plot the position and velocity estimates of each cell in every frame, respectively. It can be seen that the exact location of the cell in each frame and the knowledge of the life cycle of each cell are acquired. Some cells are more active than others, such as cell 6 and cell 14 exhibit greater maneuvering.
It is worth noting that our algorithm can give satisfactory tracking performance. All cells are continuously tracked by our algorithm in succeeding frames. It can handle the case that cells enter and leaving the image and cells closely contact with each other.
5.2.2. The Tracking Result of Sequence 2
Consider a typical tracking scenario where cells neither appear or disappear nor the number of cells is known a priori (and fixed). Since there is no appearing or disappearing cell, the existence probability of the birth cell and the probability of survival is .
Figure 8 illustrates the tracking results of selected images. The left top cells are in close contact with each other, as shown region “A” from frame 1 to frame 30. All cells move slowly and their shape keeps on changing irregularly and share very similar physical features, as shown in Figure 8(a). The overall trajectories are shown in Figure 8(c). It can be observed that our proposed algorithm can successfully track those cells with different cells motion types.
(a)
(b)
(c)
Figures 9 and 10 plot the position and instant velocity estimates of each cell in and coordinates versus time. It can be seen that cell 1 and cell 2 are more active than others.
From the above results, it can be seen that our proposed algorithm provides accurate track performance. It not only tracks all cells with shape change and different cells motion types but also manages to track cells which are in close contact with each other.
To verify the performance of proposed algorithm, 100 simulations on the sequences are performed in each frame. All True Positive, False Positive, True Negative, and False Negative reports are recorded, and the average , , , and for each frame are obtained. and are computed, as presented in Table 3.
5.3. Comparisons
To complete the evaluation, it was considered interesting to compare the behavior of the proposal with other cell-tracking algorithms in this scenario such as the particle filter (PF) [37] and the multi-Bernoulli filter [29]. To ensure an objective comparison, the likelihood function used in other filter methods take the same form as the heuristic information function used in the ant-based method.
We present the averaged position errors using a traditional quantitative performance measure, i.e., the root means square error (RMSE): , where , is the total number of successfully tracking, is the total number of cells at frame , and and are defined as the true and estimate positions of the cell at frame , respectively. Figure 11 plots the comparison of RMSE of different methods and our proposed algorithm obtained less RMSE values than other methods. It shows that our proposed method is more precise than others.
(a)
(b)
Average number estimates of cells in sequence 1 over 100 simulations are shown in Figure 12. The ground truth is manual. It can be found that the number errors are small.
To illustrate the effectiveness of the proposed approach, we have compared and of our method with the other techniques. All ,, , and in each frame are recorded over 100 simulations, and their averaged values are computed and the results are listed in Tables 4 and 5.
As shown in Tables 4 and 5, we provide performance comparisons of the proposed method with the other methods in this field. The comparison is made with respect to and .
It can be seen that our proposed algorithm produces significantly less false tracks and alleviate fragmented tracks because the tracker efficiently manages important hypotheses and keep confident tracks based on the existence probability. The proposed algorithm provides reliable tracking results.
As a common visual tracking method, the computing time is a major concern problem. We measured the time it took to process the sequences 1 and 2. Table 6 illustrates the comparison statistic results among various methods. It can be observed that the speed of our proposed method on sequence 1 is 38.621 s per frame and sequence 2 is 20.174 s per frame, respectively. Although our algorithm runs relatively slowly, its computation speed is still far less than the sampling interval (60 s), and this shows that our proposed cell tracking algorithm is quite suitable for automated cell tracking in our studied image sequences.
6. Conclusion
Multicells’ tracking is concerned with estimating the number of objects and their trajectories in the presence of cell appearance/disappearance, grouped together, or spatial overlap. In this work, the Gaussian ACO method borrowed from the concept of the multi-Bernoulli RFS for tracking clustering cells’ state and number estimator is proposed. Two datasets under different image scenarios (e.g., clustering cells, which are in close contact with each other and share very similar physical features) are used to make an evaluation for the proposed method. The results obtained were satisfactory and errors were found to be small.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Disclosure
The prior work of this paper has been presented at the 2018 International Conference on Control Automation and Information Sciences (ICCAIS 2018) entitled A Novel Ant-Based Multiple Cells Tracking Approach with Cardinality Estimation.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors’ Contributions
Mingli Lu and Di Wu contributed equally to this work.
Acknowledgments
This research was jointly supported by National Natural Science Foundation of China (nos. 61876024 and 61673075) and partly by the Six Talent Peaks Project in Jiangsu Province (no. 2017-DZXX-001), 333 Project of Jiangsu Province (no. BRA2019284), and Natural Science Fundamental Research Program of Higher Education Colleges in Jiangsu Province (no. 19KJB510015).