#### Abstract

We introduce a mathematical model that shows the interaction dynamics between the uninfected and the infected cancer cell populations with oncolytic viruses for the benign and the malignant cancer cases. There are two important parameters in our model that represent the malignancy level of the cancer cells and the efficacy of the therapy. The parameters play an important role to determine the possibility to have successful therapy for cancer. Our model is based on the predator-prey model with logistic growth, functional response, and the saturation effect that show the possibility for the virus to be deactivated and blocked by the human immune system after they reach a certain value. In this paper, we consider the appearance of the Hopf bifurcation on the system to characterize the treatment response based on the malignancy effect of the disease. We employ numerical bifurcation analysis when the value of the malignancy parameter is varied to understand the dynamics of the system.

#### 1. Introduction

The cancer growth is mainly caused by the gene mutation that changes the ability for cell apoptosis and replication. The mutation changes the behaviour of the cell regulations and prevents apoptosis that triggers the cancer cell immortality. If the mutations generate malignant cancer, the cancer cells can be transmitted to the other organs through the blood flow called metastasis.

In [1], the authors proposed a new model for nasopharyngeal carcinoma based on the enzyme reactions of the cell repair regulations and showed the initial time of the mutations as the early indications of the cancer. A mathematical model for cancer in the tissue level that represents the growth of the cancer cells has been done in [2, 3], where the important parameters that trigger the cancer growth have been shown in [2]. In [3], the authors found a boundary on the parameter space where the cancer cells have possibility for metastasis.

Immunotherapy and gene therapy are the cancer treatment methods that provide lower risk of defects for the normal cells. The immunotherapy model for cancer treatment was studied in [4–6], where the authors considered the interaction between the cancer cell population, the effector cells, which are a part of the human immune system, and the IL-2 compounds that play a role in stimulating the effector cells. The local analysis of the model was introduced in [5], while the global stability analysis was done in [6]. Furthermore, the authors in [4] proposed a new model, which was the generalization of the ones in [5, 6]. The model in [4] was done by assuming that the interaction between the effector cells and interleukin is no longer a constant, but it can be oscillated with small amplitude; then, they studied the dynamics of the system.

Some medical results of the cancer therapy by using oncolytic virus in the medical point of view were reviewed and discussed in [7]. The therapy was done by injecting some viruses to the cancer cells which will then infect the other cancer cells to increase their apoptosis. Quantitative impact of cytokine-expressing oncolytic virus therapy was studied in [8]. In [8], the authors showed that the oncolytic virus therapy gives more benefits to reduce the tumor burden without harming the normal cells. The treatment can be continued by some auxiliary treatments to remove the tumor completely once after the size and population are sufficiently small. Sometimes, the oncolytic virotherapy can be combined with the regular treatment such as chemotherapy or radiotherapy. The enhancement of the chemotherapy by using the oncolytic virus was modelled and discussed in [9].

Gene therapy is a method that effectively treats the cancer cells to increase apoptosis and reduce immortality [10]. One of the therapies is by using oncolytic virus which is done in vitro. The virus is injected to the cancer site for increasing the apoptosis and for reducing the metastasis of the cancer cells [7, 11, 12]. For some types of cancer, e.g., cervical cancer, Hodgkin lymphoma, and Burkitt lymphoma, the viruses play a role for inducing the regression of the cancer cells and serve as vectors to insert the anticancer genes into the cancer cells, see [11, 13] for the details.

A cancer therapy using oncolytic virus is important to make destructive effects to the cancer cells and increase the ability for apoptosis. There are two different characteristics for the cancer cells which depend on the malignancy level. The malignant cancer cells have immortal characteristics and ability to transmit elsewhere through the blood flow, which is called metastasis. For benign cancer, the cancer cells are localized in a certain organ, and they do not have the ability for metastasis to the other territories. Malignancy of the cancer cells not only depends on the ability for apoptosis and metastasis but also depends on the number of cells with DNA replicating error.

Cytotoxic T cells are the important part of the individual immune system for destroying the cancer cells. The response of the cytotoxic T cells against the cancer cells that have been already infected by the oncolytic virus was studied in [14]. Other studies of the cancer therapy using viral injection have been done in [15–17].

In [15], the authors proposed a model of the interaction between the uninfected cancer cells by oncolytic virus and the infected ones with logistic interactions and studied the bifurcations in several regions of the parameter space. In the medical point of view, Wong et al. [16] found that there are some obstacles for this therapy, such as the reaction of the immune system against the virus.

A mathematical model of tumor-immune-virus interaction was done in [18]. The model showed an interaction between some types of cells, oncolytic vesicular stomatitis virus, and adenovirus vaccine. There were some types of cells studied in the model, i.e., uninfected and infected tumor cells, central memory cells, effector cells in lymphoid tissues, and effector cells in the periphery.

A simpler tumor-immune-virus model was done in [17], where the authors considered only the interaction between the uninfected and infected tumor cells by oncolytic virus. The system was a two-dimensional system of ODE that involved the saturation effect. By the fact that the growth of the oncolytic viruses can be inhibited by the human immune system, the authors in [17] also introduced a parameter that shows the immune response of the humans against the viruses and studied the interaction between the uninfected tumor cells by oncolytic viruses and the infected tumor cells by the viruses. In this case, the population of the infected tumor cells will decrease due to the increased response of the human immune system.

The limitation of the model in [17] is that the model only covers the benign cancer case. In this paper, we propose a new model of the cancer therapy using oncolytic virus that covers not only benign cancer but also malignant cancer. We introduce two new parameters that show the malignancy level of the cancer cells and the therapeutic efficacy of the treatment to the model in [17]. The appearance of an unstable periodic solution in our system is important to determine the boundary of the solutions that separates the bounded and unbounded solutions. The size of the boundary shows the possibility of the cancer cells for metastasis.

By the fact that cancer malignancy is affected by the ability for cell apoptosis and the DNA replicating error, we measure the malignancy level of the cancer cells by the difference between the number of cell apoptosis and the number of DNA replicating errors divided by the total number of cancer cells. Benign cancer has a higher rate of apoptosis than the one in malignant cancer. However, the rate of the DNA replicating error for benign cancer is lower than the one on malignant cancer. Malignant cancer also has an ability to prevent the oncolytic virus infections and replications. The study of DNA replication and the prognostic significance of the DNA replicating error was done in [19, 20], and the apoptosis characteristics of the cells were studied in [21]. In this paper, we assume that, for benign cancer, the number of cell apoptosis is higher than the number of DNA replicating errors, so the malignancy level of the cancer cells is positive. For malignant cancer, we assume that the number of DNA replicating errors is higher than the number of cell apoptosis. Consequently, the malignancy level for malignant cancer is negative. In this case, the growth of the cancer cells can be unbounded depending on the ability of cancer for metastasis.

The second parameter shows the therapeutic efficacy of the treatment. The parameter has been measured by the ratio of the number of successful viral infections by the oncolytic viruses to the cancer cells with the total number of contacts between both populations. It shows the effective contacts between the uninfected and infected cancer cells by oncolytic virus and the possibility of the cancer cells to be removed from the individuals. To simplify the discussion, we use the term benign cancer for the tumor.

In this paper, we study the virotherapy characteristics by using oncolytic virus not only for the benign cancer cases but also for the malignant cancer cases that have ability for metastasis. We employ the bifurcation theory (see [22] for the details) to characterize the periodic solutions of the system when the value of the malignancy parameter is varied. The study is important to determine the treatment strategy to remove or reduce the number of cancer cells. The appearance of a Hopf bifurcation in our system is important to understand the treatment effects based on the malignancy level of the cancer.

This paper is organized as follows. We start with the introduction of the gene therapy and the virotherapy by using oncolytic virus for cancer. After that, we construct a mathematical model that shows the interaction between the uninfected and the infected cancer cells by oncolytic viruses with two important parameters, i.e., the malignancy and the therapeutic efficacy parameters. Our model is a generalization of the one in [17] in the sense of the parameters, and it has wider applications that are not only for benign cancer but also for malignant cancer. The malignancy parameter plays an important role to determine the conditions for the cancer cells that have ability for metastasis. In this case, we will use numerical bifurcation analysis to find the domain boundaries for the cancer cells that can be isolated to a certain amount. Furthermore, we will close the paper with some concluding remarks.

#### 2. The Model Formulation

The population of the cancer cells is classified into two subpopulations, i.e., uninfected cancer cells and the infected ones by oncolytic viruses, where the density of each subpopulation is denoted by and . In this paper, we generalize the system in [17] by introducing two new parameters that measure the malignancy level of the cancer cells denoted by and the therapeutic efficacy denoted by . Thus, we have a two-dimensional system of ODE as the following:where the initial value is and .

The value of can be positive or negative depending on the malignancy level of the cancer cells. The positive value of the parameter shows the situation that the cancer cells are well differentiated or benign, while the negative value of this parameter shows the undifferentiated or poorly differentiated cancer cells that are the characteristics of malignant cancer. The parameter is measured by the difference between the number of cell apoptosis and the number of DNA replicating errors divided by the total number of cancer cells. In the malignant cancer case, cancer prevents the cells for apoptosis, while the cells with the DNA replicating error are growing rapidly [21]. The moderately differentiated cancer cells are represented by the zero value of . Due to the degeneracy of system (1), we skip the case that as an open problem on this paper.

The infection of the cancer cells by the oncolytic virus will increase the therapeutic efficacy for the cancer cells. The parameter shows the effective contact rate between the uninfected cancer cells and the infected cancer cells by oncolytic virus and is measured by ratio of successful infection and the total number of contacts between both populations so that the parameter is assumed to be positive. All other parameters and variables on system (1) are supposed to be nonnegative. Furthermore, the system in [17] is a special case of system (1), i.e., for .

Our system and also the system in [17] deal with the saturation effect represented by . The saturation effect shows the maximum number of contacts that the individual’s immune system evolves to stop the viruses immediately after they enter and replicate. The parameter indicates the individual’s immune response that prevents the viruses for destroying the cancer cells, and the parameter represents the amount of viruses that can be transferred to the uninfected cancer cell population.

The parameter that shows the maximum per capita growth rate of the uninfected or the susceptible cancer cell subpopulation is supposed to be positive. It is due to the immortality ability of the cancer cells. The parameter that shows the maximum per capita growth rate of the infected cancer cell subpopulation by the oncolytic virus can be positive or negative, where the negative value of represents the decay rate of the subpopulation. The parameter shows the death rate of the infected cells caused by the viruses.

Both subpopulations on system (1) are assumed to follow logistic growth, where is the carrying capacity. For the benign cancer cells, they have limited space and resources to grow because they cannot spread to other organs through metastasis. The carrying capacity represents the maximum number of cancer cells that can grow in the human body. For the malignant cancer case, the growth of the cancer cells is not limited by the carrying capacity ; thus, they can spread throughout the body and increase the number of cells until the patient dies.

#### 3. Boundedness of Solutions

The boundedness can be interpreted as the natural restriction of the cancer growth due to the limitation of the resources. For the benign cancer cells, the growth is localized and limited to a certain area in the human body. However, the growth of the malignant cancer cells, which is represented by the negative value of the parameter , has the possibility to be unbounded due to the ability for metastasis.

Theorem 1. *Let be the solution of system (1). Suppose that for all , and parameter is positive. If , then the solutions of system (1) are bounded. Moreover,where .*

*Proof. * Suppose that , where and are the solutions of system (1), , , and . For , we haveInequality (3) has the solutionwhere is the solution of equation (3) for . By calculating the limit of both sides of inequality (4), we haveFor , we have an inequalitywhich is the same as inequality (3). For , we havewhere the limit of the solution is .

The proof of this theorem is partly motivated by the results in [23, 24].

In the next section, we will analyze the existence of the equilibria of system (1) and their stabilities. In this case, we will study which parameters play role for the existence and the stability. A stable equilibrium point represents the threshold of the cancer cells after a long time period.

#### 4. The Existence and the Stability of the Equilibria

There are three types of equilibria on system (1), i.e., the trivial, the semitrivial, and the nontrivial. The trivial equilibrium point, which is , exists for all values of the parameters and shows the nonexistence of the cancer cells. By using the linear analysis near the equilibrium , we found that the equilibrium is unstable for and saddle type for . In this case, the cancer cells, if they exist, cannot be completely removed.

The next equilibria are semitrivial, i.e., and , which show the nonexistence of one of the subpopulations. The semitrivial equilibrium exists for and shows the situation that the infected cancer cells by the oncolytic virus are removed from the population. The equilibrium , which exists for , shows the situation that all cancer cells have been infected by the virus. The efficacy of the therapy will increase for the larger value of . Consequently, the density of the infected cancer cells by the virus will decrease and goes to zero for . In this case, equilibrium plays the same role as equilibrium .

The trivial and both semitrivial equilibria coexist for and . The stability conditions of the semitrivial equilibria will be shown in Theorem 2.

Theorem 2. *Let and be the equilibrium points of system (1).*(1)*The equilibrium point is locally asymptotically stable for , and it is a saddle type for *(2)*The equilibrium point is locally asymptotically stable for , and it is a saddle type for *

*Proof. * The stability conditions of equilibrium points and are determined by direct calculation using linear analysis of system (1) near the equilibrium points. We found that the eigenvalues of the linear system near are and , and the eigenvalues of the system near are and .

The stability of equilibrium shows the therapy failure by the oncolytic virus. In this case, the viruses will be removed by the immune system faster than the cancer cells, and then the cancer cell population can be blown up until the maximum size . Equilibrium that exists for shows the situation that all of the cancer cells have been infected by the oncolytic virus. If the equilibrium is stable, all of the cancer cells in the population will be infected by the oncolytic virus, where the threshold is . The situation is in line with the fact that if the cancer cells have been detected in the earlier stage, the population can be isolated to a certain size.

The last type of the equilibria is the nontrivial. In general, system (1) has the possibility to have more than one nontrivial equilibrium point where the existence of each equilibrium depends on the parameters. Due to the dimension of the parameter space of our system, the existence of the nontrivial equilibria is complicated to study analytically. In this paper, we consider the case that there is only one nontrivial equilibrium point that exists in our system. We left the other cases of the nontrivial equilibria as open problems.

Suppose that , , and , where and . In Theorem 3, we show the existence conditions of the nontrivial equilibrium of system (1).

Theorem 3. *If and , then system (1) has a nontrivial equilibrium, i.e.,where and .*

*Proof. * For , we have the equationwhere and . By the assumption that and , it is easy to prove that . Furthermore, for , equation (9) has only one positive root of , i.e., . By substituting to the equation of system (1), we haveBy using direct calculation, equation (10) has a nonnegative solution whenMoreover, by substituting to the equation of , then we have , where and .

#### 5. Numerical Bifurcation Analysis for the Malignant Cancer Case

In Section 2, we mention that the malignant cancer case on system (1) is represented by the negative value of the parameter . As a result, the growth rate of the uninfected cancer cells by oncolytic viruses not only depends on the linear term of but also a positive quadratic term with respect to . Therefore, the growth of the uninfected cancer cell population is more faster than the one for the case .

In this section, we do some numerical simulations for the variation of . We use , and . Some of these parameter values are adopted from [17].

In Figure 1(a), we show the continuation of a nontrivial equilibrium point of system (1) for the variation of the parameter . The nontrivial equilibrium undergoes Hopf bifurcation that creates an unstable periodic solution for , and we found that the uninfected cancer cell population equilibrium represented by will increase for the lower value of . In Figure 1(b), we show that the periodic solution, which represents a cycle of the disease, raises its period when the parameter increases.

**(a)**

**(b)**

The unstable periodic solution plays a role as the boundary of two different characteristics of the other solutions. The solutions that have the initial value inside the boundary will tend to the equilibrium point, while the ones that have the initial value outside the boundary will be unbounded or go to the other invariant structure.

Inside the boundary, the solutions of system (1) will tend to the equilibrium point , see Figure 2(a). In this case, the solutions outside the boundary are unbounded. In Figure 2(b), we show the trajectory of the solutions with initial value .

**(a)**

**(b)**

The interpretation of these situations is the following. The cancer therapy using the oncolytic virus will be succeeded if the initial populations of the uninfected cancer cells and the infected ones are inside the boundary. The growth of cancer cells is unbounded if the initial populations are outside the boundary. In this case, the cancer cells have the ability for metastasis to the other sites, and the therapy is failed.

In Figure 3, we show the boundaries of the solutions represented by the periodic solutions for some values of the parameter . The smaller boundary represents higher malignancy of the cancer that implies the lower possibility to have successful therapy.

The two parameter continuations of the Hopf point have been shown in Figure 4. The Hopf point represents the situation that the therapy has the possibility to be successful. In Figure 4(a), we show the interaction between the malignancy level of the cancer cells and the individual immune response for the viruses. In this case, the higher individual immune response for the viruses will decrease the malignancy level of the cancer cells that can be successfully treated by oncolytic viruses.

**(a)**

**(b)**

In Figure 4(b), we show the interaction between the amount of virus that can be transferred to the uninfected cancer cells and the malignancy level of the cancer cells. The successful treatment of the higher malignancy level of the viruses, which is represented by the lower value of the parameter , needs higher amount of the viruses that can be transferred to the uninfected cancer cells.

#### 6. Concluding Remarks

The parameter that shows the malignancy level for cancer is important to characterize the possibility to have successful treatment for the disease. If the parameter is positive, the solution of system (1) that represents the total number of cancer cells is bounded. In this case, the cancer is benign, and if the efficacy of the therapy increases, we will have higher possibility to remove the cancer cells.

If the malignancy parameter is negative, the system has a possibility to have unbounded solutions. The appearance of the unstable periodic solution created by the Hopf bifurcation shows that the bounded domain of the system becomes larger when the malignancy parameter goes to zero. Inside the cycle, all solutions go to a stable equilibrium point. The appearance of the stable equilibrium point represents the absorbing situation of the cancer cells where they can be isolated to a certain amount. Outside the cycle, we have the unbounded solutions that represent the possibility of the cancer cells for metastasis.

The therapy efficacy can affect the existence of one of the equilibrium points. It means that there is one of the steady-state conditions of the system, where the cancer cells can be isolated in a certain amount, depending on the therapy efficacy. The optimal therapy efficacy and the optimal malignancy degree that can be treated without any effect to the healthy cells are still open problems in this paper. Understanding the role of the malignancy and the efficacy parameters in the system is important to determine the treatment strategy to remove or to isolate the cancer cells.

#### Data Availability

There are no original data included within the article. The value of parameters that have been used in this article is based on the result of other paper [17].

#### Conflicts of Interest

The authors declare that they have no conflicts of interest.

#### Acknowledgments

The authors wish to thank the Department of Mathematics, Faculty of Mathematics and Natural Sciences, Universitas Gadjah Mada, for supporting this research financially and for providing facilities. This research was also partially funded by the Directorate for the Higher Education, Ministry of Research, Technology, and Higher Education of Indonesia, through the Research Grant Penelitian Dasar Unggulan Perguruan Tinggi (PDUPT) UGM 2018. The authors would also like to thank some colleagues in UGM and the Cancer Modelling Team, UGM, for the discussions during the research.