Evidence-Based Complementary and Alternative Medicine

Evidence-Based Complementary and Alternative Medicine / 2020 / Article

Research Article | Open Access

Volume 2020 |Article ID 8961341 | https://doi.org/10.1155/2020/8961341

Yejing Shen, Mengyun Hu, Qianglong Chen, Yanyang Zhang, Junying Liang, Tingting Lu, Qinqin Ma, Ruijie Ma, "Comparative Effectiveness of Different Combinations of Treatment Interventions in Patients with Stroke at the Convalescence Stage Based on the Markov Decision Process", Evidence-Based Complementary and Alternative Medicine, vol. 2020, Article ID 8961341, 9 pages, 2020. https://doi.org/10.1155/2020/8961341

Comparative Effectiveness of Different Combinations of Treatment Interventions in Patients with Stroke at the Convalescence Stage Based on the Markov Decision Process

Academic Editor: José L. Rios
Received10 Jan 2020
Revised24 Mar 2020
Accepted02 Apr 2020
Published12 May 2020

Abstract

Objective. The objective of this study was to compare the effectiveness of different combinations of interventions in patients with stroke at the convalescence stage based on the electronic health records (EHRs) by using the Markov decision process (MDP) theory and explore the feasibility of the Markov model in the real-world study (RWS). Methods. Screening was conducted for patients with stroke at the convalescence stage who were admitted to the Third Affiliated Hospital of Zhejiang Chinese Medical University from January 2012 to January 2017 based on the EHRs. The relevant clinical data were extracted, and the appropriate conversion was made (state-action-reward) according to the Markov model. The transformed data were analysed and solved by the MDP to obtain the best interventions for patients with various stroke recovery periods. Results. 926 patients with stroke at the convalescence stage were initially selected. And according to the inclusion exclusion criteria, 854 patients were screened. Through the MDP, we obtained the following results: (1)when the patients with stroke at the convalescence stage have a medical history, but no complications, and mild neurological impairment, ≥66-year- and 18–45-year-old patients are advised to choose acupuncture treatment. 46–65-year-old patients are advised to choose rehabilitation treatment. When patients with moderate to severe neurological impairment, patients are advised to choose rehabilitation, Chinese herbal decoction, and acupuncture combined therapy. (2) Without complications or medical history, patients who are ≥ 66 years old are recommended to choose rehabilitation treatment when the nerve function impairment is mild; rehabilitation and acupuncture treatment are recommended when moderate and severe injuries are caused. (3) The combination of rehabilitation, Chinese herbal decoction, and acupuncture treatment is recommended for patients with phlegm and blood stasis. Acupuncture treatment is recommended for patients with mild impairment of nerve function in qi deficiency and blood stasis type. Rehabilitation, Chinese herbal decoction, and acupuncture treatment are recommended for moderate-severe injuries. Conclusions. The MDP makes it possible to study the effectiveness of various treatment methods in stroke patients who are at the convalescence stage. Further exploratory studies using MDP theory in other areas in which complex interventions are common would be worthwhile.

1. Introduction

Comparative effectiveness research (CER) belongs to the RWS and is also one of the clinically effective studies, also known as a “patient-centered outcome study.” CER combines such findings as the patients’ basic characteristics, economic conditions, and personal preferences and then compares the advantages and disadvantages of various current medical measures to determine the best treatment plan for individuals [1, 2]. CER improves comprehensive medical services, such as prevention, diagnosis, and treatment of certain diseases and conditions [3]. CER is divided into observational research and nonobservational research. The core of observational research is to collect large-scale data sets, including patient signs, costs, and curative effects, from health management databases and EHRs. And the data are subsequently mined, calculated, analysed, compared, and quantified by researchers.

The MDP refers to the decision-maker’s observation of the stochastic dynamic system with Markovity, periodically or continuously, and making decisions sequentially [4]. According to the state observed at each moment, an action is selected from the set of available actions, and the next state (future) of the system is decided by the environment and created by the observed data set. The probability of the transferred state has Markov property. The decision-maker makes new decisions based on the newly observed state and repeats this action accordingly. Because of its advantages in optimizing medical resources and selecting methods of diagnosis and treatments, MDP theory has played an important role in decision-making in the context of medical equipment management, hospital management, operation time, and adjustment of treatment strategies [58].

Stroke is one of the four major chronic diseases which include wind, consumptive disease, bloat, and dysphagia of traditional Chinese medicine (TCM) [9]. “Stroke” in modern medicine is a general term in TCM for acute cerebrovascular diseases. According to the statistics of the World Health Organization [10], in the 20th century, stroke was the second highest cause of death. At present, the annual incidence rate is 120/100,000 to 180/100,000, and the mortality rate is 60/100,000 to 120/100,000. Stroke has the characteristics of high incidence, high mortality, high disability rate, and many complications. Stroke threatens the health of many people and brings a heavy burden to patients, their families, and society [11, 12], which has become a global public health problem [13]. In 2015, the China Stroke Association first reported [14] that stroke was the main cause of death among Chinese residents in the Chinese Stroke Epidemic Report. TCM holds that the underlying pathogenesis is a dysfunction of yin and yang and a disordered flow of qi and blood, which then invade the brain [15]; therefore, stroke is inextricably linked with wind, fire, sputum, stagnation, qi, and deficiency.

Currently, patients with stroke at the convalescence stage are always treated with a combination of Chinese and modern medicine. At present, studies [16] have shown that the use of TCM or acupuncture has a significant effect on stroke patients at the convalescence stage. However, the question of how to comprehensively evaluate the combination of traditional Chinese and Western medicine treatment for stroke is a major clinical issue today. The main goal of this study is how to improve treatment efficacy and choose the optimal treatment to patients.

To address these issues, the author collected the hospitalized EHRs of patients with stroke at the convalescence stage who were admitted to the Third Affiliated Hospital of Zhejiang Chinese Medical University from January 2012 to January 2017. The report is as follows.

2. Methods

2.1. Establishment of the Data Set
2.1.1. Data Collection

The hospital information staff and clinical professionals were responsible for the data collection from the EHRs, and 926 patients were initially selected. The inclusion criteria were a primary diagnosis of atherosclerotic cerebral infarction, cerebral embolism, or cerebral haemorrhage and between 30 and 180 days after the onset of stroke. Patients who had the occurrence of cerebrovascular accident before or experienced limb motor dysfunction due to other reasons were excluded. Patients were eliminated from the study if other interventions (such as surgery, endovascular intervention, and arteriovenous thrombolysis) were performed during hospitalization.

According to the inclusion and exclusion criteria, 854 patients were screened. The extracted information on the patients included the basic hospitalization information, diagnosis, medical information, physical examination, and other modules.

The study has passed the ethical review of the Ethics Committee (ZSLL-KY-2015-044), and we respected and protected the privacy of patients. All of the data were collected with an information acquisition form that captured the general information of the patient, TCM and Western medicine diagnosis, all applied treatments with course detail, physical examinations, and so on.

2.1.2. Data Standardization

The standardization of Western medicine diseases was carried out according to the International Classification of Diseases Code ICD-10. TCM diagnostic syndrome information was standardized according to Traditional Chinese Medicine Diagnostics [17].

2.1.3. Data Clean-Up and Set Establishment

The data in the EHRs were extensive; thus, the occurrence of abnormal data could not be avoided. Abnormal data could lead to deviations in the analytic results and affect the authenticity of real-world evidence. Therefore, in this study, abnormal values were eliminated. Based on the research purpose, clinical professionals created analytical data sets. Through the verification of the data, the missing data were found and filled in.

2.2. MDP Analysis

Figure 1 is a schematic diagram of an MDP with three states and two actions (from http://en.wikipedia.org/wiki/Markov decision process). In this study, S0, S1, and S2 correspond to patient states with different combinations of variables. Then, a0 and a1 represent different treatment combinations (actions). These actions have a certain probability in patients with different states, according to probability transfer and average benefits to calculate which action is the optimal solution. The data of this study belonged to the discrete distribution and could be calculated and solved by using a finite MDP.

In a finite MDP, the sets of states, actions, and rewards (S, A, and R) all have a finite number of elements. In this case, the random variables Rt and St have well defined discrete probability distributions dependent only on the preceding state and action [18] (see Figure 2).

2.2.1. Data Extraction and Conversion

To determine the key characteristics for describing the condition of patients with stroke at the convalescence stage and the criterion to be optimized by using MDP theory, a panel was formed that included scholars, professors, computer experts, physicians of Western medicine, TCM practitioners, and graduate students.

Based on the basic characteristics of patients with stroke at the convalescence stage, six aspects were established (see Additional file 1: Appendix 1): including (i1) age, (i2) medical history, such as diabetes, hypertension, coronary heart disease, abnormal blood liquid level, or auricular fibrillation, (i3) complications, such as pulmonary infection, urinary tract infection, or deep vein thrombosis, (i4) Western medicine diagnosis, (i5) syndrome differentiation, such as wind-fired disturbance type, phlegm-blocking type, yin-deficiency type, qi deficiency and blood stasis type, and phlegm-heat type, and (i6) neurological function impairment scores. A score was used to describe the level of neurological function defect. The total scores were in the range of 0–29. The scores were divided into five grades, as follows: level 1 (0–2), level 2 (3–5), level 3 (6–12), level 4 (13–19), and level 5 (20–29) (see Additional file 1: Appendix 2). For the patient with an outcome of death, the default score was 29. Connected with clinical practice, level 1 and 2 represent that the severity of neurological deficits is mild, level 3 is moderate damage, and level 4 and 5 are severe injuries.

Action is known as treatment. All patients were given Western medicine for secondary prevention. Thus, the treatment methods were divided into the following three circumstances (actions) a1a3 (see Additional file 1: Appendix 1): (a1) whether to use rehabilitation therapy; (a2) whether to use Chinese herbal decoction; and (a3) whether to use acupuncture treatment. In this study, treatment strategies were carried out at the request of the physician in charge of the patient under the same theory of TCM [19]. A patient who treated by a rehabilitation specialist was considered to receive undergone rehabilitation treatment; otherwise, the one was not considered to use rehabilitation. The clinical treatment of Chinese herbal decoction and acupuncture must be consistent with TCM syndrome differentiation.

This reward refers to the value of the differential between the scores for neurological function impairment before and after treatment. According to the expert’s advice, the total reward values became the criteria to be optimized. If the value was 0, the patient’s condition was no change; if the improvement value was >0, the patient’s condition could be improved with the corresponding treatment plan. An improvement value < 0 indicated that the patient’s condition has deteriorated. Then, the larger the value was, the better the improvement in state. The action that maximized the total reward value was regarded as the optimal action. And that was the optimal intervention combination for the corresponding state.

Combined with the mathematical model, this study collected the model decision time on the first day, the 7th day, and the 14th day after admission; the included decision time did not exceed 147 days. In summary, the state, action, and rewards were assessed every 7 days.

Each state and action were converted into a digital combination, for example, a patient’s state could be expressed as 200112 and 200111. The behaviour combination was 110, 101, and so on. Patients in the same state with the same treatment actions could produce different state outcomes. The possibility of transitioning to different states represents the probability of metastasis.

2.2.2. MDP Simulation

Through the data transformation, we obtained the following: the state set S and the action set A, π←random strategy, Q (s, a)←random function, and Returns (s, a)←empty table. Initially, (a) S0∈S and A0∈A (s) are randomly selected. According to the strategy π, starting from S0 and A0, the state is terminated when the state does not occur for a long time or does not exist. In any event, after 500,000 games, the value function is very well approximated. So, the number of cycles set in this study was 500,000.

For each (s, a) that appears in the sequence, calculate G, G = Returns (s, a) =  add G to the list of Returns (s, a), Q (s, a) =   = average (Returns (s, a)). For each S update strategy,

2.2.3. Principles and Basic Methods of Treatment

The institute of acupuncture and moxibustion is the key specialty of the State Administration of TCM. Since 2010, the clinical path management of stroke has been implemented. The treatment plan is implemented in accordance with the diagnosis and treatment plan promulgated by the State Administration of TCM, which ensures the feasibility of this study.

3. Results

3.1. Case Data

The information description of the patients at the time of admission is detailed in Table 1 because progress notes’ records were incomplete, and there were missing 31 patients’ partially data. 12 patients were only missing ataxia data, 8 patients were missing sensory disturbance data, and 2 patients were missing visual field defects data. 9 patients were missing ataxia and sensory disturbance data. All missing data were randomly filled by a computer.


n = 835Male n = 501Female n = 334

Age n (%)
 18–45 years51 (6.11)40 (7.98)11 (3.29)
 46–65 years251 (30.06)173 (34.53)78 (23.35)
 ≥66 years533 (63.83)288 (57.49)245 (73.35)

Disease history n (%)
 None76 (9.10)48 (9.58)28 (8.38)
 Have one262 (31.38)165 (32.93)97 (29.04)
 Have at least two497 (59.52)288 (57.49)209 (62.57)

Complication n (%)
 None614 (73.53)372 (74.25)242 (72.46)
 Have at least one221 (26.47)129 (25.75)92 (27.54)

Western medicine diagnosis n (%)
 Ischaemic stroke637 (76.29)378 (75.45)259 (77.54)
 Haemorrhagic stroke198 (23.71)123 (24.55)75 (22.46)

Syndrome differentiation of TCM n (%)
 Wind-fired disturbance type13 (1.56)8 (1.60)5 (1.50)
 Phlegm-blocking type498 (59.64)315 (62.87)183 (54.79)
 Yin-deficiency type64 (7.66)37 (7.39)27 (8.08)
 Qi deficiency and blood stasis type227 (27.19)123 (24.55)104 (31.14)
 Phlegm-heat type33 (3.95)18 (3.59)15 (4.49)

Levels of neurological functional impairment n (%)
 Level 1 (0–2)89 (10.66)59 (11.78)30 (8.98)
 Level 2 (3–5)138 (16.53)85 (16.97)53 (15.87)
 Level 3 (6–12)398 (47.66)233 (46.51)165 (49.40)
 Level 4 (13–19)171 (20.48)104 (20.76)67 (20.06)
 Level 5 (20–29)8 (0.96)4 (0.80)4 (1.20)
 Incomplete data31 (3.71)16 (3.19)15 (4.49)

3.2. Markov Decision Analysis Results
3.2.1. State Data Set and Action Data Set

The state data set consisted of six states (i1i6), and 835 patients had 236 state combinations through data conversion. The action data set consisted of three actions (a1a3) and eight combinations (see Table 2 for details).


Codea 1a 2a 3

1111
2110
3101
4100
5011
6010
7001
8000

Each patient had a corresponding combination of behaviours at the decision-making moment. Table 3 shows the quantity of patients and the main corresponding behavioural combinations in the real world.


Patient’s quantityStateCorresponding behavioural combinations
i1i2i3i4i5i6a1a2a3

34320123000
001
111
011
101
110

25220123111
001
000
101
010
011
100
110

3.2.2. Rewards and Transition Probability

A total of 236 states were randomly selected for treatment with a combination of random behaviours to achieve the next state. The difference in neurological function between the two states is called the “reward”. The transition probability is the probability of transferring between states within a certain period of time. For patients with stroke at the convalescence stage in the same state, a combination of behaviours could be used to obtain multiple combinations of states. As shown in Table 4, combined with the reward and the probability of transition, the MDP recommended that patients in this state took the 111 regimen, and that was rehabilitation and Chinese herbal decoction combined with acupuncture treatment.


CodePretreatment stateActionPosttreatment stateRewardTransition probabilities

7112022311112022230.5151
11112022310.0303
00112022300.3333
10112022300.1212

3.2.3. Treatment Decision for Patients with Stroke at the Convalescence Stage

After the data extraction and transformation and the MDP, 835 patients obtained a total of 236 states, each of which corresponded to the optimal solution.

(1) Decision-Making for the Patients with Different Ages, Related Diseases, and No Complications. According to the results of the MDP, patients with disease history, no complications, and mild dysfunction, aged more than 66 years or between 46 and 65 years, had 7 optimal decisions. The former adopted the 001 treatment plan. In other words, the frequency of acupuncture treatment was the highest; the latter adopted the 100 treatment scheme, and the frequency of rehabilitation treatment was the highest; patients who were 18 to 45 years old had 4 types of optimal decisions, of which 001 was the highest frequency. When the neurological function was moderate to severe, the highest frequency was the rehabilitation, Chinese herbal decoction, and acupuncture combined treatment plan (see Table 5 for details).


Degree of neurological deficit≥66 years46–65 years18–45 years
ActionFrequencyActionFrequencyActionFrequency

Mild000400010013
001801021002
010410061012
110111011112
10121011
01110113
11121114

Moderate to severe000100030101
001100121101
010210031012
100111010112
101810121115
011511115
1119

(2) Decision-Making for Patients ≥66 Years Old without Complications. According to the results of the MDP, for patients who were ≥66 years without complications and with mild neurological deficits, there were a total of 4 optimal combinations, of which the 100 treatment scheme had the highest frequency. There were 7 optimal combinations for patients with a history of related diseases, of which the frequency of acupuncture treatment with the 001 treatment plan was the highest. When the neurological function was moderate to severe, the 111 regimen had the highest frequency, i.e., using a combination of rehabilitation, Chinese herbal decoction, and acupuncture treatment (see Table 6 for details).


Degree of neurological deficitWithout disease historyWith disease history
ActionFrequencyActionFrequency

Mild00010004
00110018
10030104
11011101
1012
0111
1112

Moderate to severe10010001
10110011
01120102
11131001
1018
0115
1119
0004

(3) Decision-Making for Patients with Phlegm-Blocking Type and Qi Deficiency and Blood Stasis Type. According to the results of the MDP, there were 7 kinds of optimal combinations for patients with phlegm-blocking type. Among these combinations, the 111 combination had the highest frequency, which was rehabilitation, Chinese herbal decoction, and acupuncture combined therapy. For patients with qi deficiency and blood stasis type, when the degree of neurological deficit was mild, there were 8 optimal combinations. Among them, the 001 treatment plan had the highest frequency, which was the acupuncture treatment. When the degree of neurological deficit was moderate to severe, there were 8 best combination recommendations, of which treatment plan 111, which was rehabilitation, Chinese herbal decoction, and acupuncture combined therapy, had the highest frequency (see Table 7 for details). Because the cases of wind-fired disturbance type, yin-deficiency type, and phlegm-heat type patients were small, according to the MDP, we could not obtain the corresponding optimal combination.


Degree of neurological deficitPhlegm-blocking typeQi deficiency and blood stasis type
ActionFrequencyActionFrequency

Mild00070004
00190016
01030102
10091003
10151104
01111012
111100114
1113

Moderate to severe00050006
00150012
01030104
11021003
101131104
01151015
111200114
11115

4. Discussion

Western countries have taken the lead in proposing CER in the face of increasing medical expenditures and limited medical resources. Unlike traditional randomized controlled trials (RCTs), RCTs are used in the field of efficacy research with the goal of determining the effectiveness of a treatment and focusing on whether predetermined interventions produce the desired results [20]. As a form of empirical medicine, the clinical efficacy of TCM is fundamental to its existence and development. This is a scientific method used to evaluate the actual clinical effect between Chinese and Western medical treatments during the recovery period of stroke. This method is the key to improving and researching the clinical treatment level and clinical practice of TCM [21].

The “holistic concept” is one of the primary features of the theoretical system of TCM. In the course of treatment, the holistic view requires the doctor to be “patient-centred”, rather than “disease-centred”, and the whole must be combined with local, physical and mental, medical and patient [22]. The overall view coincides with the idea of CER's “patient-centred outcome study.” Another major feature of the theoretical system of TCM is “dialectical treatment”, which reflects that Chinese medicine treatment is a complex and dynamic process. The choice of outcome indicators is also diversified. This complex change is difficult to verify with RCT, but CER is used to explore the options, and the evaluation appears to be more in line with the essence of TCM and theoretical characteristics. Stroke is a complex disease, which evolves and transforms rapidly. Therefore, one doctor chooses a treatment rely on patient’s condition.

The CER is conforming with the theoretical system of TCM and is a good way to the clinical research, which is a suitable method for excavating the clinical treatment methods and convenient for answering the question of which type of treatment is suitable for such groups [23]. Therefore, this study introduced CER into the efficacy study among treatment methods for stroke at the convalescence stage.

A total of 835 patients with stroke at the convalescence stage who were admitted to the Third Affiliated Hospital of Zhejiang Chinese Medical University from January 2012 to January 2017 were included in the study. The patients’ data were collected from EHRs, in the other word, the data came from the real clinical world, and there were many confounding factors. At present, most of the observation data are processed by the method of propensity score that achieves the aim of “close to random allocation data” [24]. However, the treatments of patients at the convalescence stage are dynamic which bring great challenges and difficulties about the statistical analysis. So, this study introduced the MDP by establishing a mathematical model to achieve the research purpose. In medicine, the MDP is commonly used in survival analysis and health economic evaluation of diseases, such as tumors [25], unstable angina [26], and AIDS [27].

Most of the CER data come from various databases, and the popularity of EHRs provides strong support for the development of CER. Recording EHRs is the medical staff's work, and these contain the diagnosis and treatment process of the patients (including outpatients, inpatients, and medical examiners) [28]. EHRs include the completed and detailed clinical information which is generated and recorded during hospitalization. Under the strict management by the Zhejiang Provincial Health Department, the medical files are relatively sound and can reflect the real-world treatment status. EHRs can provide particular information on patient characteristics, such as race, medical history, family history, disease, and treatment changes. Rich data resources and accurate data contents can reduce the cost of CER and practical trials and, meanwhile, increase the research effectiveness [29].

Past research implies that rehabilitation treatment can help stroke at convalescence stage patients to exert their own potential, promote the recovery of patients' physiological functions, avoid complications or secondary disorders caused by reduced activities, and reduce the degree of disability of patients [30, 31]. Using Chinese herbal medicine to treat patients with stroke recovery has certain advantages. It can adjust the body's physical enginery, promote the recovery of neural function, and reduce the disability rate [32, 33]. Pan’s team [34]has found that Huanyu Ditan Linao Decoction can effectively improve the clinical efficacy of patients with hypertensive cerebral haemorrhage during recovery and improve their exercise and living ability. Zhu’s team [35] had found that Qingnao Yiyuan Decoction could effectively improve patients' neurological deficit scores and downregulate the expression levels of S100 B and NES. As a green therapy, acupuncture has an irreplaceable position in the treatment of stroke at the convalescence stage. A large number of studies [3642] have confirmed that acupuncture has a satisfactory therapeutic effect on the stroke at the convalescence stage. At present, the recovery period of acupuncture treatment for stroke can be roughly divided into two categories, for the treatment of limb functional dysfunction and the treatment of nonlimb functional dysfunction. For limb functional dysfunction, Y Zheng’s team [36] conducted a meta-analysis of acupuncture treatment of limb dysfunction after stroke, including 16 RCT papers and 1280 patients. The conclusion was that acupuncture could improve the patient's motor function and quality of daily life. Acupuncture also has a good effect on nonlimb functional dysfunction, such as oculomotor palsy after stroke [37], poststroke pseudobulbar palsy [38], and poststroke dysphagia [3942]. In summary, acupuncture plays an important role in the treatment of stroke at the convalescence stage, and the guideline is also recommended as a level I evidence.

This study was based on EHRs and the innovative use of CER combined with mathematical models for data mining of patients with stroke at the convalescence stage. To some extent, this study has evaluated the real-world clinical data and suggested treatment for stroke at the convalescence stage, which made it possible for patients with various stroke recovery periods to choose the best treatment. Combined with the characteristics of TCM, MDP theory can continue to be used for certain common diseases to improve clinical efficacy and maximize the benefits for healthcare workers, patients, and other parties.

However, this study has several limitations. First, all of the data were taken from EHRs, and missing data were unavoidable. The results should be interpreted cautiously because of the possible bias caused by the replacement of missing data. In addition, due to too much variety, different components of rehabilitation therapy, Chinese herbal decoction, and acupuncture prescription were classified as one action. As a result, the effectiveness of different components of them is not comparable. This would limit the value and performance of the MDP planning for stroke treatment. Another limitation is that the key characteristics representing the patient states were based on the results of an expert panel meeting and previous study [43], so it might be missed some information about stroke patients. Finally, the research data were collected only from the Third Affiliated Hospital of Zhejiang Chinese Medical University, and area studies can be tried in the future.

5. Conclusion

In summary, the MDP makes it possible to study the effectiveness of various treatment methods in stroke patients who are at the convalescence stage. However, this study was an exploratory study and provided a different research method for the RWS. The results obtained by using the MDP need further research. Further exploratory studies using MDP theory in other areas in which complex interventions are common would be worthwhile.

Abbreviations

EHRs:Electronic health records;
MDP:Markov decision process;
RWS:Real-world study;
CER:Comparative effectiveness research;
TCM:Traditional chinese medicine;
RCTs:Randomized controlled trials.

Data Availability

The data sets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

RJM and YJS were responsible for the study design, analysis and interpretation, drafts and revision of the article, and final approval for submission. MYH, TTL, and JYL executed study design, acquisition and cleanup of the data, revision of the article, and final approval for submission. YYZ and QQM performed study design, analysis of the data, drafts and revision of the article, and final approval for submission. QLC carried out study design, drafts and revision of the article, and final approval for submission and formulate an MDP model. All authors read and approved the final manuscript.

Acknowledgments

The authors thank the Third Affiliated Hospital of Zhejiang Chinese Medical University for their substantial facilitation of all necessary materials and equipment. This work was supported by the National Natural Science Foundation of China (81574057) and Modernization of Traditional Chinese Medicine Program of Zhejiang Province (2020ZX011).

Supplementary Materials

Appendix 1: state, action, and corresponding values. Additional file 2 ( Appendix 2): neurological functional impairment assessment for patients with stroke at the convalescence stage. (Supplementary Materials)

References

  1. C. M. Ashton and N. P. Wray, Comparative Effectiveness Research: Evidence, Medicine, and Policy, Oxford University Press, New York, NY, USA, 2013.
  2. Y.-J. Shen, Q.-Q. Ma, Y.-Y. Zhang, and R.-J. Ma, “Application status and prospect of comparative effectiveness research on clinical acupuncture research,” Journal of New Chinese Medicine, vol. 49, no. 12, pp. 155–158, 2017. View at: Google Scholar
  3. Institute of Medicine, Initial National Priorities for Comparative Effectiveness Research, The National Academies Press, Washington DC, USA, 2009.
  4. D. J. White, Markov Decision Processes, Biddles Ltd, King’s Lynn, UK, 1993.
  5. L. G. N. Nunes, S. V. de Carvalho, and R. d. C. M. Rodrigues, “Markov decision process applied to the control of hospital elective admissions,” Artificial Intelligence in Medicine, vol. 47, no. 2, pp. 159–171, 2009. View at: Publisher Site | Google Scholar
  6. P. Magni, S. Quaglini, M. Marchetti, and G. Barosi, “Deciding when to intervene: a Markov decision process approach,” International Journal of Medical Informatics, vol. 60, no. 3, pp. 237–253, 2000. View at: Publisher Site | Google Scholar
  7. M. Kim, A. Ghate, and M. H. Phillips, “A Markov decision process approach to temporal modulation of dose fractions in radiation therapy planning,” Physics in Medicine and Biology, vol. 54, no. 14, pp. 4455–4476, 2009. View at: Publisher Site | Google Scholar
  8. M. Hauskrecht and H. Fraser, “Planning treatment of ischemic heart disease with partially observable Markov decision processes,” Artificial Intelligence in Medicine, vol. 18, no. 3, pp. 221–244, 2000. View at: Publisher Site | Google Scholar
  9. R. Liu and N.-G. Xu, “Literature research on acupoints selection of stroke treated by acupuncture,” China Journal of Traditional Chinese Medicine and Pharmacy, vol. 27, no. 4, pp. 819–923, 2012. View at: Google Scholar
  10. V. Hachinski, “World stroke day proclamation,” Stroke, vol. 39, no. 9, pp. 2409–2420, 2008. View at: Publisher Site | Google Scholar
  11. A. S. Go, D. Mozaffarian, V. L. Roger et al., “Heart disease and stroke statistics-- 2014 update: a report from the American Heart Association,” Circulation, vol. 129, no. 3, pp. e28–e292, 2014. View at: Google Scholar
  12. V. L. Feigin, C. M. Lawes, D. A. Bennett, S. L. Barker-Collo, and V. Parag, “Worldwide stroke incidence and early case fatality reported in 56 population-based studies: a systematic review,” The Lancet Neurology, vol. 8, no. 4, pp. 355–369, 2009. View at: Publisher Site | Google Scholar
  13. V. L. Feigin, M. H. Forouzanfar, R. Krishnamurthi et al., “Global and regional burden of stroke during 1990-2010: findings from the global burden of disease study 2010,” The Lancet, vol. 383, no. 9913, pp. 245–255, 2014. View at: Publisher Site | Google Scholar
  14. National Health and Family Planning Commission and Prevention and Control Engineering Committee, Chinese Stroke Epidemic Report2015, Pecking Union Medical College Press, Beijing, China, 2015.
  15. Z.-Y. Zhou, Chinese Internal Medicine, China traditional Chinese medicine press, Beijing, China, 2 edition, 2007.
  16. J. Wang and X. Xiong, “Current situation and perspectives of clinical study in integrative medicine in China,” Evidence-Based Complementary and Alternative Medicine, vol. 2012, pp. 1–11, 2012. View at: Publisher Site | Google Scholar
  17. W.-F. Zhu, Diagnostics of Traditional Chinese Medicine, China traditional Chinese medicine press, Beijing, China, 2 edition, 2007.
  18. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, The MIT Press, London, UK, 2017.
  19. X.-L. Mou and Y. Huang, “Application of Yin and Yang syndrome differentiation method in traditional Chinese Medcine syndrome differentiation on patients with stoke,” Journal of Guangzhou University of Traditional Chinese Medicine, vol. 26, no. 1, pp. 80–82, 2009. View at: Google Scholar
  20. R.-L. Cai, L. Hu, and Z.-J. Wu, “Evidence-based medicine and real world study in clinical study of acupuncture and moxibustion,” Chinese Acupuncture & Moxibustion, vol. 35, no. 9, pp. 949–952, 2015. View at: Google Scholar
  21. X.-C. Li, G.-H. Dai, and X.-C. Liu, “Real world study methods and its application in clinical efficacy evaluation of traditional Chinese medicine based on HIS,” Journal of Shandong University of Traditional Chinese Medicine, vol. 40, no. 5, pp. 415–418, 2016. View at: Google Scholar
  22. Y.-C. Li and Y.-H. Zhao, “Discuss the value and characteristics of the overall model of TCM from the change of medical model,” Lishizhen Medicine and Materia Medica Research, vol. 19, no. 2, pp. 480-481, 2008. View at: Google Scholar
  23. Y. Xie, X. Liao, and Y.-Y. Wang, “Introduce the idea of comparative effectiveness research to clinical research of Chinese medicine,” Journal of Chinese Integrative Medicine, vol. 9, no. 8, pp. 813–818, 2011. View at: Publisher Site | Google Scholar
  24. Y.-J. Wang, H.-W. Cai, J.-L. Xia et al., “Scores on propensity (I) conception and measures,” Chinese Journal of Epidemiology, vol. 31, no. 3, pp. 347-348, 2010. View at: Google Scholar
  25. Y. Han, Cost-effectiveness Analysis of Helicobacter pylori Eradication Therapy to Prevent Gastric Cancer: A Markov Model, Zhejiang University, Hangzhou, China, 2017.
  26. C.-L. Wang, D.-Z. Shi, Q. Wang, and al. el, “Assessing Chinese herbal medicines for benefiting qi and activating blood circulation in treating unstable Angina based on Markov model: a long-term clinical effectiveness exploration,” Chinese Journal of Integrated Traditional and Western Medicine, vol. 34, no. 4, pp. 406–411, 2014. View at: Google Scholar
  27. Q.-Q. Xin and F. Lu, “The application of Markov model in the economics evaluation of HIV/AIDS interventions,” Chinese Journal of AIDS & STD, vol. 18, no. 11, pp. 799–803, 2012. View at: Google Scholar
  28. M.-Q. Yang, H. Cai, and G.-W. Yu, “Digital hospital development trends and construction factors,” Medical Information, vol. 23, no. 3, pp. 555-556, 2010. View at: Google Scholar
  29. B. Gallego, A. G. Dunn, and E. Coiera, “Role of electronic health records in comparative effectiveness research,” Journal of Comparative Effectiveness Research, vol. 2, no. 6, pp. 529–532, 2013. View at: Publisher Site | Google Scholar
  30. Q. Zhan and L.-J. Wang, “Interpretation on 2016 AHA/ASA guidelines for adult stroke rehabilitation and recovery,” Journal of Neurology and Neurorehabilitation, vol. 139, no. 1, pp. 1–9, 2017. View at: Google Scholar
  31. S. K. Ostwald, S. Davis, G. Hersch, C. Kelley, and K. M. Godwin, “Evidence-based educational guidelines for stroke survivors after discharge home,” Journal of Neuroscience Nursing, vol. 40, no. 3, pp. 173–179, 2008. View at: Publisher Site | Google Scholar
  32. C.-Y. Gao, C.-H. Wu, J.-G. Zhao et al., “Chinese guidelines for diagnosis and treatment of ischemic stroke with integrated traditional Chinese and western medicine2017),” Chinese Journal of Integrated Traditional and Western Medicine, vol. 38, no. 2, pp. 136–144, 2018. View at: Google Scholar
  33. M.-G. Yuan, J.-X. Li, H. Gu et al., “Protective effect and related mechanisms of rhubarb in intracerebral hemorrhage,” Chinese Archives of Traditional Chinese Medicine, vol. 35, no. 7, pp. 1766–1768, 2017. View at: Google Scholar
  34. C.-Y. Pan, H.-J. Sheng, and S.-Y. Xue, “Clinical efficacy of Huanyu ditan Linao decoction combined with vinpocetine tablets in treatment of elderly hypertensive cerebral hemorrhage,” Chinese Archives of Traditional Chinese Medicine, vol. 37, no. 3, pp. 746–570, 2019. View at: Google Scholar
  35. M.-Z. Zhu, L.-J. Luo, L. Wu et al., “Effects of Qingnao yiyuan decoction on NSE and S100B in treating ischemic stroke,” China Journal of Traditional Chinese Medicine and Pharmacy, vol. 34, no. 2, pp. 854–857, 2019. View at: Google Scholar
  36. Y. Zheng, W.-L. Liu, G.-H. Shang, and al. el, “Meta-analysis of the therapeutic effect of acupuncture on limb motor dysfunction in stroke patients,” Chinese Journal of Rehabilitation Medicine, vol. 31, no. 2, pp. 217–221, 2016. View at: Google Scholar
  37. L.-S. Pan and J. WU, “The clinical research into stroke oculomotor nerve palsy treated with acupuncture for resuscitation in combination with decoction of astragali seu hedysari and ramuli cinnamomi with five ingredients,” Henan Traditional Chinese Medicine, vol. 8, no. 35, pp. 1762–1764, 2015. View at: Google Scholar
  38. L.-F. Chen, “The clinical study on treatment of nape needle & drug pillow to pseudobulbar palsy after cerebralinfarction,” Chinese Archives of Traditional Chinese Medicine, vol. 30, no. 6, pp. 1284–1286, 2012. View at: Google Scholar
  39. T.-J. Hu, X.-L. Wang, J. Yu, and al. el, “A meta-analysis of the therapeutic effect of nape acupuncture on post-stroke dysphagia in domestic literature,” Shanghai Journal of Acupuncture and Moxibustion, vol. 34, no. 12, pp. 1250–1254, 2015. View at: Google Scholar
  40. Y. Zhang, X.-H. Zhang, Y.-L. Xiong, and al. el, “Meta- analysis of randomized controlled trials on electro- acupuncture in the treatment of dysphagia after stroke,” Journal of Clinical Acupuncture and Moxibustion, vol. 33, no. 8, pp. 75–79, 2017. View at: Publisher Site | Google Scholar
  41. J. He, M. Zheng, C.-Q. He et al., “Systematic assessment of acupoint stimulating therapy for deglutition disorders after apoplexy,” Chinese Acupuncture & Moxibustion, vol. 29, no. 1, pp. 66–71, 2009. View at: Google Scholar
  42. C. Yu, B. Shen, and S.-W. Xu, “Systematic review of acupuncture-moxibustion for deglutition disorders after cerebral stroke,” Shanghai Journal of Acupuncture and Moxibustion, vol. 35, no. 9, pp. 1126–1129, 2016. View at: Google Scholar
  43. D.-R. Wu, Y.-F. Cai, J.-X. Cai et al., “Comparative effectiveness research on patients with acute ischemic stroke using Markov decision processes,” BMC Medical Research Methodology, vol. 12, p. 23, 2012. View at: Publisher Site | Google Scholar

Copyright © 2020 Yejing Shen et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


More related articles

 PDF Download Citation Citation
 Download other formatsMore
 Order printed copiesOrder
Views168
Downloads153
Citations

Related articles

We are committed to sharing findings related to COVID-19 as quickly as possible. We will be providing unlimited waivers of publication charges for accepted research articles as well as case reports and case series related to COVID-19. Review articles are excluded from this waiver policy. Sign up here as a reviewer to help fast-track new submissions.