Deep Learning for Mobile Crowdsourcing Techniques, Methods, and Challenges: A Survey

Liu, Bingchen; Zhong, Weiyi; Xie, Jushi; Kong, Lingzhen; Yang, Yihong; Lin, Chuang; Wang, Hao

doi:https://doi.org/10.1155/2021/6673094

Mobile Information Systems

On this page

Abstract Introduction Preliminaries Conclusions Conflicts of Interest Acknowledgments References Copyright Related Articles

Special Issue

Deep Learning in Mobile Computing: Architecture, Applications, and Future Challenges

View this Special Issue

Review Article | Open Access

Volume 2021 | Article ID 6673094 | https://doi.org/10.1155/2021/6673094

Deep Learning for Mobile Crowdsourcing Techniques, Methods, and Challenges: A Survey

Bingchen Liu,¹Weiyi Zhong,²Jushi Xie,²Lingzhen Kong,²Yihong Yang,²Chuang Lin,³and Hao Wang⁴

Academic Editor: Xiaoxian Yang

Received12 Oct 2020

Revised30 Dec 2020

Accepted13 Jan 2021

Published28 Jan 2021

Abstract

With the ever-increasing popularity of mobile computing technology and the wide adoption of outsourcing strategy in labour-intensive industrial domains, mobile crowdsourcing has recently emerged as a promising resolution for solving complex computational tasks with quick response requirements. However, the complexity of a mobile crowdsourcing task makes it hard to pursue an optimal resolution with limited computing resources, as well as various task constraints. In this situation, deep learning has provided a promising way to pursue such an optimal resolution by training a set of optimal parameters. In the past decades, many researchers have devoted themselves to this hot topic and brought various cutting-edge resolutions. In view of this, we review the current research status of deep learning for mobile crowdsourcing from the perspectives of techniques, methods, and challenges. Finally, we list a group of remaining challenges that call for an intensive study in future research.

1. Introduction

With the gradual maturity of mobile devices (e.g., mobile phones and PDAs) and ever-increasing popularity of the services-oriented computing paradigm, mobile computing technology has recently emerged as one of the key technologies to solve the complex business requirements in the mobile environment. Many companies or organizations have begun to encapsulate their value-added business processes to make them easy to be invoked and executed in the mobile environment. Typical applications include mobile payment, mobile game, mobile shopping, and mobile office. These mobile applications or businesses have significantly benefitted the daily life of human beings and attracted more and more attention from various domains.

However, a computational task in the mobile environment is often difficult to solve due to the inherent complexity of the computational task and the dynamic nature of the mobile computing environment [1]. Therefore, it is promising to use crowdsourcing ideas to pursue an optimal resolution with limited computing resources (e.g., limited computing capability of mobile devices) and various task constraints (e.g., spatial constraint and temporal constraint). However, it is a nontrivial task to find an optimal resolution for addressing such a complex mobile crowdsourcing issue.

In this situation, deep learning has provided a promising way to pursue an optimal resolution for a complex mobile computing task by training a set of optimal parameter settings [2]. Typically, through tuning the necessary parameters, deep learning can help to achieve an optimal or near-to-optimal solution so as to provide users with more satisfactory service performances or quality of experience (QoE). Therefore, deep learning has recently been regarded as a hot and promising research topic and direction in the related fields. In the past decades, many researchers have devoted themselves to this hot topic and brought various cutting-edge resolutions that bridge deep learning and mobile crowdsourcing.

In view of this hot research topic, we summarize the current research status of deep learning technologies for mobile crowdsourcing applications from various perspectives including techniques, methods, and challenges. Finally, we list a group of remaining challenges that call for intensive study in future research. We believe that this survey is helpful for beginners to know the current research directions and research challenges quickly.

The reminder of this paper is structured as follows: In Section 2, we introduce the basic concepts or preliminaries associated with deep learning and mobile crowdsourcing technologies. Mobile crowdsourcing system frameworks are investigated in Section 3. In Section 4, we summarize the techniques of mobile crowdsourcing with deep learning. In Section 5, we study the typical applications of mobile crowdsourcing, and in Section 6, we point out the mobile crowdsourcing challenges and potential solutions. Finally, we conclude the whole paper in Section 7.

2. Preliminaries

To ease the understanding of readers, we summarize all the acronyms used throughout the manuscript with Table 1.

2.1. Deep Learning (DL)

As a subclass of machine learning approaches, DL was put forward in 2006 at first [2] and is currently applied to many research fields successfully, such as E-health [3], network [4, 5], and communications [6, 7]. Deep learning models usually learn deep representations, i.e., learning multiple levels of representations and abstractions from data; the high-level abstract features of the data can be defined from the low-level features. Along with traditional machine learning and reinforcement learning paradigms, deep learning models are divided into two categories: unsupervised learning and supervised learning [8]. In supervised learning, the data used must be clearly labeled, and as a result, the output result can be classified as incorrect or correct or supervised. Convolutional neural networks (CNNs) and recurrent neural networks (RNNs) are two kinds of models with supervision or without supervision. Compared with supervised learning models, unsupervised learning can train models with unlabeled data to obtain an incorrect or correct result; restricted Boltzmann machines (RBMs) and autoencoders (AEs) are two unsupervised learning models. In the following, four models will be briefly introduced.(1)Convolutional neural networks (CNNs) A typical CNN structure contains multiple convolutional layers, fully-connected layers, and pooling layers. Feature extraction is carried out in convolutional layers, while the pooling layer can be applied to map feature and the data dimension can be reduced [9]. CNN has many different structures, such as AlexNet [10], VGGNet [11], and hyperbolic graph convolutional neural networks [12].(2)Recurrent neural networks (RNNs) RNN is a type of neural network for handling sequential data. In addition to the feed-forward neural network structure, RNN also has directed cycles. The directed cycles can allow information to circulate in the network [1]. Therefore, experimental results contain not only the input at present but also the input of previous timestamps. However, in backpropagation process, there are some issues about gradient vanishing, and as a result, only RNN is used for short-term memory in most instances. To address this issue, some improved structures were put forward, e.g., long short-term memory (LSTM) and gated recurrent unit (GRU). They are very good at processing time-series information [13]. They can model the hidden state to determine what to save in the current and previous memory. A hybrid LSTM-RNN model is presented in Ref. [14] to combine the advantages of both LSTM and RNN.(3)Restricted Boltzmann machines (RBMs) RBM model is a type of stochastic neural network consisting of a two-layer neural network, i.e., a hidden layer and a visible layer [2]. It contains m visible units and n hidden units. The observable data can be represented by visible units, while hidden units can capture correlations between observed variables. Here, there are no intralayer communications in the hidden layer or visible layer. In Ref. [15], the authors use conditional restricted Boltzmann machines to model complex consumer shopping patterns. In Ref. [16], the authors employ enhanced echo-state restricted Boltzmann machines to predict the network traffic conditions.(4)Autoencoders (AEs) AE is mainly utilized to process high-dimensional complex data. Compared to RBMs, an AE contains three layers, i.e., an input layer, an output layer, and a hidden layer. The hidden layer can describe a code representing the input. Notably, its output is a reconstruction of the input. In order to enhance the performance of AE, there are some improved structures like denoising autoencoder [17]. The authors in Ref. [18] propose an unsupervised speech representation learning approach using WaveNet autoencoders. Likewise, in Ref. [19], the authors put forward a novel transfer learning method with deep manifold regularized autoencoders.

2.2. Crowdsourcing

Crowdsourcing is a computing paradigm where human contributions need to be involved in a computing task, especially for those intrinsically much easier for human than computers. And the definition of crowdsourcing was first proposed by Jeff Howe in Ref. [20]. In Ref. [20], there was an emerging distributed problem-solving model. After that, the technology has been explored and utilized in various scenarios from both academic and industry. For traditional crowdsourcing, it involves three important components which are tasks, workers, and platforms [21]. And with the development of the Internet and sharing economy, spatial crowdsourcing which has some special requirements containing actual physical locations has gained more popularity [21]. Compared with traditional crowdsourcing, the spatial crowdsourcing consists of location-specific tasks which require people to physically be at specific locations to complete them. There are also various kinds of uses of the crowdsourcing in soft engineering. In Ref. [22], the paper has analyzed the models, motivations, and challenges of crowdsourcing in soft engineering. Other than that, it also introduces a new paradigm called mobile crowdsourcing(MCS), which emerged due to the development of many mature technologies and smart phone equipment such as built-in sensors and radio interfaces. Thus, a new paradigm has been emerged called mobile crowdsourcing (MCS). And in Ref. [23], the authors make a classification and present a review on mobile crowdsourcing as well as its difficulties and some research directions.

2.3. Mobile Crowdsourcing

Mobile crowdsourcing emerges as a new issue, which collects a cluster of workers with mobile devices to collaboratively finish hard tasks settled by a requester. It involves crowdsourcing activities on smart phones or other mobile devices. A requester can assign a broad range of complex tasks which are time consuming for workers, and workers only with a mobile device can participate in the task.

In Ref. [24], how to maximize social welfare through selecting appropriate tasks for workers and selecting appropriate workers for tasks are considered. It utilized an optimization algorithm in selecting tasks and a discrete particle swarm optimization (DPSO) algorithm for work-centric task selection.

In Ref. [25], Zhao et al. take social networks into consideration during the process of task allocation. They adopt GeoHash coding mechanism to compute the weight of friendship and protect the location privacy of workers. Thus, a solution for allocating tasks according to the friendship relationship is put forward in work [25].

In view of the different goals in different mobile crowdsourcing systems, Shen et al. [26] designed a participant selection algorithm, which was applied to different mobile crowdsourcing systems to achieve multiple goals and formulate the participant selection problem as a reinforcement learning problem.

However, recent pieces of literature overlook the fact that the internal dependency relationship exists on subtasks when they concentrate on the multipletask allocation for MCS. So Yang et al. [27] take the task internal dependencies into consideration and propose an allocation method SGTA to guarantee that the assigned works or jobs would not exceed the capability of each worker.

In Ref. [28], Miao et al. point that a basic problem of mobile crowdsourcing is when both tasks and crowd workers appear in the platforms dynamically, how to assign an appropriate set of tasks to each other is a problem. However, whether the crowd workers are reliable or not is a problem worth considering. Thus, Miao et al. investigate the quality-aware online task assignment (QAOTA) problem in mobile crowdsourcing and propose a probabilistic model to measure the quality of tasks and a hitchhiking model to characterize worker’s behavior patterns.

With the emergence of spatial mobile crowdsourcing (SMCS), which is a new class of mobile crowdsourcing, Hamrouni et al. [29] raise a photo-based SMCS framework for event reporting, which allows event report requesters to solicit photos of ongoing events and keep track of any updates. This event reporting platform helps the potential requesters to recruit ideal and best workers, select highly-relevant data from an evolving picture stream, and receive accurate responses.

In Ref. [30], according to Markov and collaborative filtering techniques, Wang et al. employed the similarity calculation, worker trajectory, dwell time, and trust to put forward a novel task recommendation algorithm. Then, according to Walrasian equilibrium, an optimal resolution was investigated to enhance the welfare contributions of MCS systems.

To balance the workers’ private information and the tasks’ availability, researchers develop advanced attribute-based encryption with preparation/online encryption and outsourced decryption technology. So it establishes win-win incentives for both requesters and workers’ participation.

In summary, the development of mobile crowdsourcing is at a full speed in various aspects such as the assignment of tasks, the considerations of context, the credibility of staff, and the protection of privacy [31]. Meanwhile, under the age of mobile devices, mobile crowdsourcing is playing an increasingly important role in human life in the global world.

3. Survey on Mobile Crowdsourcing System Framework

As an emerging research and application area, mobile crowdsourcing brings numerous issues such as task management, quality control, incentives, security, and privacy, which motivate academic researchers, industrial designers, or developers as well as policy makers to actively explore the framework of the mobile crowdsourcing system.

Generally, a comprehensive mobile crowdsourcing framework mainly consists of three roles: (1) crowdsourcers who release a computing task to the public; (2) crowdworkers who execute a computing task for a certain incentive; (3) a crowdsourcing platform where a crowdsourcing task is performed. To deal with these challenges, Wang et al. [32] propose a generic mobile crowdsourcing framework, which is composed of multiple functional modules such as task management module, mobile crowdsourcing frontend module, crowdsourcer module, and knowledge discovery module that are independent of specific applications and can accommodate multimodal data sources. Furthermore, Wang et al. [23] introduce two kinds of mobilization patterns to extend the previous mobile crowdsourcing framework.

In order to solve crowdworkers’ incentive issues in crowdsourcing applications and motivate them to join the MCS process (i.e., incentives), Thepvilojanapong et al. is proposed in Ref. [33], which is a participation-aware incentive framework and utilizes the microeconomics concept to depict the performances collected by sensors. The authors in Ref. [34] put forward a location-aware MCS solution. It describes a context-based motivating mechanism and produces a specific social network. The authors in Ref. [35] consider a motivating strategy according to an optimal tournament solution. The strategy aims at enhancing the worker’s profits. Typically, those crowdworkers often gain a reward according to the ranked order.

Privacy is a vital issue in the mobile crowdsourcing system. Large amounts of sensitive information about crowdworkers may be included by collecting data. Considering the privacy protection, Wang et al. [36] propose a framework with privacy protection which selects worker candidates statically, and then dynamically selects winners after bidding. A k for protecting worker location privacy in spatial crowdsourcing is presented in Ref. [37]. It employs a mechanism based on differential privacy and geocasting to protect worker’s privacy. Wang et al. [31] propose a framework with privacy awareness and incentive measures, which is achieved through developing advanced attribute-based encryption with preparation/online encryption and outsourced decryption technologies.

Resource constraints in mobile crowdsourcing systems refer to the shortcomings of low battery, processor, and memory of smartphones. In view of the difficulties of resource constraints, Wu et al. [38] propose a framework leveraging photo metadata instead of real images such as location, orientation, and field of view to select photos when accomplishing the task of crowdsourcing photos.

What is more, a three-layered incentive framework is put forward in [39], which includes a resource control layer, privacy protection layer, and motivation layer. Unlike the aforementioned framework, the three-layered incentive framework considers all the three issues including incentives, privacy protection, and resource constraints.

In summary, a lot of research work related to the system framework has been launched to address multiple challenges in mobile crowdsourcing, which could facilitate future research. Table 2 summarizes the main contributions of the work we study in this section.

4. Techniques of Mobile Crowdsourcing with Deep Learning

4.1. Common Techniques for Mobile Crowdsourcing

Mobile crowdsourcing can employ mobile workers to achieve certain tasks in a specific location. It is a combination of spatial crowdsourcing and smart phone technology [40]. Based on the published mode of tasks, mobile crowdsourcing can be divided into two types: serve allocation task (SAT) and worker selection task (WST) [41]. For SAT mode, according to the location information, tasks in the platform of mobile crowdsourcing can be directly allocated to works. Conversely, for WST mode, tasks can be published to all workers and workers can choose any task. Current research on mobile crowdsourcing generally includes task allocation; hence, in this paper, we only introduce the SAT mode.

In recent years, task allocation in mobile crowdsourcing has gained increasing development.

For SAT mode, works need to report their location to the mobile crowdsourcing platform, and therefore, the platform needs to track the workers’ locations and protect the workers’ spatial privacy [25]. To protect workers’ location privacy, a larger number of protection mechanisms of location-based privacy have been put forward. In Ref. [42], To et al. utilized a real-time algorithm to achieve spatial task assignment in server-assigned crowdsourcing. The mobile workers’ real locations can be protected by using this framework and the crowdsourcing success rates would be maximized. In Ref. [43], Wang et al. put forward a novel location privacy-preserving approach, which integrates k-anonymity and differential privacy-preserving. Gaussian white noise is introduced into differential privacy-preserving in this paper. In Ref. [44], the authors used economical solutions to secure sensitive location information and further put forward a novel strategy to drop tasks with high privacy risks.

However, none of the above-mentioned privacy protection task allocation mechanisms can provide personalized location protection for different protection needs. Thus, in Ref. [45], Wang et al. proposed a novel privacy-aware task allocation method, which can effectively balance task assignment and privacy protection. In Ref. [25], in order to enhance the system performances of MCS, the authors put forward a task assignment model taking social networks’ attributes into account. The mechanism of GeoHash coding was employed in the process of measuring worker relationship strength. This mechanism can effectively protect workers’ location privacy. Moreover, in Ref. [28], Miao et al. found that the majority of existing studies ignored the fact that crowd workers might be unreliable, and thus, their task assignment approaches cannot ensure the overall quality. Therefore, Miao et al. proposed a quality-aware online task assignment approach in mobile crowdsourcing. In this paper, the tasks’ quality levels can be measured by a probabilistic model, and workers’ behavior patterns were characterized by a hitchhiking model. They regarded task assignment as a problem of quality maximization problem and then designed an optimal assignment algorithm of polynomial-time complexity.

4.2. Mobile Crowdsourcing Techniques with Deep Learning

In this section, we focus on two representative applications of mobile crowdsourcing: mobile crowdcomputing and mobile crowdsensing, introducing the current research status of mobile crowdsourcing in deep learning.

Mobile crowdcomputing is used to outsource data computation tasks to mobile users. In uncertain systems of mobile crowdsourcing, for large-scale task allocation, this process may become more challenging. Therefore, in Ref. [46], Sun and Tan firstly formulated a task allocation optimization issue to achieve workers’ trustworthiness maximized and movement distance costs minimized and then presented a model of Markov decision process-based. Finally, combined deep Q-learning and trust-aware task assignment and proposed an improved deep Q-learning-based task assignment approach. At present, many approaches for reinforcement learning employ standard neural networks, such as AEs and LSTMs on. However, here, Sun and Tan designed a novel neural network that contains two sequences of fully connected layers. The proposal proposed by Sun and Tan can effectively find an optimal solution for the large-scale task allocations. However, this approach does not take task availability into account at all. In view of this, the authors defined availability as the probability that a task invocation will achieve successfully under the specified time constraints [47].

From a perspective of the crowdsourced task, in Ref. [48], Said et al. pointed out that the availability is constrained by the task spatial-temporal features, i.e., only at a particular time slot and a particular location, the crowdsourced task is available. Therefore, they introduced a novel prediction model based on deep learning technology. Firstly, the model can cluster mobile crowdsourced tasks into regions. Then, given a particular location and time slot, it uses a strategy of deep learning-based to identify whether a task is available. Indeed, determining temporal attributes of the available tasks is important for task assignment, and thus, Said et al. employed GASF and GADF to predict the time series information accurately. Both of them consist of ResNet-18 pathways, which is an 18-layer residual CNN. The experimental results can show the effectiveness of the proposed framework. In addition, in order to predict task allocation, Zhao et al. [49] combine a greedy algorithm (for large-scale application) and an optimal algorithm (based on graph partition decomposition) and further put forward a new variant with ST-RNN.

Mobile crowdsensing means that data collection and processing, such as environmental awareness and monitoring, usually require considerable technical efforts and a number of economic resources.

In order to collect high-quality image data, Hamrouni et al. [50] proposed a photo-based MCS framework. It combines with convolutional neural network in deep learning to filter redundant photo information. In order to capture the characteristics of users, Tseng et al. [51] proposed a speech recognition model named RNNLM, which uses RNN to capture the features of different users. Toman et al. [52] used the basic deep learning model DNN for crowdsourcing voice data collection, through data selection and enhancement of a large number of voices. In Ref. [53], Anagnostopoulos et al. proposed a smartphone crowdsensing system that is based on citizens’ reactions as human sensors at the edge of a municipality infrastructure, to supplement malfunctions exploiting environmental crowdsourcing location-allocation capabilities. A long short-term memory (LSTM) neural network is incorporated to learn the occurrence of such emergencies. The LSTM is able to stochastically predict future emergency situations, acting as an early warning component of the system. And Trinh et al. [54] designed an anomaly detection system based on LSTM neural networks, to deal with sequential and recurrent inputs. Figure 1 shows the development of deep learning and its applications in mobile crowdsourcing. Next, Table 3 summarizes the main contributions of the work that we study in this section.

5. Typical Applications of Mobile Crowdsourcing

5.1. Surveys on Proposed Applications (Application, Methods, and Feature)

As with traditional crowdsourcing, there are also three important components involved in the MCS system. They are crowdsourcers, crowd workers, and crowdsourcing platforms, respectively. The procedure of the operation of mobile crowdsourcing is described specifically as follows: in order to crowdsource a task, the owner who is a crowdsourcer who has to submit the task to the third party after obtaining reports collected by crowdworkers and they need to rate their quality. In the process, the crowdsourcing platform has played an intermediate role between the crowdsourcers and crowdworkers.

Generally, there are two different kinds of paradigms in applications: human intelligence and human sensors [23]. As for human intelligence, in Ref. [55], authors make a clear description about the category which utilizes human intelligence to perform tasks that are easier than machines. And in Ref. [55], authors utilize the definition “human as a sensor” to a mobile crowdsourcing system. And this idea has been utilized in various real settings such as smart transportation and environment monitoring.

In this part, we are aiming to introduce several typical MCS applications and also make some comparisons. And these applications are classified into two different categories which are direct mode and WoM mode. In terms of direct mode, in Ref. [56], authors have proposed Traffic Info which utilizes the power of the crowd to collect required data and then share information. Traffic Info can visualize the physical location of different vehicles and traffic conditions and give support to passenger feedback at the same time. In Ref. [57], authors have proposed an urban noise pollution system called EarPhone which utilizes participatory urban sensing to create an open and expensive platform in order to draw an up-to-date noise map. In Ref. [58], authors designed a system called crowdfound which aims to find lost items through utilizing mobile crowdsourcing. And the results of the system showed that the system is visible and has triggered more focus on this topic. In addition to all above-mentioned researches, there is another kind of paradigm called WoM. The 2009 DARPA Red Balloon was launched to explore internationally to identify whether the Internet and social network could help to solve a distributed geolocation problem. They need to identify all ten balloons and recognize their physical locations [58].

5.2. Structure of Typical Applications

The framework of the application is similar to the skeleton of the human body, which can well show the internal components and levels of the application. Different applications will have different frameworks, but the components of each framework are closely related while having a clear division of labor.

In the application proposed in Ref. [23], there are mainly nine modules. The MCS task management module is responsible for collecting relevant data and displaying it in a visual form; the front-end module is user-oriented and can perform recruitment-related operations; in the crowdsourcer module, tasks can be posted on the platform and related fees paid; the reward mechanism is used to motivate and reward crowdworkers and crowdsourcers by means of money or rights; in addition, the application can manage the information of participants, good behaviors can increase credibility, while bad behaviors can reduce credit scores. Noise data are unavoidable in most information. This application has a module that specializes in processing noise data to provide valuable information; user privacy is the most important, and privacy protection module can prevent user privacy from being disclosed; it can also collect external data and enrich the information of the platform; finally, you can manage all parts as a whole.

Phuttharak and Loke analyzed in Ref. [59] and proposed that crowdsourcing application frameworks are mainly divided into centralized and decentralized frameworks. The centralized architecture is divided into mobile sensing and integration layer, connectivity and network layer, crowd processing layer, and enduser layer. In contrast, the distributed architecture is a parallel architecture, and different levels of structure make equal contributions to the application, and the performance becomes better while the complexity becomes higher. In Ref. [60], Pu et al. proposed a comprehensive framework and the task layer is mainly used to generate crowdsourcing tasks; for crowdworkers, they use the worker arrival layer; in order to evaluate the work ability, use the work ability layer to evaluate the completion of the task, and finally, there is a dedicated level for overall coordination. Kong et al. divided the crowdsourcing architecture into three entities, namely, crowdsourcer, platform, and worker. The platform allocates tasks requested by crowdsourcers to interested workers and then feeds back the results of the tasks to the crowdsourcers [61].

In the description in Ref. [62], the task management component and communication network are the core of the crowdsourcing application framework, and the optimization algorithm of crowd selection is added to it, which effectively solves the real-time and complexity problems. The authors introduced a deep learning framework in Ref. [63], which can make predictions based on historical usage data, and this framework consists of two parts. Firstly, the spatial-temporal data of crowdsourcing services are preprocessed by K-means algorithm, and the characteristics of historical spatiotemporal data are trained by the deep learning method to predict the availability of crowdsourcing services at relevant locations and times. Then, an hour is constructed to predict the duration of service availability. Hu et al. proposed a framework named CROWDSERVICE in [64], which is based on genetic algorithms that comprehensively consider time and cost factors to provide almost the best crowdsourcing workers’ choices. It connects service calls with crowdsourcing tasks based on scalable architecture technology. In short, many scholars have proposed an effective structure to deal with various challenges faced by mobile crowdsourcing services. Table 4 summarizes the main contributions of the work we study in this section.

6. Mobile Crowdsourcing Challenges and Potential Solutions

The current research on mobile crowdsourcing has provided some effective solutions to the key problems of mobile crowdsourcing, which has triggered the explosive growth of mobile crowdsourcing applications. However, the development of mobile crowdsourcing still has a long way to go, and there are still challenges. In this section, we will introduce challenges of mobile crowdsourcing and corresponding solutions, which will promote continued progress and continuous development in this emerging field. The main challenges include security threats, privacy leakage, and trust crisis.

6.1. Security Threats

Mobile crowdsourcing requires human participation. When people accept tasks, the platform will collect user data. In addition, the platform will also collect sensitive information from mobile devices. Therefore, the platform needs to protect the collected data from being disclosed to unauthorized parties and maintain the normal operation of the platform [65]. But in fact, the platform may be subject to malicious attacks, which cannot protect the data.

Security and sensitive information. Task crowdsourcing also contains security risks. When assigning tasks to users, users are dynamic so they cannot be determined in advance. At this time, protecting data and information is particularly difficult. The preference of the task recipient, the energy consumption of the equipment [66], and the power consumption of the equipment [67] cause users to join and leave dynamically, and the network topology is constantly changing, which increases the difficulty of security protection. Wu et al. [68] provide a privacy-aware task allocation and data aggregation scheme using bilinear pairing and homomorphic encryption to achieve the ideal security goal. Tao et al. [69] proposed an anonymous identity authentication mechanism that has met the needs of user security.

6.2. Privacy Leakage

Common privacy threats include leaking the user’s location, leaking the user’s activities, and the user's lack of privacy protection awareness [70]. When the user accepts and executes the task, it is easy to leak the user's location privacy and activity content. For privacy leakage, more complete encryption technology or anonymity technology can be used to protect the information transmission between the user and the server. Even incentive strategies are useful for providing sufficient rewards to workers; the users are still unwilling to contribute to mobile crowdsourcing if sensitive private information cannot be fully guaranteed. Some users are less aware of privacy protection, who do not care much about their personal private information, giving attackers an opportunity. Tang et al. [71] proposed a privacy protection scheme that considers both requesters and workers’ mutual incentives, using online encryption and outsourcing decryption technology to provide privacy protection in mobile crowdsourcing. Wan and Zhang [72] proposed a signature encryption algorithm to protect the privacy of crowdsourcing by using pseudonyms instead of real identities.

6.3. Trust Threats

Trust threats mainly include data trust, work trust, and service provider trust [40]. Due to the inherent openness of mobile crowdsourcing, some workers may launch attacks for their benefit. And different task recipients have differences in their preferences, device types, credibility, and communication capabilities. When a staff member's trust level is low, mobile device capabilities are poor, and communication capabilities are insufficient, the data collected are also data with poor reliability. Zhu et al. [73] combined the blockchain consensus algorithm to track malicious behavior and improved the trust relationship. In order to cope with the crisis of trust, more effective incentives can be adapted to enable people to participate in mobile crowdsourcing tasks, in addition to adopting safer methods to increase trust between workers and platforms.

Although the research on solving the problem of mobile crowdsourcing has never stopped, the problem has not been solved perfectly. We still have to make continuous research, so that mobile crowdsourcing can benefit society earlier.

7. Conclusions

The gradual maturity of mobile devices and ever-increasing popularity of the services-oriented computing paradigm has raised the mobile computing technology. Due to the complexity of mobile computing tasks, mobile crowdsourcing has recently emerged as a promising resolution for solving complex mobile computational tasks with quick response requirements. However, the complexity of a mobile crowdsourcing task makes it hard to pursue an optimal resolution with limited computing resources and various task constraints. In this situation, deep learning has provided a promising way to pursue such an optimal resolution by training a set of optimal parameter settings.

Currently, there are several surveys for mobile crowdsourcing [40, 74, 75] or deep learning (e.g., deep learning for generic object detection [76], deep learning for agriculture [77], and deep learning for sentiment analysis [78]). However, current research work lacks a comprehensive survey on deep learning for mobile crowdsourcing. Therefore, in this paper, we review the current research status of deep learning for mobile crowdsourcing from the perspectives of techniques, methods, and challenges. This survey is helpful for beginners to know the current research directions and research challenges quickly.

However, current research of deep learning for mobile crowdsourcing is still confronted with several shortcomings or drawbacks such as a lack of a standard dataset or benchmark. In future work, we will further focus on comparisons and evaluation of standard datasets.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This research was supported by the Fundamental Research Funds for the Central Universities under Grant no. 30919011282 and the National Science Foundation of China under Grant nos. 61702277, 61772283, 61672276, and 61872219. This work was also supported the Shenzhen Basic Research under Grant nos. JCYJ20170413152804728 and JCYJ20180507182508857.

References

H. Gao, C. Liu, Y. Li, and X. Yang, “V2VR: reliable hybrid-network-oriented V2V data transmission and routing considering RSUs and connectivity probability,” IEEE Transactions on Intelligent Transportation Systems, vol. 99, pp. 1–14, 2020.
View at: Publisher Site | Google Scholar
X. Du, Y. Cai, S. Wang, and L. Zhang, “Overview of deep learning,” in Proceedings of the 31st Youth Academic Annual Conference of Chinese Association of Automation, pp. 159–164, Wuhan, China, November 2016.
View at: Google Scholar
X. Liu, L. Faes, A. U. Kale et al., “A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis,” The Lancet Digital Health, vol. 1, no. 6, pp. E271–E297, 2019.
View at: Publisher Site | Google Scholar
G. Aceto, D. Ciuonzo, A. Montieri, and A. Pescapé, “Mobile encrypted traffic classification using deep learning: experimental evaluation, lessons learned, and challenges,” IEEE Transactions on Network and Service Management, vol. 16, no. 2, pp. 445–458, 2019.
View at: Publisher Site | Google Scholar
X. Ma, H. Gao, H. Xu, and M. Bian, “An IoT-based task scheduling optimization scheme considering the deadline and cost-aware scientific workflow for cloud computing,” EURASIP Journal on Wireless Communications and Networking, vol. 2019, no. 1, 2019.
View at: Google Scholar
H. Huang, S. Guo, G. Gui et al., “Deep learning for physical-layer 5G wireless techniques: opportunities, challenges and solutions,” IEEE Wireless Communications, vol. 27, no. 1, pp. 214–222, 2019.
View at: Google Scholar
Z. Qin, H. Ye, G. Y. Li, and B.-H. F. Juang, “Deep learning in physical layer communications,” IEEE Wireless Communications, vol. 26, no. 2, pp. 93–99, 2019.
View at: Publisher Site | Google Scholar
W. G. Hatcher and W. Yu, “A survey of deep learning: platforms, applications and emerging research trends,” IEEE Access, vol. 6, pp. 24411–24432, 2018.
View at: Publisher Site | Google Scholar
I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, MIT Press, Cambridge, MA, USA, 2016.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Neural Information and Processing Systems, vol. 60, no. 6, pp. 1097–1105, 2012.
View at: Publisher Site | Google Scholar
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2014, https://arxiv.org/abs/1409.1556.
View at: Google Scholar
I. Chami, Z. Ying, C. Ré, and J. Leskovec, “Hyperbolic graph convolutional neural networks,” Advances in Neural Information Processing Systems, vol. 32, pp. 4869–4880, 2019.
View at: Google Scholar
H. Gao, L. Kuang, Y. Yin, B. Guo, and K. Dou, Mining Consuming Behaviors with Temporal Evolution for Personalized Recommendation in Mobile Marketing Apps, ACM/Springer Mobile Network and Applications, Berlin, Germany, 2020.
F. Wang, Z. Xuan, Z. Zhen, K. Li, T. Wang, and M. Shi, “A day-ahead PV power forecasting method based on LSTM-RNN model and time correlation modification under partial daily pattern prediction framework,” Energy Conversion and Management, vol. 212, Article ID 112766, 2020.
View at: Publisher Site | Google Scholar
F. Xia, R. May, and H. M. Jerrold, “Using conditional restricted Boltzmann machines to model complex consumer shopping patterns,” Marketing Science, vol. 38, no. 4, pp. 711–727, 2019.
View at: Publisher Site | Google Scholar
X. Sun, S. Ma, Y. Li et al., “Enhanced echo-state restricted Boltzmann machines for network traffic prediction,” IEEE Internet of Things Journal, vol. 7, no. 2, pp. 1287–1297, 2019.
View at: Google Scholar
P. Vincent, H. Larochelle, Y. Bengio, and P. Manzagol, “Extracting and composing robust features with denoising autoencoers,” in Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103, Helsinki, Finland, July 2008.
View at: Google Scholar
J. Chorowski, R. J. Weiss, S. Bengio, and A. van den Oord, “Unsupervised speech representation learning using wavenet autoencoders,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 27, no. 12, pp. 2041–2053, 2019.
View at: Publisher Site | Google Scholar
Y. Zhu, X. Wu, P. Li, Y. Zhang, and X. Hu, “Transfer learning with deep manifold regularized auto-encoders,” Neurocomputing, vol. 369, pp. 145–154, 2019.
View at: Publisher Site | Google Scholar
J. Howe, “The rise of crowdsourcing,” Wired, vol. 14, no. 6, pp. 1–4, 2006.
View at: Google Scholar
Y. Tong, Z. Zhou, Y. Zeng, L. Chen, and C. Shahabi, “Spatial crowdsourcing: a survey,” The VLDB Journal, vol. 29, no. 1, pp. 217–250, 2020.
View at: Publisher Site | Google Scholar
T. D. LaToza, “Crowdsourcing in software engineering: models, motivations, and challenges,” IEEE Software, vol. 33, no. 1, pp. 74–80, 2015.
View at: Google Scholar
Y. Wang, X. Jia, Q. Jin, and J. Ma, “Mobile crowdsourcing: framework, challenges, and solutions,” Concurrency and Computation: Practice and Experience, vol. 29, no. 3, p. e3789, 2017.
View at: Publisher Site | Google Scholar
Y. Wang, Z. Cai, Z.-H. Zhan, Y.-J. Gong, and X. Tong, “An optimization and auction-based incentive mechanism to maximize social welfare for mobile crowdsourcing,” IEEE Transactions on Computational Social Systems, vol. 6, no. 3, pp. 414–429, 2019.
View at: Publisher Site | Google Scholar
B. Zhao, Y. Wang, and Y. Li, “Task allocation model based on worker friend relationship for mobile crowdsourcing,” Sensors, vol. 921, 2019.
View at: Google Scholar
W. Shen, X. He, and C. Zhang, “Auxiliary-task based deep reinforcement learning for participant selection problem in mobile crowdsourcing,” 2020, https://arxiv.org/abs/2008.11087.
View at: Google Scholar
C. Yang, Z. Yu, and Y. Liu, “Dynamic allocation for complex mobile crowdsourcing task with internal dependencies,” IEEE, pp. 818–825, 2019.
View at: Google Scholar
X. Miao, Y. Kang, and Q. Ma, “Quality-aware online task assignment in mobile crowdsourcing,” ACM Transactions on Sensor Networks (TOSN), vol. 1–21, 2020.
View at: Google Scholar
A. Hamrouni, H. Ghazzai, M. Frikha, and Y. Massoud, “A spatial mobile crowdsourcing framework for event reporting,” IEEE Transactions on Computational Social Systems, vol. 7, no. 2, pp. 477–491, 2020.
View at: Publisher Site | Google Scholar
Y. Wang, Z. Cai, and Z. Zhan, “Walrasian equilibrium-based multiobjective optimization for task allocation in mobile crowdsourcing,” IEEE Transactions on Computational Social Systems, vol. 99, 2020.
View at: Publisher Site | Google Scholar
H. Gao, W. Huang, and Y. Duan, “The cloud-edge based dynamic reconfiguration to service workflow for mobile ecommerce environments: a QoS prediction perspective,” ACM Transactions on Internet Technology, vol. 21, no. 1, p. 6, 2020.
View at: Google Scholar
Y. Wang, X. Jia, Q. Jin, and J. Ma, “Mobile crowdsourcing: architecture, applications, and challenges,” in Proceedings of the IEEE 15th International Conference on scalable computing and communications and its associated Workshops (UIC-ATC-ScalCom), pp. 1127–1132, IEEE, Beijing, China, 2015.
View at: Google Scholar
N. Thepvilojanapong, K. Zhang, T. Tsujimori, O. Yoshikatsu, Y. Zhao, and T. Yoshito, “Participation-aware incentive for active crowd sensing,” in Proceedings of the 2013 IEEE 10th International Conference on High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing, pp. 2127–2134, IEEE, Zhangjiajie, China, 2013.
View at: Google Scholar
M. A. Rahman and M. S. Hossain, “A location-based mobile crowdsensing framework supporting a massive ad hoc social network environment,” IEEE Communications Magazine, vol. 55, no. 3, pp. 76–85, 2017.
View at: Publisher Site | Google Scholar
Y. Zhang, C. Jiang, L. Song, M. Pan, Z. Dawy, and Z. Han, “Incentive mechanism for mobile crowdsourcing using an optimized tournament model,” IEEE Journal on Selected Areas in Communications, vol. 35, no. 4, pp. 880–892, 2017.
View at: Publisher Site | Google Scholar
Y. Wang, Z. Cai, G. Yin, Y. Gao, X. Tong, and G. Wu, “An incentive mechanism with privacy protection in mobile crowdsourcing systems,” Computer Networks, vol. 102, pp. 157–171, 2016.
View at: Publisher Site | Google Scholar
H. To, G. Ghinita, and C. Shahabi, “A framework for protecting worker location privacy in spatial crowdsourcing,” Proceedings of the VLDB Endowment, vol. 7, no. 10, pp. 919–930, 2014.
View at: Publisher Site | Google Scholar
Y. Wu, Y. Wang, and G. Cao, “Photo crowdsourcing for area coverage in resource constrained environments,” in Proceedings of the IEEE INFOCOM 2017-IEEE Conference on Computer Communications, pp. 1–9, IEEE, Atlanta, GA, USA, May 2017.
View at: Google Scholar
L. Islam, M. Islam, and M. Uddin, “Three-layered incentive framework for mobile crowdsourcing,” in Proceedings of the 2018 10th international Conference on electrical and computer engineering (ICECE), pp. 153–156, IEEE, Dhaka, Bangladesh, December 2018.
View at: Google Scholar
W. Feng, Z. Yan, H. Zhang, K. Zeng, Y. Xiao, and Y. T. Hou, “A survey on security, privacy, and trust in mobile crowdsourcing,” IEEE Internet of Things Journal, vol. 5, no. 4, pp. 2971–2992, 2017.
View at: Publisher Site | Google Scholar
L. Kazemi and C. Shahabi, “GeoCrows: enabling query answering with apstial crowdsourcing,” in Proceedings of the International Conference on Advances in Geographic Information Systems, pp. 189–198, Los Angeles, CA, USA, 2012.
View at: Google Scholar
H. To, L. Tran, L. Fan, and C. Shahabi, “A real-time framework for task assignment in hyperlocal spatial crowdsourcing,” ACM Transactions on Intelligent Systems and Technology, vol. 9, no. 3, pp. 1–26, 2018.
View at: Publisher Site | Google Scholar
Y. Wang, Z. Cai, Z. Chi, X. Tong, and L. Li, A Differentially K-Anonymity-Based Location Privacy-Preserving for Mobile Crowdsourcing Systems, Elsevier, Amsterdam, Netherlands, 2018.
View at: Publisher Site
B. Liu, W. Zhou, T. Zhu, H. Zhou, and X. Lin, “Invisible hand: a privacy preserving mobile crowd sensing framework based on economic models,” IEEE Transactions on Vehicular Technology, vol. 99, pp. 4410–4423, 2017.
View at: Publisher Site | Google Scholar
Z. Wang, J. Hu, R. Lv et al., “Personalized privacy-preserving task allocation for mobile crowdsensing,” IEEE Transactions on Mobile Computing, vol. 18, no. 6, pp. 1330–1341, 2019.
View at: Publisher Site | Google Scholar
Y. Sun and W. Tan, “A trust-aware task allocation method using deep q-learning for uncertain mobile crowdsourcing,” Human-centric Computing and Information Sciences, vol. 9, no. 1, 2019.
View at: Publisher Site | Google Scholar
M. Silic, G. Delac, I. Krka, and S. Srbljic, “Scalable and accurate prediction of availability of atomic web services,” IEEE Transactions on Services Computing, vol. 7, no. 2, pp. 252–264, 2014.
View at: Publisher Site | Google Scholar
A. B. Said, A. Erradi, A. Ghari Neiat, and A. Bouguettaya, “A deep learning spatiotemporal prediction framework for mobile crowdsourced services,” Mobile Networks and Applications, vol. 24, no. 6, 2018.
View at: Publisher Site | Google Scholar
Y. Zhao, K. Zheng, Y. Cui, H. Su, F. Zhu, and X. Zhou, “Predictive task assignment in spatial crowdsourcing: a data-driven approach,” in Proceedings of the IEEE International Conference on Data Engineering, pp. 13–24, Dallas, TX, USA, April 2020.
View at: Google Scholar
A. Hamrouni, H. Ghazzai, M. Frikha, and Y. Massoud, “A photo-based mobile crowdsourcing framework for event reporting,” in Proceedings of the IEEE 62nd International Midwest Symposium on Circuits and Systems, pp. 198–202, Dallas, TX, USA, August 2019.
View at: Google Scholar
B. Tseng, H. Lee, and L. Lee, “Personalizing universal recurrent neural network language model with user characteristic features by social network crowdsourcing,” IEEE Workshop on Automatic Speech Recognition and Understanding, pp. 84–91, 2015.
View at: Google Scholar
M. Toman, G. Meltzner, and R. Patel, “Data requirements, selection and augmentation for DNN-based speech synthesis from crowdsourced data,” in Proceedings of the 19th Annual Conference of the International Speech Communication Association, pp. 2878–2882, Hyderabad, India, September 2018.
View at: Google Scholar
T. Anagnostopoulos, T. Xanthopoulos, and Y. Psaromiligkos, “A smartphone crowdsensing system enabling environmental crowdsourcing for municipality resource allocation with LSTM stochastic prediction,” Sensors, vol. 20, no. 14, p. 3966, 2020.
View at: Publisher Site | Google Scholar
H. Trinh, L. Giupponi, and P. Dini, “Urban anomaly detection by processing mobile traffic traces with LSTM neural networks,” in Proceedings of the IEEE International Conference on Sensing, Communication, and Networking, pp. 1–8, Boston, MA, USA, June 2019.
View at: Google Scholar
Y. Zheng, L. Capra, O. Wolfson, and H. Yang, “Urban computing: concepts, methodologies, and applications,” ACM Transaction on Intelligent Systems and Technology, vol. 5, no. 3, p. 38, 2014.
View at: Publisher Site | Google Scholar
K. Farkas, A. Z. Nagy, T. Tomas, R. Szab, and F. Károly, “Participatory sensing based real-time public transport information service,” in Proceedings of the 2014 IEEE International Conference on Pervasive Computing and Communication Workshops, IEEE, Budapest, Hungary, March 2014.
View at: Google Scholar
R. K. Rana, C. T. Chou, S. S. Kanhere, N. Bulusu, and W. Hu, “Ear-phone: an end-to-end participatory urban noise mapping system,” in Proceedings of the 9th ACM/IEEE International Conference on Information Processing in Sensor Networks, Stockholm, Sweden, April 2010.
View at: Google Scholar
J. C. Tang, M. Cebrian, N. A. Giacobe, H.-W. Kim, T. Kim, and D. B. Wickert, “Reflecting on the DARPA red balloon challenge,” Communications of the ACM, vol. 54, no. 4, pp. 78–85, 2011.
View at: Publisher Site | Google Scholar
J. Phuttharak and S. W. Loke, “A review of mobile crowdsourcing architectures and challenges: toward crowd-empowered Internet-of-Things,” IEEE Access, vol. 99, 2018.
View at: Publisher Site | Google Scholar
L. Pu, X. Chen, J. Xu, and X. Fu, “Crowd foraging: a QoS-oriented self-organized mobile crowdsourcing framework over opportunistic networks,” IEEE Journal on Selected Areas in Communications, vol. 35, no. 4, pp. 848–862, 2017.
View at: Publisher Site | Google Scholar
X. Kong, X. Liu, B. Jedari, M. Li, L. Wan, and F. Xia, “Mobile crowdsourcing in smart cities: technologies, applications, and future challenges,” IEEE Internet of Things Journal, vol. 6, no. 5, pp. 8095–8113, 2019.
View at: Publisher Site | Google Scholar
A. Korthaus and W. Dai, “Opportunities and challenges for mobile crowdsourcing-conceptualisation of a platform architecture,” International Journal of High Performance Computing and Networking, vol. 8, no. 1, pp. 16–27, 2015.
View at: Publisher Site | Google Scholar
A. B. Said, A. Erradi, and A. G. Neiat, “A deep learning spatiotemporal prediction framework for mobile crowdsourced services,” Mobile Networks and Applications, vol. 24, no. 3, pp. 1120–1133, 2019.
View at: Google Scholar
J. Hu, H. Lin, X. Guo, and J. Yang, “DTCS: an integrated strategy for enhancing data trustworthiness in mobile crowdsourcing,” IEEE Internet of Things Journal, vol. 5, no. 6, pp. 4663–4671, 2018.
View at: Publisher Site | Google Scholar
K. Yang, K. Zhang, J. Ren, and X. Shen, “Security and privacy in mobile crowdsourcing networks: challenges and opportunities,” IEEE Communications Magazine, vol. 53, no. 8, pp. 75–81, 2015.
View at: Publisher Site | Google Scholar
A. Faggiani, E. Gregori, L. Lenzini, and V. Luconi, “Smartphone-based crowdsourcing for network monitoring: opportunities, challenges, and a case study,” IEEE Communications Magazine, vol. 51, no. 1, pp. 106–113, 2014.
View at: Publisher Site | Google Scholar
G. Chatzimilioudis, A. Konstantinidis, C. Laoudias, and D. Zeinalipour-Yazti, “Crowdsourcing with smartphones,” IEEE Internet Computing, vol. 16, no. 5, pp. 36–44, 2012.
View at: Publisher Site | Google Scholar
H. Wu, L. Wang, and G. Xue, “Privacy-aware task allocation and data aggregation in fog-assisted spatial crowdsourcing,” IEEE Transactions on Network Science and Engineering, vol. 7, no. 1, pp. 589–602, 2019.
View at: Publisher Site | Google Scholar
D. Tao, P. Ma, and M. S. Obaidat, “Anonymous identity authentication mechanism for hybrid architecture in mobile crowd sensing networks,” International Journal of Communication Systems, vol. 32, no. 5, p. e4099, 2019.
View at: Publisher Site | Google Scholar
Y. Wang, Y. Huang, and C. Louis, “Respecting user privacy in mobile crowdsourcing,” Science, vol. 50, 2013.
View at: Google Scholar
W. Tang, K. Zhang, J. Ren, Y. Zhang, and X. Shen, “Privacy-preserving task recommendation with win-win incentives for mobile crowdsourcing,” Information Sciences, vol. 527, pp. 477–492, 2020.
View at: Publisher Site | Google Scholar
C. Wan and J. Zhang, “SCRPM: securing crowdsourcing-based road pavement monitoring system with location privacy,” Wireless Networks, vol. 26, no. 2, pp. 1139–1149, 2020.
View at: Publisher Site | Google Scholar
X. Zhu, Y. Li, and L. Fang, “An improved proof-of-trust consensus algorithm for credible crowdsourcing blockchain services,” IEEE Access, vol. 8, pp. 102177–102187, 2020.
View at: Publisher Site | Google Scholar
J. Phuttharak and S. W. Loke, “A review of mobile crowdsourcing architectures and challenges: toward crowd-empowered Internet-of-Things,” IEEE Access, vol. 7, pp. 304–324, 2018.
View at: Google Scholar
S. R. B. Gummidi, X. Xie, and T. B. Pedersen, “A survey of spatial crowdsourcing,” ACM Transactions on Database Systems, vol. 44, no. 2, pp. 1–46, 2019.
View at: Publisher Site | Google Scholar
L. Liu, W. Ouyang, X. Wang et al., “Deep learning for generic object detection: a survey,” International Journal of Computer Vision, vol. 128, no. 2, pp. 261–318, 2020.
View at: Publisher Site | Google Scholar
A. Kamilaris and F. X. Prenafeta-Boldú, “Deep learning in agriculture: a survey,” Computers and Electronics in Agriculture, vol. 147, pp. 70–90, 2018.
View at: Publisher Site | Google Scholar
L. Zhang, S. Wang, and B. Liu, “Deep learning for sentiment analysis: a survey,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 8, no. 4, p. e1253, 2018.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2021 Bingchen Liu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1458

Downloads

877

Citations