Understanding Service Providers’ Competency in Knowledge-Intensive Crowdsourcing Platforms: An LDA Approach

Yang, Biyu; Wang, Xu; Ding, Zhuofei

doi:https://doi.org/10.1155/2021/6653410

Complexity

On this page

Abstract Introduction Related Works Results Discussion Conclusion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Special Issue

Machine Learning Applications in Complex Economics and Financial Networks

View this Special Issue

Research Article | Open Access

Volume 2021 | Article ID 6653410 | https://doi.org/10.1155/2021/6653410

Understanding Service Providers’ Competency in Knowledge-Intensive Crowdsourcing Platforms: An LDA Approach

Biyu Yang,¹Xu Wang,^1,2and Zhuofei Ding³

Academic Editor: Liang Zhao

Received12 Nov 2020

Revised05 Feb 2021

Accepted08 Jul 2021

Published17 Jul 2021

Abstract

Knowledge-intensive crowdsourcing (KIC) is becoming one of the most promising domains of crowdsourcing by leveraging human intelligence and building a large labor-intensive service network. In this network, the service providers (SPs) constitute the backbone of the KIC platform and play an important role in connecting the platform and service requesters. The SPs are a group of distributed crowds with a complex composition and high level of uncertainty, resulting in great challenges in service quality and platform management. Understanding the SPs’ competency is an effective way for the platform to manage them. Therefore, we attempt to connect the competency analysis to the environment of KIC to investigate and identify the criteria of SPs’ competency (i.e., the competency factors and dimensions required for being competent for the SPs’ business). To this end, we leverage the Latent Dirichlet Allocation (LDA) model to explore and extract hidden competency dimensions from online interview records. We then introduce the competency theory to identify and label the competency factors and dimensions and construct the three-level KSAT competency model, which presents a comprehensive vision of the SPs’ performance standards in the context of KIC. Given the competency criteria in the KSAT competency model, we use the Best-Worst Method (BWM) to determine their weights, which reflect their importance when evaluating the SPs’ competency from the platforms’ perspective. The results show that skill and knowledge are the two most important competency factors, and customer relationship management and communication ability are the two most valuable competency dimensions when evaluating the SPs’ competency. Furthermore, the KSAT competency model can be applied to analyze the competency of individuals or organizations in many other industries as well.

1. Introduction

As facilitated by highly developed information technologies, knowledge-intensive crowdsourcing (KIC) taps into the creative and innovation fields (e.g., designers), changing the traditional way of how business is conducted [1]. For example, in the design industry, penetration of technology, especially Internet and smartphone apps, has changed how design is practiced, produced, and traded. In terms of the critical role, it plays in today’s knowledge-based economy, KIC is considered to be one of the most promising domains of crowdsourcing in the future [2, 3].

The KIC platform operates as a two- or multisided market, meaning that each side of the market derives externalities from the participation of the respective other side, which is called the network effects [4, 5]. Due to this, the KIC platform experiences tremendous growth in the user base and becomes a large complex network. In this network, a large number of open and crowdsourcing service providers (SPs) constitute the backbone of the KIC platform. Also, based on the resource-based view, the network effects allow the SPs to be converted into critical resources, creating a continuous competitive edge for the platform [6], and bring certain benefits. As a vital connection between the platform and service requesters, they assist the KIC platform in running a wide range of business and services catering to numerous end customers of different types and attain business success in their areas of professionals. In this process, they can meet the service requesters’ demand. Especially for those high-quality and competent SPs, they can perform knowledge-based tasks well and generate satisfying outcomes. For the SPs, delivering services with committed quality and performance to achieve sustainable business growth is a concrete demonstration of competency, which enhances service requesters’ satisfaction and trust toward the platform [7]. In turn, service requesters who previously had a good shopping or interactive experience are more likely to engage with the platform on an ongoing basis. By delivering anticipated services to their customers on the platform, the SPs enable the platform to attract more customers, generate more traffic, and capture more market share, so that the platform can remain competitive in the market [8, 9].

Despite the certain benefits the SPs have brought to the platform and service requesters, some potential risks in terms of service quality and platform health may arise from the crowdsourcing activities. For instance, some SPs may prove precarious, due to the fact that the supplier database consists of complex and previously unknown crowds [10]. There may also be potential hazards such as cheating, manipulating task outputs, or extracting sensitive information from crowdsourcing systems [10, 11]. Even so, the KIC platform still weakens the input controls and simplifies the registration procedures to attract SPs. Additionally, the relationship between the platform and the SPs is not the classic principal-agent relationship (i.e., the platform does not hire these SPs), which offers SPs the flexibility of their work time and schedules [5]. Accordingly, concerns about the uncertainties of the operation process, including SPs’ availability, service awareness, financial and intellectual property, and privacy risks, are growing, which may, in turn, jeopardize the platform’s reputation and affect its healthy operation [10, 11]. Under this circumstance, it is imperative for the platform to take some necessary actions to identify those who are not competent for the knowledge-intensive tasks.

Regarding the above benefits and risks, it is the responsibility of the platform, as an indispensable manager and regulator of the KIC activities, to manage the SPs on the platform. To this end, conducting competency analysis and understanding the SPs’ competency is an effective way for the platform to fulfill the responsibilities and to have a clear vision of the SPs’ performance standards and expectations [12]. On the one hand, the competency analysis offers a solution for the platform to construct a competency evaluation framework for the SPs [12, 13] and differentiate high performance from middle and low performance [14]. On the other hand, the competency analysis helps the SPs to achieve certain goals under the standardized framework developed by the platform. Therefore, we attempt to associate the competency analysis with the context of KIC.

Practically, we have noted that some KIC platforms have taken measures to facilitate the SPs’ performance. According to our survey, some platforms in China will release a list of leaders online in terms of different criteria, such as bidding price and daily or total sales. Also, the reputation system of the SPs about their service quality, speed, and attitude is visible on their homepage. However, it is biased to consider only reputation, total sales volume, or platform score. Some other factors that may determine the SPs’ competency are not taken into consideration, such as entrepreneurial experience [15], communication ability [16], and innovativeness [17, 18]. It shows that not enough attention has been paid by practitioners to those aspects, and, as yet, a comprehensive understanding of the SPs’ competency is still lacking.

Therefore, both the theoretical backgrounds and the practical applications motivate us to investigate the SPs’ competency (i.e., the detailed components of competency that are required for being competent for the SPs’ business in KIC). To this end, this paper aims to address the following question:

What is about the SPs’ competency in the context of KIC and to which competency criteria should the practitioners pay more attention when assessing the SPs’ competency and performance?

To understand the SPs’ competency, we aim to explore and identify competency criteria (i.e., the competency factors and dimensions). We analyze online interview records posted by several KIC platforms in China, which includes successful SPs’ opinions about the qualifications to obtain sustainable business growth and conduct their career well. We first crawl these online interview records and then extract and identify the competency dimensions leveraging the Latent Dirichlet Allocation (LDA) model. We then construct the KSAT competency model for the KIC platform to evaluate and manage the SPs. Further, questionnaires are developed to collect experts’ opinions and the Best-Worst Method (BWM) is applied to determine the weights and priorities of the competency criteria in the proposed KSAT competency model. Therefore, the main contributions of this research are summarized as follows:(1)This study expands the outreach of the competency theory by introducing it to the KIC environment. To the best of our knowledge, it is the first time that the competency theory is applied in the field of KIC.(2)This study presents a novel research framework enabling the KIC platform to transform the online textual information into useful knowledge about SPs’ competency. The incorporation of text mining and decision-making methods offers insights for researchers and practitioners to understand the hidden content behind the large collection of unstructured textual information on the KIC platform.(3)The proposed KSAT competency model provides a rich set of indicators and variables that allow both researchers and practitioners to flexibly design, build, and formulate specific evaluation framework in terms of different requirements and goals.

In the following sections, the related works are presented in Section 2. The methods used in this research are detailed in Section 3, followed by the data analysis and results in Section 4. We then discuss the KSAT competency model and criteria importance in Section 5. Finally, the conclusion is drawn in Section 6.

2.1. Knowledge-Intensive Crowdsourcing

The collaboration landscape has changed remarkably over recent decades where users can shape the Web and availability of information via highly developed information technology. Traditionally, collaborations were mostly concentrated within organizations’ internal function departments [19] and also limited to messaging tools such as e-mail. However, it is nowadays possible to leverage the knowledge and intelligence of an immense number of people across geographic and organizational boundaries through crowdsourcing [20, 21]. Since its appearance, crowdsourcing has attained much success and has become a widely commercial phenomenon [22]. Meanwhile, the KIC is considered to be one of the most promising domains of crowdsourcing in the future, and in terms of the critical role, it plays in today’s knowledge-based economy [3]. It refers to crowdsource human intelligence- and expertise-related tasks, such as question answering, image annotation, product development, website design, logo design, and software development [23], which cannot be performed by computers, to a crowd of people in an open call [24, 25]. In recent two decades, many KIC companies like Threadless, InnoCentive, Amazon Mechanical Turk, and some Chinese crowdsourcing platforms, such as ZBJ.COM, epwk.com, and 680.com, are established to support posting and performing various KIC tasks [23].

The KIC intermediaries are typical two- or multisided markets, where the network effects will attract an increasing number of participants from both supply and demand sides, to join the platform for valuable advantages [26]. Oosterman et al. [27] suggested that the KIC has the advantage of low cost for service requesters; therefore, it allows cost-saving and efficient use of resources. Additionally, by inviting a crowd of customers to new product development through a crowdsourcing practice, taking Dell IdeaStorm [28] for example, innovative product or service ideas can be generated and then applied to the production process, thus making the products more attractive to markets and adding value to companies’ business [19]. Further, crowdsourcing eliminates geographical limitations and offers SPs the chance to develop their careers and pursue valuable and interesting jobs [29]. Consequently, an increasing number of service requests turn to the KIC platform for business ideas from the SPs and many SPs adopt online KIC crowdsourcing to gain knowledge and monetary benefits from transactions with service requests [23, 30, 31].

Based on the resource-based view, the network effects can turn SPs into critical resources that bring sustained competitive advantages to the platform [6]. As an intermediary, the KIC platform has to better manage SPs for customer satisfaction, competitive advantages, and sustainable growth. The most direct measure is to control the output quality and enhance customer satisfaction. Existing research proposed different quality control approaches to estimate the SPs’ quality for a specific task. Vakharia and Lease [32] and Li et al. [33] reviewed different task-oriented quality control methods implemented by crowdsourcing platforms practically and researchers academically. The qualification test refers to a set of golden questions with known true answers that the SPs have to answer [34]. They are allowed to perform the real tasks until they pass the test and achieve a threshold score. For example, Clickworker, Crowdsource, and MTurk provide prequalification systems to assess the skill level (e.g., language level) of potential SPs. The gold-injected method mixes golden tasks with real tasks and workers do not know which tasks are gold ones during the task completion process [35]. For example, CrowdComputing Systems and CrowdFlower enable service requesters to inject gold standard data, i.e., a collection of tasks with known correct answers, into their tasks to measure the SPs’ performance automatically. Iterative computation methods iteratively compute and update the SPs’ quality by leveraging all other SPs’ answers for all tasks. The underlying concept is that the SPs who frequently submit reliable answers will be assigned with high quality scores and answers provided by the SPs with high scores will be selected as true answers [36]. Furthermore, many other iterative computation approaches have been applied and developed to compute and measure the SPs’ quality, such as EM-based (expectation estimation) methods, and graph-based methods [37]. However, these SPs’ quality assessment methods are task-specific and have limited applications [38]. In addition, these approaches focused on the determination of a single label [39], e.g., the quality score of the SPs, or whether the SPs are allowed to perform a task.

There also exists another steam of research using multicriteria decision-making (MCDM) methods considering varied attributes to select the appropriate SPs and ensure the output quality for a task in the KIC context. Gong [3] proposed an integrated AHP (Analytic Hierarchy Process) and TOPSIS (Technique for Order Preference by Similarity to an Ideal Solution) approach and took attributes such as credit, skill test score, and the number of completed tasks into consideration, to evaluate and select SPs in a network of crowdsourcing marketplaces. Zhang and Su [40] considered several criteria of the SPs, namely, interests, competence, reputation, and availability to participate, and offered a combined fuzzy DEMATEL (Decision Making Trial and Evaluation Laboratory) and TOPSIS approach to select the candidate SPs for a KIC task. The relationships among these criteria and their indicators were further explored in [41]. However, the MCDM methods mainly aggregate the SPs’ attributes from the perspective of the platform operators or service requesters and only consider a single or a small part of the criteria on behalf of their research goals [40].

Existing studies indicated that the management of SPs is of great significance for the KIC platform. Much attention has been paid to task-specific test design, quality computation methods improvement, and the combination of MCDM approaches to determine the weights of criteria. However, the competency analysis about the SPs and investigation of the competency components are not included, even though it is vital for the SPs to be competent in their crowdsourcing business. As a result, we attempt to connect the competency analysis with KIC and thus the competency analysis is introduced.

2.2. Competency Analysis

The concept of the competency analysis was coined by McClelland and the McBer and Company in the 1970s, and it is defined as components of performance associated with “clusters of life outcomes” [42]. This definition views competency very broadly as any psychological or behavioral attributes associated with long-term success [43]. Later on, the competency movement started in the 1970s. Gibbons [44] argued that the movement was mainly caused by the disconnection between education and the labor market. Professional organizations had to articulate performance standards and requirements and develop competency profiles with which candidates have to comply to be professional [8]. The concept of competency is multidimensional, and various conceptions emerged. Now competency is generally conceptualized as “knowledge, skills, abilities, or other characteristics (KSAOs) that differentiate high from low performance” [14]. To date, the competency analysis is widely applied in many facets of human resource management. Kurz and Bartram [45] and Bartram [46] introduced the Great Eight competency model as a generic competency model that can be applied across a variety of jobs and organizations. By including the research of McClelland, Mirabile [14] proposed the KSAOs competency model that consists of a set of attributes possessed by the workers, typically indicated as knowledge, skills, attitudes, and personal traits required for effective work performance [47]. Based on the Great Eight, Krumm et al. [48] developed the KSAOs model for virtual teamwork which contains 60 potentially relevant items and compared the differences of KSAOs requirements between virtual and traditional teams. According to the empirical study in Hertel et al. [49], the authors found that a set of KSAOs (e.g., persistence, willingness to learn, creativity, independence, interpersonal trust, and intercultural skills) were related to tele-cooperation performance and indicated that creativity and independence significantly contributed to the team performance. Cogliser et al. [50] indicated that computer self-efficacy was the main performance predictor of a virtual management organization. Maurer and Lippstreu [51] conducted a survey to rate a varied set of KSAOs in terms of improvability, importance, and “needed at entry” facets. Prahalad and Hamel [52] pointed out that focusing on a collection of core competencies, i.e., the company’s collective knowledge about how to coordinate diverse production skills and technologies, will help the corporation gain competitive advantages. Boyatzis [53] offered a “total” system approach that determines which characteristics of the managers enable them to be effective in various management jobs. Wu and Lee [54] developed the competencies of the global managers by using eight different IQs and proposed the fuzzy DEMATEL method to segment the required competencies into meaningful portions. Li et al. [7] proposed a multicriteria competency analysis framework for the crowdsourcing delivery personnel.

In general, these studies present a fact that individual or organizational performance is influenced by various competency items. Across different domains of surveys, however, models are rather heterogeneous in terms of which specific competencies are significant to performance. Although these studies offer valuable contributions and insights that help us to understand the competency of different roles, the differences in their findings highlight that their model structures are by no means universal and strongly depend on the characteristics of the specific context.

3. Methodology

To investigate the competency dimensions and understand the SPs’ competency in KIC, this paper takes the advantages of the SPs’ experience sharing information and explores the SPs’ competency based on their understanding and perception. We first apply the topic modelling techniques to the crawled raw data and then construct the three-level KSAT competency model based on the extracted and identified competency dimensions. Later, the BWM is leveraged to explore the importance and weights of competency criteria in the KSAT competency model. Our research framework is depicted in Figure 1. We detail our methods in this section.

3.1. Topic Modelling

The rising development and accessibility of large electronic archives, along with increased computational facilities, have led to an interest in the textual content analysis [55]. The topic modelling is an effective modelling method for extracting implied themes in large-scale text based on word cooccurrence for each document in the corpus [56–58]. Moreover, the topic modelling is a useful way to “let the text talk” due to the independence from the evaluator’s personal opinions or experiences [59], and it has been studied and applied in various fields, such as recommendation systems [60], online health communities [61], and customer-generated content analysis [62].

The LDA is a well-known unsupervised machine learning technique for natural language processing [63] and is the simplest and most popular topic modelling algorithm [64, 65], which has the advantage of recognizing the hidden topics and mining deep semantics of huge amounts of textual documents through an effective way. The basic idea of the LDA is that each document exhibits a mixture of latent topics wherein each topic is characterized by a distribution over the words, i.e., per-document topic distributions and per-word topic distributions [66, 67]. The generative probabilistic model of the LDA is represented in Figure 2.

The LDA defines the following terms:(1)A word is the basic unit of discrete data, defined to be an item from a vocabulary indexed by (2)A document is a sequence of N words denoted by , where is the nth word in the sequence(3)A corpus is a collection of M documents denoted by

As shown in Figure 2, the words within documents are observable variables, while the other variables, including the topics , (the distributions over words), the topic distribution per document , and the per-word topic distribution , are not known. These latter items represent unobservable variables (white circles in Figure 2) that should be estimated from the analysis of observable variables (shaded circles in Figure 2). Parameters and are the hyperparameters for prior distributions of and , respectively. The plate notations at the bottom of each rectangle denote their usage to illustrate the replications; i.e., the K plate represents the number of topics, the N plate represents the total number of unique words within documents, and the M plate represents the number of documents. The arrows represent conditional dependencies among components in the following way: per-word topic distribution is dependent on the topic distribution per document , and the observed word in each document is dependent on and all the topics . The conditional dependencies enable the definition of the joint distribution of observed and unobserved variables. As a result, the LDA has remarkable advantages in employing Bayesian learning to infer latent variables by calculating their posterior distribution from the joint distribution. Learning the unobservable components allows us to capture the hidden semantic structure in the documents. More specifically, the main outputs yielded by the LDA are, namely, topics and their weights in each document , and per-word weight within each topic . That is to say, the outputs of LDA consist of K topics, wherein each topic is denoted as a combination of words with different probability of occurrence. However, the combination of words cannot deliver the precise meaning of each topic; it would be better to label them. In other words, human judgments and intervention are a usual way to label the topics on the basis of the semantic similarities of included words [68]. For more details about the LDA, one can refer to [66, 67].

3.2. Best-Worst Method

The BWM is a recently developed MCDM method [69]. It uses a structured way to conduct the pairwise comparisons, which has several major advantages [7]. (i) The decision-makers are required to identify the best and worst criteria (or alternatives) prior to conducting a pairwise comparison, which enables them to have a clear understanding of the range of evaluation, consequently, resulting in more consistent and reliable pairwise comparisons [70]. (ii) By using two opposite references (best and worst) in a single optimization model, which is called consider-the-opposite-strategy, the BWM has been proven to be effective in mitigating possible anchoring bias arising during the pairwise comparisons process [71, 72]. (iii) The BWM better balances the data and time efficiency and, at the same time, enables the decision-makers to check the consistency of the provided pairwise comparisons [71]. On the one hand, compared to other pairwise comparison-based methods using a single vector such as the Swing and SMART family, the BWM overcomes the main weakness that it is unavailable to check pairwise comparison consistency, while maintaining the high data (and time) efficiency of such single vector input-only methods [71]. On the other hand, the number of pairwise comparisons needed to be conducted in the BWM is less than that of full-matrix-based methods, such as AHP, which effectively enhances time and data efficiency. Although the number of pairwise comparisons under the full-matrix-based method is sufficient to check the consistency, decision-makers have to answer too many questions, which can result in confusion and inconsistency [71].

The method has been widely applied in many real-world problems, such as the supply chain, manufacturing, logistics, airline industry, supplier selection, and service quality evaluation. For a review of the applications, one could refer to Mi et al. [73].

In this study, we use the BWM due to its advantages and wide applications. Specifically, the steps to determine the weights of criteria using the BWM are as follows [69, 70]:(1)Determine a set of decision criteria by experts or decision-makers.(2)Identify the best (B) and the worst () criteria by experts or decision-makers.(3)Determine the preference of the best over all the other criteria by experts or decision-makers using a number between 1 and 9 (where 1 is “equally important” and 9 is “extremely more important”). The result of best-to-others comparisons is vector , where shows the preference of criterion B over criterion .(4)Determine the preference of all the criteria over the worst by experts or decision-makers using the same scale (1 to 9). The result of others-to-worst comparisons is vector , where indicates the preference of criterion j over criterion .(5)Compute the optimal weights .

The optimal weights are computed by minimizing the maximum absolute differences of for all j, which can be expressed by the following optimization problem:subject to

Equation (2) is converted intosubject to

is the result of equation (4), indicating the optimal weight of criteria. is the result of the objective function in equation (4), indicating the consistency of the provided pairwise comparisons. If is closer to zero, a higher level of consistency is in the pairwise comparisons by experts.

When the MCDM problem is a hierarchical criteria tree, then the results of equation (4) are called local weights. To determine the global weights of subcriteria in the last level of the tree, their local weights are multiplied by the weights of the category to which they belong. When we have a number of experts, we follow all the five steps for each expert individually. To aggregate the final results (global weights), we use the geometric mean.

4. Data Analysis and Results

In this section, we give details of the data collection and preprocessing, the data analysis, and the results of our model. Related procedures are conducted on an Intel Core i7 CPU, 16 GB RAM machine. The raw data are crawled and analyzed based on scikit-learn under Python version 3.6.5.

4.1. Data Collection and Preprocessing

According to the report [74], the transaction size of the KIC market in 2020 increased by 306.3 billion RMB compared to 2019 in China. As facilitated by the Chinese policy “mass entrepreneurship and innovation” [75], crowdsourcing platforms in China, such as ZBJ.COM, epwk.com, and 680.com that are specialized in supporting KIC activities, also experienced booming growth. Given that SPs participate in these platforms at different time and with various business capabilities, in order to facilitate their business growth and career development, these platforms provide them the opportunities for knowledge and experience sharing and learning. Particularly, these platforms will regularly organize interviews with successful SPs (i.e., top order winners, top income earners, or long-established SPs) to share experience regarding a series of questions, such as “how did you manage to get so many orders?”; “what did you do to get an order at such a high price?”; “what efforts did you make to be a long-established SP?”; and “What is your plan to further facilitate your business in the future?”. Then, the interview records will be posted on their platforms in order that more SPs will access and learn from these successful experiences. According to our survey with SPs of ZBJ.COM, they considered the experiences of the successful SPs to be highly informative, providing effective guidance on the issues they were facing at that time. We think that these online interview records are of great significance for the KIC platform to understand SPs competency and derive competency dimensions that can improve SPs management. To undertake this study, we apply the following steps to collect the raw data, i.e., the interview records, as the corpora for this research.(1)Selecting the data sources: to ensure the quality and quantity of the raw data, we select the most three popular KIC platforms, namely, ZBJ.COM, epwk.com, and 680.com, as the data source.(2)Writing the clawer programming: due to the large amount of content available on these platforms, accessing the data online through the programming is an effective and efficient way. Consequently, we code the programming in Python.(3)Crawling the raw data: running the programming in Step 2, we crawl the online interview records on the platforms. Key information including the title, content, poster, post time, view times, comments, and responses is collected and finally the corpora contain 1760 records in total within a time frame from January 2014 to July 2019.

The raw data collected from the KIC platform are all unstructured and in Chinese. To effectively extract implied topics from these large-scale texts, we apply the general text preprocessing steps to clean the unstructured text for topic modelling as follows:(1)Cutting each article into sentences and eliminating all numbers and alphabets, just leaving the Chinese characters: note that Chinese text is processed in UTF-8 encoding format.(2)Defining the user dictionary and tokenizing each sentence into multiple space-separated words: in this step, we refer to a predefined user dictionary to know where to pause in a sentence. Unlike in English, there is no space between two Chinese characters or phrases. Given the different segmentation, the meaning of the sentence may be completely different. With a predefined dictionary, a complete Chinese sentence will be cut into several tokens.(3)Removing the stopwords from each sentence: to better clean the data, except for using the popular Chinese stopwords list, we construct our own stopwords list with words that are domain-specific and highly appear in our corpora but useless for analysis (e.g., crowdsourcing, the platform, innovation design).(4)Constructing the term-document matrix: after removing the stopwords, the term-document matrix (i.e., the distribution/frequency of terms (rows) within documents (columns)) used as the main input to the LDA is constructed.

4.2. Competency Dimensions Identification

Prior to building a topic model from the experience sharing articles, we need to exogenously give the number of topics K. To obtain the proper topic number, Aletras and Stevenson [76] and Ramage et al. [77] introduced cosine measures to capture the similarities of generated topics, where the lowest average cosine similarity denotes the best model and thus determines the appropriate topic number. In this sense, we determine the optimal number of topics from a discrete range rather than a continuous range. We calculate the average cosine similarity setting the number of topic K over a number set . Figure 3 presents a comparison over discrete topic numbers in terms of the average cosine similarity of topics. As shown in Figure 3, we find that the average cosine similarity score becomes the lowest (0.0375) when the number of topics is set to 18. Therefore, the appropriate number of topics is 18. We then set the topic number K to 18 and we obtain 18 clusters of words from the corpora (see the second column in Table 1). As discussed earlier, the topics are distributions over words; the top five keywords with the highest probability (most frequency) derived from the posterior distribution (i.e., ) are provided for each topic in Table 1. Given the corpora are in Chinese, we translate the words in English for readers to better understand the topics and attach the original Chinese words.

However, presenting as the combinations of words cannot characterize the underlying content of the topics. Unfortunately, automatic labeling of the topics is infeasible because the topic extraction by the LDA is an unsupervised learning process. Instead, it requires human judgment and intervention to check the coherence and meaningfulness of these topics and then label them through their judgment [66, 68]. As a result, the authors interviewed the managers and the SPs in ZBJ.COM and the researchers in related fields and then validated and labeled the extracted topics, by referring to the Spencer’s competency dictionary [13]. As in Kurz and Bartram [45], we refer to the label of each topic as a competency dimension, as indicated in the last column of Table 1.

4.3. The KSAT Competency Model

Mirabile [14] proposed the KSAOs competency model that consists of a set of attributes possessed by workers, typically indicated as knowledge, skills, attitudes, and personal traits required for effective work performance. In order to frame our findings in theory and also offer more significant insights the LDA results, the eighteen competency dimensions are classified into four competency factors, namely, knowledge, skill, ability, and trait, according to the aforementioned interviews with managers, the SPs and related researchers, on the basis of the meaning of topics and competency dimensions. As in Kurz and Bartram [45], we define the main criteria as competency factors and subcriteria as competency dimensions. Particularly, we summarize the two competency dimensions, demand understanding and reasonable suggestion, into one single dimension, task analysis, and summarize team composition and team environment as team management, likewise. Subsequently, the three-level KSAT competency model is constructed and presented in Figure 4, and the descriptions of the competency criteria in the model are presented in Table 2. The model embraces a comprehensive and hierarchical set of competency criteria that offers the researchers and practitioners in KIC the flexibility to systematically build, verify, and change the SPs’ selection and evaluation mechanisms to suit their requirements.

4.4. Weights of Competency Criteria

Due to the KIC platform’s goals on healthy and sustainable operation process, the proposed KSAT competency model can be further employed to evaluate the SPs’ competency and select and keep the competent SPs. Identifying the importance of each competency criterion plays a significant role in better guiding both the SPs and the KIC platform to plan, design, and implement mechanisms and strategies for sustainable development and management. We develop an online questionnaire involving the aforementioned KSAT competency criteria shown in Table 2 and employ the BWM to determine the weights of competency criteria. In this step, we invite 15 experts and ask for their opinion. Of the 15 respondents involved in this research, 9 with over 5 years’ working experience are employed as managers in crowdsourcing companies, like ZBJ.COM. The remaining 6 are researchers in the crowdsourcing field with an average of 7 years’ research experience. To ensure that all the respondents have adequate information to conduct the comparisons, the description of the BWM and the competency factors and dimensions are also provided.

In this paper, the importance of the competency criteria for the KIC platform to evaluate the SPs’ competency is examined and evaluated based on the four competency factors (i.e., knowledge, skill, ability, and trait) and twenty competency dimensions (eighteen in level 2 and two in level 3). Tables 3–5 present the local and global weights of the competency factors and dimensions in levels 1–3, respectively.

5. Discussion

In this study, our concentration is to investigate and understand the SPs’ competency by recognizing detailed competency criteria (i.e., competency factors and competency dimensions) in KIC. We first explore the successful SPs’ experience sharing information using the LDA approach to identify natural and hidden competency dimensions. Then, we construct the KSAT competency model and determine the weights of competency criteria for the KIC platform evaluating the SPs’ competency. Such information acts as a source on how to evaluate, manage, and encourage SPs.

5.1. Results Analysis Related to KSAT Competency Model

The highly competent SPs are necessary for the KIC platform to meet fluctuating demands for numerical and functional flexibility [80]. In KIC markets, the SPs’ competency acts as decision factors that have a great influence on the quality of task outcomes, customer satisfaction and loyalty, and the platform’s reputation and sustainable growth. This study aims to understand the SPs’ competency in KIC and identifies a list of competency criteria by extracting from the competent SPs’ experience. The KSAT competency model is subsequently constructed based on competency theory that Mirabile [14] conceptualized competency as “knowledge, skills, abilities, or other characteristics” that differentiate high from low performance. Specifically, the KSAT competency model proposed in this work includes knowledge (entrepreneurial experience, profession experience, and customers’ industry background), skill (task analysis, modification and after-sales service, customer relationship management, branding, and team management), ability (communication ability, customer acceptance, innovation ability, and online and offline coordination), and trait (professional dedication, trustworthiness, competitive spirit, and achievement orientation).

Knowledge has been deemed to be a key factor for success in the empirical research of entrepreneurship and crowdsourcing [15, 79]. In KIC, the SPs are mainly regarded as self-employed workers or small- and medium-sized enterprises (SMEs) and viewed as entrepreneurs [93, 94]. Entrepreneurial experience is thought to be mandatory when starting up a new business as it can help focus on strategic issues to alleviate the liabilities of newness, such as establishing new business partnerships [15]. Also, entrepreneurs will be more sensitive to business opportunities, future technologies, and customer demand via learning by doing process [15, 95]. Furthermore, profession experience can boost experiential learning, enhance the development of operational knowledge, and ease the transfer process of prior knowledge before starting the new business [15]. Relying on entrepreneurial and professional experience, entrepreneurs can also save costs on routine development. In addition to its importance in entrepreneurship, knowledge is the determinant of the good performance in crowdsourcing. According to [96], knowledge diversity of the individual crowds facilitates all types of contribution to open innovation projects as having knowledge in diverse fields allows the contributors to understand the task requirements [97] or blend disparate solution elements in novel ways. Being familiar with the unique industry background of the targeted customers will help the SPs better understand customer requirements, cut to the heart of the matter, and achieve consensus on the solution with customers [97].

Skill, defined as the domain-specific occupational expertise in this work, appears to be advantageous for both the KIC platform and the SPs. In the crowdsourcing context, skill is regarded as the basis for preselecting the proper SPs in auction systems as it demonstrates how the SPs are at doing particular tasks [40, 41]. Rather than exploring and extracting dimensions of skill, the state of the art mainly concentrated on the indictors in a specific context to measure the level of skill or directly assign a numerical value to quantify [41]. The KIC tasks are commonly complex and creative, which cannot be done by computers [3]. A complete understanding of the task requirements and sound advice can help service requesters to better visualize the desired outcomes, thus reducing the perception gap between the service requesters and the SPs [97]. We conclude the two subdimensions (i.e., demand understanding and reasonable suggestion) as task analysis, which is viewed as a critical starting point to the task success. Meanwhile, the SPs in KIC have to offer modification and after-sales service, because the task outcome is not going to be perfect and matches the customers’ expectations all at once. Offering such post-sale services could reduce customers’ risk perception of task failure and poor transaction experience. Accordingly, offering high-quality services is a powerful way to enhance customer satisfaction and thus retain the targeted customers [86]. Customer relationship management, a strong tool in marketing and business [87], is another competency dimension of the SPs to maintain a long-term relationship with their service requesters. In addition, the SPs commonly own one or several virtual stores on the KIC platform. Brand image of the stores plays a vitally important role in attracting customers as it can create a positive attitude towards the SPs’ virtual stores that will, in turn, encourage the intention to repurchase and produce positive word-of-mouth [86]. Furthermore, team management is also mentioned by the SPs as a core competency dimension in conducting KIC business. In China, many SPs register on the KIC platform as a team so as to organically aggregate human resources and leverage each member’s strengths. However, existing research rarely explores the team management competency of SPs in the environment of crowdsourcing.

Despite the domain-specific skill, ability, defined as the task performance or the effective outcomes achieved during the crowdsourcing business process, is also regarded as a main competency factor by the SPs. The KIC activities involve many intelligence-related tasks and intangible services that everyone is likely to have a different understanding of the task requirements and outcomes. To mitigate such cognitive differences, communication is essential. Concretely, the accurate and useful information about the understanding of customer demand and expected outcomes, problem-solving plans, and modification suggestions need to be efficiently and actively conveyed by the SPs. As communication helps narrow cognitive gaps, innovation ability accounts for divergent thinking and creative ideas, differentiating services, and products from others [98]. In innovation contests, the innovativeness of solutions is considered an important reference for the selection of the winner [17]. In addition, customer acceptance reflects the degree of customers’ trust in the SPs and the probability that the customers expect to obtain high-quality outcomes from the SPs [40]. Gaining customers’ trust and acceptance has always been considered as one of the ultimate goals of marketing [92]. Additionally, the online SPs may also offer services offline as a complementary sales channel, which is a distraction of SPs’ time and effort on online business. Hence, coordinately operating online and offline crowdsourcing business at the same time is a challenge for the SPs.

Besides, personal traits play a critically important role in determining the SPs’ competency in the KIC activities. Previous research has widely investigated the significance of personal traits in exploring individual work performance or entrepreneur success [81]. Batey and Furnham [99] found that the “big five” personality traits account for up to 47% of the variation in divergent thinking. A meta-analysis by Feist [100] showed that the individuals who are open to new experiences, conscientious, hostile, confident, and emotionally impulsive are more likely to generate creative outcomes. Findings by Sebora et al. [81] asserted that the achievement orientation of the founders is positively related to the success of e-commerce entrepreneurial ventures as it helps the entrepreneurs overcome obstacles and compensates for other weaknesses. In crowdsourcing delivery, personal traits were considered as a main criterion in [7] for evaluating the delivers’ competence and responsibility is one of the most important subcriteria when quantifying the competence score. These researches indicate that personal traits have been found to be a robust factor of high-quality outcomes and performance [81]. Specifically, in knowledge-intensive crowdsourcing, the SPs regard professional dedication, trustworthiness, competitive spirit, and achievement orientation as the competency dimensions.

5.2. Results Analysis Related to Competency Importance

The BWM results show that the competency criteria have different importance when the KIC platform evaluates the SPs’ competency. Table 5 indicates that “customer relationship management” and “communication ability” have the highest priorities of all the competency factors, while “team environment” is the least important. “Customer relationship management” emerged in the 1970s [101] as a useful tool for managing and optimizing sales-force automation within companies, enhancing customer satisfaction and loyalty [102, 103], and consequently reaching and retaining long-term partnerships with customers [104]. In this sense, the SPs with high-level customer relationship management will encourage the loyal end customers to retain on the platform, thus helping generate sustainable value and maintaining long-term growth. This attests the business model of Upwork (one of the largest online KIC marketplaces) charging its fees on a sliding scale to encourage the longer-term relations between the SPs and service requesters [105].

As to “communication ability,” the KIC activities involve intensive and dynamic interaction among the end customers, the platform operators, and the SPs. Online chatting, instant messaging, and social media are the most popular ways for the SPs to interact with their customers [16]. With more information processed, communication about the task ideas may shift the interpretation or understanding of the task at hand [106], towards a way that helps to reach consensus between the SPs and the service requesters about the way how tasks are performed and the form of outcomes, thus reducing the gap between the service requesters’ perception and expectation about the service and enhancing their overall satisfaction. Foundational research emphasized the role of communication in the form of dialogue, feedback, and other contextual factors, in the way a message is received and interpreted by the viewers, as well as how they respond [106, 107], which indicates that the KIC platform may develop different types of communication systems to assist the SPs with their interaction with customers in various manners. Also, a high-level of communication ability that focuses on professional interactions, being honest and responsive, and respecting customer culture during interactions, will impress the customers with professional knowledge, and high-quality services, leading to a good customer relationship and reputation [89]. Therefore, it is suggested that the SPs need to focus on the language used, cultural awareness, and promptness of their response during the interaction process. While “team environment” is weighted as the least important competency factor by experts, it is in line with the actual situation that, as a platform does not hire the SPs, it is relatively impossible for a platform to observe their interactions within the organization. However, the KIC platform need to pay attention to this factor as it is considered as an essential dimension by the SPs that influences their competency and performance.

As shown in Table 3, “skill” is the most important of all the four competency factors. The main reason for that is domain-specific skills are regarded as the key to a successful business [108, 109]. Unlike the short and low-complexity simple tasks [110], the KIC tasks are domain-specific, high in complexity, and not easy to decompose apparently, which put high requirements on the participating SPs [111]. Gong [3] pointed out that the lack of domain-specific skill and expertise has limited the development of the KIC marketplaces, which revealed the dominating role of the SPs’ skill in the KIC context. Further, the SPs’ skill demonstrates how capable he or she is at doing particular tasks in the domain, and the SPs with high skill are expected to contribute to high-quality outcomes [40, 108]. The prerequisite for the SPs conducting a specific task is that they have the relevant and necessary skills to perform at the required quality [112]. This implies that the KIC platform could grade the SPs’ skill levels, such that the resources can be strategically allocated and assigned. For example, the high-skill level SPs could be assigned to perform more complex KIC tasks that require a comprehensive usage of different skills. For the SPs, they need to develop and enhance different types of skills, so as to perform more complex tasks to gain more income.

The “ability” factor ranks in second place, which means that, in absolute terms, it is still more important than “knowledge” and “trait.” Among all the four competency factors, it is not surprising that “knowledge” and “trait” are weighted as the two least important. This is in line with the actual situation involving KIC activities in China, due to the fact that, to attract more SPs to participate in KIC activities, the input control of the KIC platform is relatively weak. For example, the steps for applicants to be SPs at ZBJ.COM (https://help.zbj.com/fw/detail?articleId=14762) are (1) registering by a telephone number and a password; (2) filling in some required information, such as self-introduction, location, e-mail address, and task types willing to perform; and (3) uploading photos of identification card. The steps of epwk.com (https://i.epwk.com/User/Basicinfo/index.html) and 680.com (http://help.680.com/view_4.html) are similar to ZBJ.COM.

In addition, the fact that “knowledge” is not ranked the lowest infers that entrepreneurial- and business-related knowledge experience are also regarded by the experts as essential competency dimensions in demonstrating the SP’s competency and performance.

6. Conclusion

In this study, by combining text mining and decision-making techniques, we conducted a comprehensive competency analysis of SPs in the environment of KIC. In this process, we leveraged online interview records posted by several KIC platforms in China, which includes the successful SPs’ opinions about the qualifications to obtain sustainable business growth and conduct their career well. By applying the topic modelling approach to these materials, we identified four competency factors and twenty competency dimensions in general and thus constructed the hierarchical KSAT competency model. Further, we employed the BWM to identify the weights and priorities of the competency criteria in the KSAT competency model. The relationships between the competency criteria are also discussed, leading to the following conclusions:(1)The proposed KSAT competency model gives practitioners of the KIC platform a comprehensive vision of SPs’ performance standards in the context of KIC. Further, it also provides the practitioners flexibility to choose different criteria according to different application scenarios and objective. In KIC markets, the SPs, viewed as entrepreneurs, are varied in backgrounds, skills, and abilities and contributions, leading to great challenges to their management for the KIC platform. Also, the SPs in the start-up stages lack the benefits of continuous competition. To stabilize the market environment and advances the platform’s interests, the platform managers have to develop better input control mechanisms, incentive mechanisms, and SPs’ lifecycle management systems to retain and incubate the valuable SPs. Also, the KSAT competency model can be considered as a self-check or learning system rather as a mere grading tool to help the SPs realize their strengths and weaknesses and allow them to improve in aspects where they are weaker, in a structured and targeted manner.(2)In the operation system of KIC, the platform is the rule maker and it is better for the SPs to focus on developing and enhancing those competency criteria that the decision-makers considered as important to achieve a higher ranking in the platform’s evaluation system. Particularly, as “skill” and “ability” are viewed as the most important competency factors, it is necessary for the SPs to master not only excellent occupational expertise but also comprehensive business operation capabilities to get ahead in the fierce competition.(3)The KIC platform also needs to pay attention to the competency criteria weighted as least important ones, such as team environment, when developing evaluation mechanisms and management systems. Insufficient attention to these criteria may lead to unfair or biased evaluations as the SPs view them as essential elements that affect their business success.

This research can be extended in several ways. First, although the LDA model shows popular applications in text analysis, it still has some shortcomings. As an unsupervised learning algorithm, the LDA inherently has disadvantages in fully understanding natural languages but it requires no human intervention. Future research may use supervised learning algorithms for identifying the SPs’ competency components from online interview records. Second, further longitudinal exploration can be conducted to analyze the antecedent and consequential relationships among these competency criteria. Experiments are also needed to help the crowdsourcing researchers establish causality by eliminating extraneous factors and endogeneity issues. Additionally, experimental approaches will possibly enable further explorations into situations where various management mechanisms are existing, in order that more insights can be derived from their interactions and how they could jointly work to enhance performance and customer purchasing intentions. Third, this research is limited by the single source of materials, which may result in incomplete competency identification. It might be necessary to collect and use more data from different data sources in future research to obtain more generalized and significant findings. Finally, it may be worthwhile to extend our study to where it could include other industries, other text mining techniques, and other data as future research for more significant analytical results on the SPs’ competency and performance in KIC.

Data Availability

The data used in this paper consist of two parts: the online interview records of SPs posted on the KIC platforms and the BWM questionnaires that were sent to the selected experts. They are available upon request to the corresponding author at [email protected].

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was supported by the National Science and Technology Support Program of China (project no. 2018YFB1403602), the Fundamental Research Funds for the Central Universities (project no. 2020CDCGJX019), the Graduate Research and Innovation Foundation of Chongqing (Project no. CYS20007), the Technological Innovation and Application Program of Chongqing (project no. cstc2019jscx-mbdxX0008), and the National Social Science Foundation of China (project no. 18BJY066). The authors would like to thank the members in the team of R&D of technology consulting service platform for their assistance and guidance.

References

X. Tian, J. Shi, and X. Qi, “Talent crowdsourcing via stochastic sequential assignments,” SSRN Electronic Journal, 2018.
View at: Publisher Site | Google Scholar
A. Kittur, J. V. Nickerson, M. Bernstein et al., “The future of crowd work,” in Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW, pp. 1301–1317, San Antonio, TX, USA, February 2013.
View at: Publisher Site | Google Scholar
Y. Gong, “Estimating participants for knowledge-intensive tasks in a network of crowdsourcing marketplaces,” Information Systems Frontiers, vol. 19, no. 2, pp. 301–319, 2017.
View at: Publisher Site | Google Scholar
M. Wessel, F. Thies, and A. Benlian, “Opening the floodgates: the implications of increasing platform openness in crowdfunding,” Journal of Information Technology, vol. 32, no. 4, pp. 344–360, 2017.
View at: Publisher Site | Google Scholar
F. Thies, M. Wessel, and A. Benlian, “Network effects on crowd funding platforms: exploring the implications of relaxing input control,” Information Systems Journal, vol. 28, no. 6, pp. 1239–1262, 2018.
View at: Publisher Site | Google Scholar
M. Sun and E. Tse, “The resource-based view of competitive advantage in two-sided markets,” Journal of Management Studies, vol. 46, no. 1, pp. 45–64, 2009.
View at: Publisher Site | Google Scholar
L. Li, X. Wang, and J. Rezaei, “A bayesian best-worst method-based multicriteria competence analysis of crowdsourcing delivery personnel,” Complexity, vol. 2020, Article ID 4250417, 17 pages, 2020.
View at: Publisher Site | Google Scholar
M. Mulder, “Conceptions of professional competence,” in International Handbook of Research in Professional and Practice-Based Learning, C. H. H. G. S. Billett, Ed., pp. 107–137, Springer, Dordrecht, The Netherlands, 2014.
View at: Publisher Site | Google Scholar
N. Otani, Y. Baba, and H. Kashima, “Quality control of crowdsourced classification using hierarchical class structures,” Expert Systems with Applications, vol. 58, pp. 155–163, 2016.
View at: Publisher Site | Google Scholar
F. Daniel, P. Kucherbaev, C. Cappiello, B. Benatallah, and M. Allahbakhsh, “Quality control in crowdsourcing,” ACM Computing Surveys, vol. 51, no. 1, pp. 1–40, 2018.
View at: Publisher Site | Google Scholar
K. Kritikos, B. Pernici, P. Plebani et al., “A survey on service quality description,” ACM Computing Surveys, vol. 46, no. 1, pp. 1–58, 2013.
View at: Publisher Site | Google Scholar
P. Hager, A. Gonczi, and J. Athanasou, “General issues about assessment of competence,” Assessment & Evaluation in Higher Education, vol. 19, no. 1, pp. 3–16, 1994.
View at: Publisher Site | Google Scholar
L. M. Spencer and S. M. Spencer, Competence at Work: Models for Superior Performance, John Wiley & Sons, London, UK, 1993.
R. J. Mirabile, “Everything you wanted to know about competency modeling,” Training & Development, vol. 51, no. 8, p. 73, 1997.
View at: Google Scholar
A. Oe and H. Mitsuhashi, “Founders’ experiences for startups’ fast break-even,” Journal of Business Research, vol. 66, no. 11, pp. 2193–2201, 2013.
View at: Publisher Site | Google Scholar
L. Aroean, D. Dousios, and N. Michaelidou, “Exploring interaction differences in microblogging word of mouth between entrepreneurial and conventional service providers,” Computers in Human Behavior, vol. 95, pp. 324–336, 2019.
View at: Publisher Site | Google Scholar
T. Mack and C. Landau, “Winners, losers, and deniers: self-selection in crowd innovation contests and the roles of motivation, creativity, and skills,” Journal of Engineering and Technology Management, vol. 37, pp. 52–64, 2015.
View at: Publisher Site | Google Scholar
L. B. Jeppesen and K. R. Lakhani, “Marginality and problem-solving effectiveness in broadcast search,” Organization Science, vol. 21, no. 5, pp. 1016–1033, 2010.
View at: Publisher Site | Google Scholar
E. H. Hwang, P. V. Singh, and L. Argote, “Jack of all, master of some: information network and innovation in crowdsourcing communities,” Information Systems Research, vol. 30, no. 2, pp. 389–410, 2019.
View at: Publisher Site | Google Scholar
J. N. Cummings, J. A. Espinosa, and C. K. Pickering, “Crossing spatial and temporal boundaries in globally distributed projects: a relational model of coordination delay,” Information Systems Research, vol. 20, no. 3, pp. 420–439, 2009.
View at: Publisher Site | Google Scholar
F. Leal, B. M. Veloso, B. Malheiro, H. González-Vélez, and J. C. Burguillo, “Scalable modelling and recommendation using wiki-based crowdsourced repositories,” Electronic Commerce Research and Applications, vol. 33, Article ID 100817, 2019.
View at: Publisher Site | Google Scholar
A. R. Kurup and G. P. Sajeev, “A task recommendation scheme for crowdsourcing based on expertise estimation,” Electronic Commerce Research and Applications, vol. 41, Article ID 100946, 2020.
View at: Publisher Site | Google Scholar
X. Zhang, B. Gong, H. Ni, Z. Liang, and J. Su, “Identifying participants’ characteristics influencing participant estimation in knowledge-intensive crowdsourcing,” in Proceedings of the 2019 8th International Conference on Industrial Technology and Management (ICITM), Cambridge, UK, March 2019.
View at: Publisher Site | Google Scholar
J. Howe, “The rise of crowdsourcing,” Wired Magazine, vol. 14, no. 6, pp. 1–4, 2006.
View at: Publisher Site | Google Scholar
J. Howe, “Croudsourcing: why the power of the crowd is driving the future of business,” Nieman Reports, vol. 62, no. 4, p. 47, 2008.
View at: Google Scholar
Y. Xu, D. E. Ribeiro-Soriano, and J. Gonzalez-Garcia, “Crowdsourcing, innovation and firm performance,” Management Decision, vol. 53, no. 6, pp. 1158–1169, 2015.
View at: Publisher Site | Google Scholar
J. Oosterman, A. Bozzon, G.-J. Houben et al., “Crowd vs. experts,” in Proceedings of the 23rd International Conference on World Wide Web - WWW ’14 Companion, pp. 567-568, Seoul, South Korea, April 2014.
View at: Publisher Site | Google Scholar
B. L. Bayus, “Crowdsourcing new product ideas over time: an analysis of the Dell IdeaStorm community,” Management Science, vol. 59, no. 1, pp. 226–244, 2013.
View at: Publisher Site | Google Scholar
M. Christoforaki and P. G. Ipeirotis, “A system for scalable and reliable technical-skill testing in online labor markets,” Computer Networks, vol. 90, pp. 110–120, 2015.
View at: Publisher Site | Google Scholar
A. Hagiu and H. Hałaburda, “Information and two-sided platform profits,” International Journal of Industrial Organization, vol. 34, no. 1, pp. 25–35, 2014.
View at: Publisher Site | Google Scholar
J. Brustle, Y. Cai, F. Wu, and M. Zhao, “Approximating gains from trade in two-sided markets via simple mechanisms,” in Proceedings of the 2017 ACM Conference on Economics and Computation, pp. 589-590, Cambridge, MA, USA, June 2017.
View at: Publisher Site | Google Scholar
D. Vakharia and M. Lease, “Beyond AMT: an analysis of crowd work platforms,” 2013, http://arxiv.org/abs/1310.1672.
View at: Google Scholar
G. Li, J. Wang, Y. Zheng, and M. J. Franklin, “Crowdsourced data management: a survey,” IEEE Transactions on Knowledge and Data Engineering, vol. 28, no. 9, pp. 2296–2319, 2016.
View at: Publisher Site | Google Scholar
S. Zhu, S. Kane, J. Feng, and A. Sears, “A crowdsourcing quality control model for tasks distributed in parallel,” in Proceedings of the 2012 ACM Annual Conference Extended Abstracts on Human Factors in Computing Systems Extended Abstracts-CHI EA’12, pp. 2501–2506, ACM Press, New York, NY, USA, 2012.
View at: Publisher Site | Google Scholar
X. Liu, M. Lu, B. C. Ooi, Y. Shen, S. Wu, and M. Zhang, “Cdas: a crowdsourcing data analytics system,” Proceedings of the VLDB Endowment, vol. 5, no. 10, pp. 1040–1051, 10 edition, 2012.
View at: Publisher Site | Google Scholar
L. Shamir, D. Diamond, and J. Wallin, “Leveraging pattern recognition consistency estimation for crowdsourcing data analysis,” IEEE Transactions on Human-Machine Systems, vol. 46, no. 3, pp. 474–480, 2016.
View at: Publisher Site | Google Scholar
M. Venanzi, J. Guiver, G. Kazai, P. Kohli, and M. Shokouhi, “Community-based bayesian aggregation models for crowdsourcing,” in Proceedings of the 23rd International Conference on World Wide Web-WWW’14, pp. 155–164, Seoul, South Korea, April 2014.
View at: Publisher Site | Google Scholar
K. Li, S. Wang, and X. Cheng, “Crowdsourcee evaluation based on persuasion game,” Computer Networks, vol. 159, pp. 1–9, 2019.
View at: Publisher Site | Google Scholar
D. Dang, Y. Liu, X. Zhang, and S. Huang, “A crowdsourcing worker quality evaluation algorithm on MapReduce for big data applications,” IEEE Transactions on Parallel and Distributed Systems, vol. 27, no. 7, pp. 1879–1888, 2016.
View at: Publisher Site | Google Scholar
X. Zhang and J. Su, “A combined fuzzy DEMATEL and TOPSIS approach for estimating participants in knowledge-intensive crowdsourcing,” Computers & Industrial Engineering, vol. 137, Article ID 106085, 2019.
View at: Publisher Site | Google Scholar
X. Zhang, B. Gong, Y. Cao, Y. Ding, and J. Su, “Investigating participants’ attributes for participant estimation in knowledge-intensive crowdsourcing: a fuzzy DEMATEL based approach,” Electronic Commerce Research, vol. 145, pp. 1–32, 2020.
View at: Publisher Site | Google Scholar
D. C. McClelland, “Testing for competence rather than for “intelligence”,” American Psychologist, vol. 28, no. 1, pp. 1–14, 1973.
View at: Publisher Site | Google Scholar
T. R. Athey and M. S. Orth, “Emerging competency methods for the future,” Human Resource Management, vol. 38, no. 3, pp. 215–225, 1999.
View at: Publisher Site | Google Scholar
R. Gibbons, “On competence: a critical analysis of competence-based reforms in higher education,” The Journal of Higher Education, vol. 51, no. 6, pp. 695–697, 1980.
View at: Publisher Site | Google Scholar
R. Kurz and D. Bartram, “Competency and individual performance: modelling the world of work,” in Organizational Effectiveness: The Role of Psychology, I. T. Robertson, M. Callinan, and D. Bartram, Eds., pp. 227–255, John Wiley & Sons, Ltd, Chichester, UK, 2002.
View at: Publisher Site | Google Scholar
D. Bartram, “The great eight competencies: a criterion-centric approach to validation,” Journal of Applied Psychology, vol. 90, no. 6, pp. 1185–1203, 2005.
View at: Publisher Site | Google Scholar
L. Ploum, V. Blok, T. Lans, and O. Omta, “Toward a validated competence framework for sustainable entrepreneurship,” Organization & Environment, vol. 31, no. 2, pp. 113–132, 2018.
View at: Publisher Site | Google Scholar
S. Krumm, J. Kanthak, K. Hartmann, and G. Hertel, “What does it take to be a virtual team player? The knowledge, skills, abilities, and other characteristics required in virtual teams,” Human Performance, vol. 29, no. 2, pp. 123–142, 2016.
View at: Publisher Site | Google Scholar
G. Hertel, U. Konradt, and K. Voss, “Competencies for virtual teamwork: development and validation of a web-based selection tool for members of distributed teams,” European Journal of Work and Organizational Psychology, vol. 15, no. 4, pp. 477–504, 2006.
View at: Publisher Site | Google Scholar
C. C. Cogliser, W. L. Gardner, M. B. Gavin, and J. C. Broberg, “Big five personality factors and leader emergence in virtual teams,” Group & Organization Management, vol. 37, no. 6, pp. 752–784, 2012.
View at: Publisher Site | Google Scholar
T. J. Maurer and M. Lippstreu, “Expert vs. general working sample differences in KSAO improvability ratings and relationships with measures relevant to occupational and organizational psychology,” Journal of Occupational and Organizational Psychology, vol. 81, no. 4, pp. 813–829, 2008.
View at: Publisher Site | Google Scholar
C. Prahalad and G. Hamel, “The core competence of the corporation,” Strategic Learning in a Knowledge Economy, Routledge, Abingdon, UK, 2000.
View at: Publisher Site | Google Scholar
R. Boyatzis, The Competent Manager: A Model for Effective Performance, Wiley, Hoboken, NJ, USA, 1982.
W.-W. Wu and Y.-T. Lee, “Developing global managers’ competencies using the fuzzy DEMATEL method,” Expert Systems with Applications, vol. 32, no. 2, pp. 499–507, 2007.
View at: Publisher Site | Google Scholar
C. Lucas, R. A. Nielsen, M. E. Roberts, B. M. Stewart, A. Storer, and D. Tingley, “Computer-assisted text analysis for comparative politics,” Political Analysis, vol. 23, no. 2, pp. 254–277, 2015.
View at: Publisher Site | Google Scholar
D. Blei, L. Carin, and D. Dunson, “Probabilistic topic models,” IEEE Signal Processing Magazine, vol. 27, no. 6, pp. 55–65, 2010.
View at: Publisher Site | Google Scholar
K. Chen, G. Kou, J. Shang, and Y. Chen, “Visualizing market structure through online product reviews: integrate topic modeling, TOPSIS, and multi-dimensional scaling approaches,” Electronic Commerce Research and Applications, vol. 14, no. 1, pp. 58–74, 2015.
View at: Publisher Site | Google Scholar
R.-S. Wu and P.-H. Chou, “Customer segmentation of multiple category data in e-commerce using a soft-clustering approach,” Electronic Commerce Research and Applications, vol. 10, no. 3, pp. 331–341, 2011.
View at: Publisher Site | Google Scholar
U. H. Graneheim and B. Lundman, “Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness,” Nurse Education Today, vol. 24, no. 2, pp. 105–112, 2004.
View at: Publisher Site | Google Scholar
A. Ansari, Y. Li, and J. Z. Zhang, “Probabilistic topic model for hybrid recommender systems: a stochastic variational bayesian approach,” Marketing Science, vol. 37, no. 6, pp. 987–1008, 2018.
View at: Publisher Site | Google Scholar
Y. Lu, P. Zhang, J. Liu, J. Li, and S. Deng, “Health-related hot topic detection in online communities using text clustering,” PLoS One, vol. 8, no. 2, Article ID e56221, 2013.
View at: Publisher Site | Google Scholar
J. Mou, G. Ren, C. Qin, and K. Kurcz, “Understanding the topics of export cross-border e-commerce consumers feedback: an LDA approach,” Electronic Commerce Research, vol. 19, no. 4, pp. 749–777, 2019.
View at: Publisher Site | Google Scholar
B. Cao, X. Liu, J. Liu, and M. Tang, “Domain-aware mashup service clustering based on LDA topic model from multiple data sources,” Information and Software Technology, vol. 90, pp. 40–54, 2017.
View at: Publisher Site | Google Scholar
W. Wang, Y. Feng, and W. Dai, “Topic analysis of online reviews for two competitive products using latent Dirichlet allocation,” Electronic Commerce Research and Applications, vol. 29, pp. 142–156, 2018.
View at: Publisher Site | Google Scholar
F. Al-Obeidat, B. Spencer, and E. Kafeza, “The opinion management framework: identifying and addressing customer concerns extracted from online product reviews,” Electronic Commerce Research and Applications, vol. 27, pp. 52–64, 2018.
View at: Publisher Site | Google Scholar
K. Bastani, H. Namavari, and J. Shaffer, “Latent Dirichlet allocation (LDA) for topic modeling of the CFPB consumer complaints,” Expert Systems with Applications, vol. 127, pp. 256–271, 2019.
View at: Publisher Site | Google Scholar
D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent Dirichlet allocation,” Journal of Machine Learning Research, vol. 3, no. 4-5, pp. 993–1022, 2003.
View at: Publisher Site | Google Scholar
J. Chang, S. Gerrish, C. Wang, J. Boydgraber, and D. M. Blei, “Reading tea leaves: how humans interpret topic models,” in Proceedings of the 22nd International Conference on Neural Information Processing Systems, pp. 288–296, Vancouver, Canada, December 2009.
View at: Publisher Site | Google Scholar
J. Rezaei, “Best-worst multi-criteria decision-making method,” Omega, vol. 53, pp. 49–57, 2015.
View at: Publisher Site | Google Scholar
J. Rezaei, “Best-worst multi-criteria decision-making method: some properties and a linear model,” Omega, vol. 64, pp. 126–130, 2016.
View at: Publisher Site | Google Scholar
J. Rezaei, “A concentration ratio for nonlinear best worst method,” International Journal of Information Technology & Decision Making, vol. 19, no. 3, pp. 891–907, 2020.
View at: Publisher Site | Google Scholar
A. J. Bradley, “Training in the mitigation of anchoring bias: a test of the consider-the-opposite strategy,” Learning and Motivation, vol. 53, pp. 36–48, 2016.
View at: Publisher Site | Google Scholar
X. Mi, M. Tang, H. Liao, W. Shen, and B. Lev, “The state-of-the-art survey on integrations and applications of the best worst method in decision making: why, what, what for and what’s next?” Omega, vol. 87, pp. 205–225, 2019.
View at: Publisher Site | Google Scholar
Sharing Economy Research Center, “China Sharing Economy Development Report 2020,” State Information Center, Beijing, China, 2020, https://www.ndrc.gov.cn/xxgk/jd/wsdwhfz/202003/t20200310_1222769.html.
View at: Google Scholar
X. Tian, G. Kou, and W. Zhang, “Geographic distance, venture capital and technological performance: evidence from Chinese enterprises,” Technological Forecasting and Social Change, vol. 158, Article ID 120155, 2020.
View at: Publisher Site | Google Scholar
N. Aletras and M. Stevenson, “Measuring the similarity between automatically generated topics,” in Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pp. 22–27, Stroudsburg, PA, USA, April 2014.
View at: Publisher Site | Google Scholar
D. Ramage, D. Hall, R. Nallapati, and C. D. Manning, “Labeled LDA: a supervised topic model for credit attribution in multi-labeled corpora,” in Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP 2009): A Meeting of SIGDAT, a Special Interest Group of ACL, pp. 248–256, Singapore, August 2009.
View at: Google Scholar
C. M. Beckman, M. D. Burton, and C. O’Reilly, “Early teams: the impact of team demography on VC financing and going public,” Journal of Business Venturing, vol. 22, no. 2, pp. 147–173, 2007.
View at: Publisher Site | Google Scholar
M. W. Staniewski, “The contribution of business experience and knowledge to successful entrepreneurship,” Journal of Business Research, vol. 69, no. 11, pp. 5147–5152, 2016.
View at: Publisher Site | Google Scholar
C. M. V. D. Heijde and B. I. J. M. Van Der Heijden, “A competence-based and multidimensional operationalization and measurement of employability,” Human Resource Management, vol. 45, no. 3, pp. 449–476, 2006.
View at: Publisher Site | Google Scholar
T. C. Sebora, S. M. Lee, and N. Sukasame, “Critical success factors for e-commerce entrepreneurship: an empirical study of Thailand,” Small Business Economics, vol. 32, no. 3, pp. 303–316, 2009.
View at: Publisher Site | Google Scholar
A. Van Riel, J. Semeijn, and W. Janssen, “E-service quality expectations: a case study,” Total Quality Management & Business Excellence, vol. 14, no. 4, pp. 437–450, 2003.
View at: Publisher Site | Google Scholar
S. Chorev and A. R. Anderson, “Success in Israeli high-tech start-ups; critical factors and process,” Technovation, vol. 26, no. 2, pp. 162–174, 2006.
View at: Publisher Site | Google Scholar
A. K. Kohli and B. J. Jaworski, “Market orientation: the construct, research propositions, and managerial implications,” Journal of Marketing, vol. 54, no. 2, pp. 1–18, 1990.
View at: Publisher Site | Google Scholar
S. F. Slater and J. C. Narver, “Market orientation and the learning organization,” Journal of Marketing, vol. 59, no. 3, pp. 63–74, 1995.
View at: Publisher Site | Google Scholar
V. Kaushik, A. Khare, R. Boardman, and M. B. Cano, “Why do online retailers succeed? The identification and prioritization of success factors for Indian fashion retailers,” Electronic Commerce Research and Applications, vol. 39, Article ID 100906, 2020.
View at: Publisher Site | Google Scholar
H. Gil-Gomez, V. Guerola-Navarro, R. Oltra-Badenes, and J. A. Lozano-Quilis, “Customer relationship management: digital transformation and sustainable business model innovation,” Economic Research-Ekonomska Istraživanja, vol. 33, no. 1, pp. 2733–2750, 2020.
View at: Publisher Site | Google Scholar
C. Veloutsou and C. Ruiz Mafe, “Brands as relationship builders in the virtual world: a bibliometric analysis,” Electronic Commerce Research and Applications, vol. 39, Article ID 100901, 2020.
View at: Publisher Site | Google Scholar
S. Wang, Y. Hong, N. Archer, and Y. Wang, “Modeling the success of small and medium sized online vendors in business to business electronic marketplaces in China,” Journal of Global Information Management, vol. 19, no. 4, pp. 45–75, 2011.
View at: Publisher Site | Google Scholar
G. N. Chandler, B. Honig, and J. Wiklund, “Antecedents, moderators, and performance consequences of membership change in new venture teams,” Journal of Business Venturing, vol. 20, no. 5, pp. 705–725, 2005.
View at: Publisher Site | Google Scholar
R. Wentrup, “The online–offline balance: internationalization for Swedish online service providers,” Journal of International Entrepreneurship, vol. 14, pp. 562–594, 2016.
View at: Publisher Site | Google Scholar
Y. Zhao, Y. Zhao, X. Yuan, and R. Zhou, “How knowledge contributor characteristics and reputation affect user payment decision in paid Q&A? An empirical analysis from the perspective of trust theory,” Electronic Commerce Research and Applications, vol. 31, pp. 1–11, 2018.
View at: Publisher Site | Google Scholar
C. Gieure and I. Buendía-Martínez, “Determinants of translation-firm survival: a fuzzy set analysis,” Journal of Business Research, vol. 69, no. 11, pp. 5377–5382, 2016.
View at: Publisher Site | Google Scholar
Z. Hasnain, B. Yvonne, D. John, and E. Mahesh, “Straight from the horse’s mouth: founders’ perspectives on achieving “traction” in digital start-ups,” Computers in Human Behavior, vol. 95, pp. 262–274, 2019.
View at: Publisher Site | Google Scholar
C. E. Helfat and M. B. Lieberman, “The birth of capabilities: market entry and the importance of pre-history,” Industrial and Corporate Change, vol. 11, no. 4, pp. 725–760, 2002.
View at: Publisher Site | Google Scholar
K. Frey, C. Lüthje, and S. Haag, “Whom should firms attract to open innovation platforms? The role of knowledge diversity and motivation,” Long Range Planning, vol. 44, no. 5-6, pp. 397–420, 2011.
View at: Publisher Site | Google Scholar
J. Mo, S. Sarkar, S. Sarkar, and S. Menon, “Know when to run: recommendations in crowdsourcing contests,” MIS Quarterly, vol. 42, no. 3, pp. 919–944, 2018.
View at: Publisher Site | Google Scholar
X. Tian, M. Niu, W. Zhang, L. Li, and E. Herrera-Viedma, “A novel todim based on prospect theory to select green supplier with Q-rung orthopair fuzzy set,” Technological and Economic Development of Economy, vol. 27, no. 2, pp. 284–310, 2020.
View at: Publisher Site | Google Scholar
M. Batey and A. Furnham, “The relationship between measures of creativity and schizotypy,” Personality and Individual Differences, vol. 45, no. 8, pp. 816–821, 2008.
View at: Publisher Site | Google Scholar
G. J. Feist, “A meta-analysis of personality in scientific and artistic creativity,” Personality and Social Psychology Review, vol. 2, no. 4, pp. 290–309, 1998.
View at: Publisher Site | Google Scholar
F. Buttle, Customer Relationship Management: Concepts and Tools, Elsevier Butterworth-Heinemann, Oxford, UK, 2004.
C. Claycomb, C. Dröge, and R. Germain, “The effect of just‐in‐time with customers on organizational design and performance,” The International Journal of Logistics Management, vol. 10, no. 1, pp. 37–58, 1999.
View at: Publisher Site | Google Scholar
S. Aggarwal, “Flexibility management: the ultimate strategy,” Industrial Management, vol. 39, no. 1, pp. 5–14, 1997.
View at: Google Scholar
L. Y. M. Sin, A. C. B. Tse, and F. H. K. Yim, “CRM: conceptualization and scale development,” European Journal of Marketing, vol. 39, no. 11-12, pp. 1264–1290, 2005.
View at: Publisher Site | Google Scholar
S. E. S. Dijkstra, Upwork at Work: Labor as A Service: A Transformation of Labor in the Platform Society, Utrecht University, Utrecht, Netherlands, 2017.
K. L. Guth and D. C. Brabham, “Finding the diamond in the rough: exploring communication and platform in crowdsourcing performance,” Communication Monographs, vol. 84, no. 4, pp. 510–533, 2017.
View at: Publisher Site | Google Scholar
C. E. Shannon and W. Weaver, The Mathematical Theory of Communication, University of Illinois Press, Champaign, IL, USA, 1949.
M. Allahbakhsh, B. Benatallah, A. Ignjatovic, H. R. Motahari-Nezhad, E. Bertino, and S. Dustdar, “Quality control in crowdsourcing systems: issues and directions,” IEEE Internet Computing, vol. 17, no. 2, pp. 76–81, 2013.
View at: Publisher Site | Google Scholar
C. Stasz and D. J. Brewer, Academic Skills at Work: Two Perspectives, 1999.
N. Luz, N. Silva, and P. Novais, “A survey of task-oriented crowdsourcing,” Artificial Intelligence Review, vol. 44, no. 2, pp. 187–213, 2015.
View at: Publisher Site | Google Scholar
S. Basu Roy, I. Lykourentzou, S. Thirumuruganathan, S. Amer-Yahia, and G. Das, “Task assignment optimization in knowledge-intensive crowdsourcing,” The VLDB Journal, vol. 24, no. 4, pp. 467–491, 2015.
View at: Publisher Site | Google Scholar
L. B. Erickson and E. M. Trauth, “Getting work done: evaluating the potential of crowdsourcing as a model for business process outsourcing service delivery,” in Proceedings of the 2013 Annual Conference on Computers and People Research, pp. 135–140, Cincinnati, OH, USA, May 2013.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2021 Biyu Yang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

686

Downloads

871

Citations

Complexity

Machine Learning Applications in Complex Economics and Financial Networks

Understanding Service Providers’ Competency in Knowledge-Intensive Crowdsourcing Platforms: An LDA Approach

Abstract

1. Introduction

2. Related Works

2.1. Knowledge-Intensive Crowdsourcing

2.2. Competency Analysis

3. Methodology

3.1. Topic Modelling

3.2. Best-Worst Method

4. Data Analysis and Results

4.1. Data Collection and Preprocessing

4.2. Competency Dimensions Identification

4.3. The KSAT Competency Model

4.4. Weights of Competency Criteria

5. Discussion

5.1. Results Analysis Related to KSAT Competency Model

5.2. Results Analysis Related to Competency Importance

6. Conclusion

Data Availability

Conflicts of Interest

Acknowledgments

References

Copyright