The Nonlinear Impact of Task Rewards and Duration on Solvers’ Participation Behavior: A Study on Online Crowdsourcing Platform

Abstract

Crowdsourcing has attracted significant attention in the past decade because it has more competitive advantages than traditional methods for mobilizing distributed labor and utilizing innovation. Crowdsourcing contests are one of the most popular and effective crowdsourcing modes. Reasonable task rewards and duration are the key factors for seekers to attract solvers who can efficiently participate in the crowdsourcing contest task. Previous studies have mainly focused on task results to analyze solvers’ participation behavior in crowdsourcing contests, but have paid little attention to the task process, and there have been conflicting conclusions regarding the impact of task rewards and duration on solvers’ participation behavior and the performance of crowdsourcing contests. In view of this gap, this study collected 2706 logo design task data points from 2015–2017 on an online crowdsourcing platform and measured the performance of solvers’ participation behavior in two stages. The participation time was used to represent the performance of solvers’ participation behavior in the task process, while the number of submissions of solutions was used to represent the performance of participation behavior in the task result. The results show that task rewards and duration have an inverted U-shaped effect on the number of submissions, money rewards have a positive impact on participation time, and duration has an inverted U-shaped relationship with participation time. This study proposes the nonlinear effects of task rewards and duration on participation behavior and explains the reason for the conflicting results of previous studies. This paper also expands upon existing research by using solvers’ participation time in the task process to measure the performance of solvers’ participation behavior.

1. Introduction

Crowdsourcing is a process in which tasks traditionally performed by internal employees are outsourced to the public through open collection [1]. One of the most popular and effective models of crowdsourcing is the contest. The crowdsourcing contest process contains three steps: first, a seeker issues a task and specifies the task requirements, duration, and rewards; second, solvers submit solutions to solve the problem or task issued by the seeker; third, the seeker evaluates the solutions, identifies the best solution, and pays rewards to the solver whose solution is selected [2].

Previous studies have found that there are many features in crowdsourcing contests that affect the participation of solvers. From the perspective of task design characteristics, the main influencing factors include task rewards [3], task duration [4], task description [4], and difficulty level [5]. In terms of environmental factors, these include contest intensity, market price [5], and task seekers’ characteristics [6]. Regarding solvers’ characteristics, self-knowledge and experience are also the influencing factors [7,8]. For seekers, the factors they need to focus on are the task rewards and duration, which are closely related to the benefits and costs for task seekers to solve problems. The goal of a seeker is to maximize solution innovation while minimizing task rewards and duration, which have effects on contest performance as measured by the number of submissions and the level of solvers’ participation efforts. If the task rewards in a crowdsourcing contest are too high, this will increase costs for seekers and thereby affect seekers’ returns. If the task duration is too long, it will increase seekers’ time costs, which is not conducive to saving working time and also affects seekers’ returns. For solvers, there are different conclusions about the effects of task rewards and duration on their participation. For example, Shao et al. (2012) found that task rewards and duration have positive effects on solvers’ participation [5]. However, Walter and Back (2011) found that task rewards will adversely affect contest performance [9]. Korpeoglu et al. (2017) argued that as the task duration increases, seekers’ returns will decrease, and the likelihood of solvers’ participation will be reduced [10].

The majority of the literature mainly analyzes the influence of various factors on solvers’ participation behavior in task results, where solvers’ participation behavior is measured by the number of submissions, but few studies focused on the influence of different factors on solvers’ participation behavior during the task process. Moreover, existing studies do not present consistent conclusions regarding the influence of task rewards and duration on solvers’ participation behavior. Some scholars believe that task rewards and duration positively affect solvers’ participation behavior, while others have reached the opposite conclusion. The reason for this divergence may be due to the fact that existing studies only examine the linear relationship between task rewards and duration and solvers’ participation behavior, but fail to consider the nonlinear relationship between them.

According to these, this study makes three contributions. Firstly, it contributes to the literature on the effects of task rewards and duration by dividing a crowdsourcing contest into two stages—task process and task result—and solvers’ behavior into participation and submissions. Secondly, it verifies that task rewards and task duration have inverted U-shaped relationships with the number of submissions, which explains the conflicting conclusions of previous studies. Thirdly, this study introduced participation time into research models, which indicates the degree of solvers’ participation efforts. This incorporates the impact of task rewards and duration on participation time into the studies on solvers’ participation, and verifies the differential impact of task rewards and duration on solvers’ participation behavior.

This article consists of four main parts. First, a literature review is presented summarizing studies on the effects of task rewards and duration on solvers’ participation. Then, based on the theories of expected value and self-efficacy evaluation, two models are constructed to analyze how task rewards and duration affect task submissions and solvers’ average time in solving tasks (participation time is used below). After this, the methodology of the sample data collection from Taskcn.com and its empirical analysis are presented, along with robustness checks for the two models. Finally, the research results are described.

2. Literature Review

This section reviews existing studies on the effects of rewards on solvers’ behavior. Research on rewards effects was studied even before the emergence of crowdsourcing contests. For example, Hars and Ou (2001) did a pre-crowdsourcing contests study and found that one of the motivations for developers to participate in open-source projects is to obtain the expected cash rewards [11]. With the emergence of crowdsourcing contests, corresponding research has also grown. Dipalantino and Vojnovic (2009) found that task rewards have a logarithmic diminishing effect on solvers’ participation [12]. Walter and Back (2011) conducted an empirical study of online creative contests and built a model to analyze the effects of rewards, duration, and other external factors on the contest results. They determined that task rewards have an adverse effect on the outcome of the contest [9]. Chen et al. (2010) empirically concluded that an increase in task rewards would not increase the number of submissions [13]. Brabham (2010) found that solvers’ main motivations for participating in crowdsourcing contests were obtaining rewards, improving their abilities, or building social relationships [14]. Martinez and Walton (2014) indicated that task rewards even inhibit participation if not properly set [15].

However, several other studies have reached different conclusions. For example, Rich et al. (2010) proposed that an increase in task rewards can further stimulate solvers’ motivations and efforts [16]. Liu et al. (2014) divided the participation process of crowdsourcing contests into two stages: registration and submission. By implementing a random field experiment through Taskcn.com (Taskcn.com is one of the biggest online crowdsourcing platforms in China, and was founded in 2006), they found that the higher the task rewards, the greater the registrations and submissions in both stages [17]. Meng et al. (2014) used the technology acceptance model to analyze the influence of expected income and other factors on solvers’ participation willingness and behavior in crowdsourcing contests. The results demonstrated that the expected income of solvers has a significantly positive impact on their participation willingness and behavior [18]. By using the data from Taskcn.com, Li and Hu (2017) established a two-equation model based on expectation theory, and explored the impact of task rewards and contest intensity on solvers’ participation. The results confirmed that task rewards are positively correlated with registrations and submissions [19]. Thus, there are differences in the impacts of rewards on solvers’ participation, and higher task rewards do not necessarily lead to better contest performance.

Task duration is also an important factor that researchers should take into consideration. Dipalantino and Vojnovic (2009) pointed out that task rewards are not the only factor affecting the degree of participation [12], and Yang et al. (2009) illustrated that a longer task duration attracts greater participation by solvers [20]. Shao et al. (2012) came to the same conclusion [5]. Wang et al. (2014) studied the influencing factors of innovation contest performance in China’s network innovation contest community, positing that setting a longer duration could increase solution submissions [21]. However, some authors observed different results. Lang et al. (2014) argued that a shorter duration reduces solvers’ solving time, but increases the intensity of the contest in solvers, noting that seekers can achieve higher returns in a shorter period of time [22]. Dong et al. (2016) used data from Taskcn.com and analyzed the impact of task duration on the performance of crowdsourcing contests, finding that with increases in task duration, the effort costs of solvers became higher than the rewards, thereby reducing the number of submissions [23]. Korpeoglu et al. (2017) found that there is an optimal task duration for seekers to set: a longer duration increases the workload of solvers, which improves the quality of solutions and increases seekers’ revenue. At the same time, as task duration increases, solvers may have to work harder, which reduces their likelihood of participating and decreases seekers’ returns [10]. There are also different findings in the impact of task duration on solvers’ participation.

In terms of solvers’ participation behavior, seekers pay more attention to the performance of solvers’ participation (such as participation effort, energy input, etc.) during the task process, as well as to the results of solvers’ submissions (number of submissions, solution quality, etc.). As mentioned earlier, most of the research on participation behavior involves the number of submissions. Most researchers have measured participation effort by using subjective questionnaires. For example, Ke and Zhang (2009) designed eight motivation-related indicators in the questionnaire and found that intrinsic motivation can improve levels of participation effort [24]. Huang et al. (2012) found that the policy simulation process reveals a trade-off between task rewards utility and the probability of winning. When task rewards increase, if the utility of increasing task rewards is higher than the utility loss of decreasing winning probability, solvers are more likely to participate, and seekers receive higher revenue [25]. Zhao and Zhu (2014) used the theory of self-determination to analyze the role of various motivations in crowdsourcing contests, and found that external motivation is positively correlated with participation effort levels [26]. Sun (2015) performed a nonlinear analysis of the relationship between incentives and effort, determining that rewards have a positive impact on effort [27]. Liang et al. (2018) explained how external incentives and intrinsic motivations jointly influence task effort and adjust participation effort in crowdsourcing contests by establishing an intermediary adjustment model, discovering that both intrinsic motivation and external motivation increase solvers’ efforts. At the same time, external incentives weaken the impact of intrinsic motivation on participation efforts [28].

In order to clearly compare the relevant studies summarized above,

Table 1. Summary of findings of related studies.

Classification	Related Literature	Research Conclusions
The impact of rewards on solvers’ participation is negative or not significant	12	Rewards will have a logarithmic diminishing effect on the level of participation.
	9	Rewards will adversely affect the performance of crowdsourcing contest.
	13	The increase in task rewards does not increase the number of submissions.
	15	If set incorrectly, monetary rewards may even inhibit participation.
Rewards positively affect solvers’ participation	16	Increased task rewards can further stimulate solvers’ motivation.
	5,20,21	A higher reward amount is good for attracting solvers to participate in a contest.
	17	The higher the rewards, the higher the number of solvers in the two-stage registration and submission.
	18	Expected earnings have a positive impact on solvers’ participation willingness.
Task duration positively affects solvers’ participation	5,20,21	The longer the task duration, the more solvers will be attracted.
The impact of task duration on solvers’ participation is negative or not significant	22	Seekers can obtain higher effort in a shorter period of time.
	23	The longer the task lasts, the less likely solvers’ participation is, which leads to high-quality solvers’ withdrawing from the crowdsourcing contest.
	10	As the task duration increases, seekers’ returns decrease, and longer game durations may reduce the likelihood of solvers’ participation.
Solvers’ participation behavior: the degree of participation effort	24	Intrinsic motivation can improve the level of participation effort.
	25	Solvers have a trade-off between increasing the probability of winning and increasing costs as rewards increase.
	26	External motivation in crowdsourcing contests is positively related to the level of participation effort.
	27	Rewards have a positive impact on participation effort.
	28	Both intrinsic motivation and external incentives will increase solvers’ participation effort in crowdsourcing contests.

Source: this table was compiled by the authors according to the literature review above.

presents the various findings. These studies analyzed the impact factors of task rewards and duration on solvers’ participation behavior, but present different conclusions; this may be because the theoretical hypotheses of these studies are based on linear relationships. Non-linear relationships between the two factors and solvers’ participation behavior have not yet been taken into consideration. More importantly, the impact of task duration on solvers’ participation efforts has rarely been addressed. The success of a crowdsourcing contest depends not only on the number of solvers participating, but also on the effort they decide to invest. Based on the analyses of existing studies, solvers’ participation behavior is analyzed in this article from two angles. The first is the solution submission number, which denotes the number of solvers who participate in a contest. Additionally, solvers’ participation effort is quantified by using another variable, participation time, to compensate for the deficiencies of the questionnaire used in previous studies. Finally, in order to fill the gap in existing literature, this study focuses on the non-linear effects of task rewards and duration on the number of submissions and participation time of solvers.

3. Basic Theories

According to the literature review, task rewards and duration affect solvers’ participation behavior by influencing their participation motivations. There are two kinds of motivations for solvers: extrinsic and intrinsic. For extrinsic motivation, the expected value theory is an important basic theory. Because solvers who participate in crowdsourcing contests are not guaranteed to obtain a certain reward, it is the expectation value that simulates solvers to make a participation decision. Intrinsic motivation can be explained according to the self-efficacy assessment from cognitive theory. When solvers make decisions by analyzing the impact of external factors on their own earnings, they also need to assess whether their ability is suitable for the tasks within a certain timeframe and whether their efforts can be fully rewarded.

3.1. Expectation Value Theory

Expectation value theory is one of the most popular theories in motivation psychology. In expectation value theory, the motivation of an individual to solve various tasks is determined by their expectation of the probability of winning and the value assigned to the task. If one solver believes that he or she is more likely to succeed in solving the task and thus can obtain higher revenue expectations, his or her motivation to solve the task will be stronger. Atkinson (1957) initially defined expectations as the individual expected probability of winning and value as the attraction of a task’s success. Using these definitions, he established a theoretical model to analyze how the motivation for success and the motivation to avoid failure affect participation behavior. He believed that an individual’s decision to participate in a task is determined by motivation, expectations, and rewards [29]. Vroom (1964) argued that a person’s behavior is related to expected value, the probability of success, and the outcome that can be achieved [30]. Wigfield and Eccles (1992) further expanded this area of research and discussed the effects of expectation value or success on the motivation and performance of participation behavior [31]. Shepperd (1999) held that goal-oriented behavior is a function of expectancy, instrumentality, and outcome value [32]. Nagengast et al. (2011) found that expectancy value can adjust the relationship between task rewards and participation behavior [33].

In the field of crowdsourcing contests, expectation value theory is mainly used to explain how an individual decides to participate in a task largely based on expected value. Shah and Higgins (1997) suggested that the level of incentives for task rewards often depends on the level of expectations that participants expect from rewards. When the expected value is high, even if the amount of task rewards is small, the incentive level will increase, meaning that solvers’ enthusiasm and effort input level will also increase [34]. Wigfield et al. (2016) also found that it is the expectation value that actually motivates greater effort, not the absolute rewards. If the expected value is low, even if the task rewards are high, the enthusiasm of the decision-maker for participating in the task decreases [35]. Li and Hu (2017) argued that in crowdsourcing contests, task rewards are the most important tangible compensation for solvers’ time and efforts [19].

Thus, the level of expected value usually indicates the incentive level of task rewards for solvers. In crowdsourcing contests, when the expected value of solvers is higher, the incentive level of task rewards increases, and solvers’ enthusiasm and effort also increase as a result. If the expected value is lower, solvers’ enthusiasm about participating in a task declines. To the point that the expected value is lower than the cost of solving the task, the incentive level is minimized, and the number of solvers in crowdsourcing contests decreases. Therefore, the reward amount does not necessarily increase the number of submissions, and expected value should also be considered.

3.2. Self-Efficacy Assessment Theory

Self-efficacy is a core concept of social cognitive theory. Self-efficacy assessment is the assessment and cognition of individuals or organizations regarding their own abilities. The results of these assessments directly affect individual behavioral motivation [36]. Schunk (1990) found that people with high self-efficacy are more likely to accomplish tasks and persist in the task for a longer duration of time than those with low self-efficacy [37]. Stajkovic and Luthans (1998) used self-efficacy assessment theory to explain the causes of behavioral motivations in specific situations. The level of self-efficacy assessment determines whether an individual can demonstrate expected behavior and how long an individual can continue in the event of difficulty. In general, individuals with high self-efficacy will work harder, and are more likely to have successful outcomes, while those with low self-efficacy may stop trying and fail early [38]. Luszczynska and Schwarzer (2005) argued that self-efficacy assessment affects individual behavioral choices as well as motivation [39]. Bandura and Locke (2003) forwarded that high self-efficacy assessments motivate solvers to work harder, indicating a positive correlation between self-efficacy and effort [36]. However, Mann and Eland (2005) maintained that individuals with high self-efficacy spend less time on tasks than those with lower self-efficacy, suggesting a negative correlation between self-efficacy assessment and participation effort [40]. Similarly, Vancouver and Kendall (2006) agreed that self-efficacy is negatively correlated with motivation [41]. Shao et al. (2012) argued that in crowdsourcing contests, task rewards and duration affect correct and reasonable self-efficacy assessment among solvers [5]. Sun et al. (2015) combined the expectation value theory with self-efficacy assessment theory and proposed a nonlinear relationship between self-efficacy and effort. The results showed that task rewards have a positive impact on effort, but when the task becomes more complicated, the self-efficacy assessment has a convex relationship with effort [27]. In addition, the self-efficacy assessment of a solver varies according to the type of task, as well as the effort. For example, solvers will work harder and spend more time for a deliberate task, while a creative task usually have little relationship with time investment of solvers [42].

Thus, different task designs will result in different self-efficacy assessments from solvers. Different self-efficacy assessments lead to different time, energy, and effort investments, which produce different effects on solvers’ participation behavior. As such, simply increasing task rewards and duration does not necessarily improve solvers’ self-efficacy assessments or participation effort.

4. Research Model and Hypotheses

4.1. Research Model

Solvers’ participation in a crowdsourcing contest can be divided into three phases: task registration, task process, and task result (see Figure 1). The research objects of this paper include solvers who have already registered for a contest. Therefore, this study mainly focuses on the impact of task rewards and duration on solvers’ participation behavior in the latter two stages.

Based on the theories outlined above, this article argues that solvers perform cost–benefit analysis by assessing expected value and self-efficacy, after which they make non-linear choices. As such, a non-linear function of task rewards and duration for participation behavior was constructed (see Figure 2). Using the same task, the effects of the difference in task rewards and duration on the number of submissions and participation time are explored.

4.2. Research Hypotheses

The number of submissions reflects solvers’ participation, while the level of task rewards—which relates to expected value—determines whether solvers decide to participate. According to expectation value theory, a solver’s expected value is positively correlated with the task rewards set by seekers. The higher the task rewards, the higher the expected reward of solvers, and the higher the submission number [27]. This interpretation is consistent with most empirical tests, but it ignores whether task rewards affect the expected value in another way. As discussed, solvers are motivated by expected value rather than the total amount of task rewards [35]. Expected value is also related to winning probability. Therefore, task rewards cannot be equated with the expected value of participation and bidding. Glazer and Hassin (1988) noted that high task rewards attract more solvers to participate, but the probability of successfully bidding then decreases because of the increased competition [43]. Yang and Saremi (2015) contended that task rewards play a dual role in solvers’ participation decisions. On the one hand, a higher reward amount will increase solvers’ enthusiasm and make them more willing to register and compete. On the other hand, higher prices often denote significant and more complex work, which will reduce the number of solvers who are able to provide adequate solutions [44]. Xu and Zhang (2018) found that, due to limited ability, solvers cannot be motivated infinitely in the same task. As rewards increase, solvers’ expected value will first increase and then decrease [45].

Based on this analysis, expectation value theory and self-efficacy assessment theory can explain the effects of task rewards on submission numbers. With the increase of task rewards, the expected value of solvers, as well as the number of submissions, also increase. If the rewards are increased continually, solvers’ submissions will rise, and competition will be more intense. Until the task rewards exceed a given threshold, solvers may not participate in the contest because the increase in the degree of competition lowers winning probability and expected value. Thus, excessive task rewards reduce the number of submissions. Furthermore, in crowdsourcing contests, solvers need to assess self-efficacy and make decisions about whether to participate. The level of task rewards often reflects seekers’ task requirements. According to self-efficacy evaluation theory, if the rewards increase beyond a certain range, some solvers with low self-efficacy assessment are squeezed out. Therefore, the relationship between task rewards and submission may be an inverted U-shape, hence Hypothesis 1:

Hypothesis 1.

Task rewards have an inverted U-shaped relationship with the number of submissions.

Participation time reflects the efforts of solvers who have chosen to participate in the task. Task rewards not only affect the number of submissions, but also the level of solvers’ efforts during the task process. The more efforts solvers make, the higher quality the solutions have. Levels of effort are affected by the state of participation, which is a psychological state in which a solver is willing to invest their own energy for the purpose of accomplishing a task [46]. Solvers are usually interest-oriented [47], meaning they have an extrinsic motivation to pursue higher economic returns. This motivation prompts solvers to continue to focus on tasks and invest more time and energy [27]. In order to increase the probability of receiving task rewards, solvers who are in the participation stage have the motivation to contribute more time and energy to improving the quality of the solution, and therefore increase the attention given to the task. Task rewards stimulate this motivation [16].

Thus, this study uses participation time to indicate effort level, and argues that solvers in the participation state will invest more time to increase the probability of successful bidding and obtain more revenue. However, there may only be a positive linear relationship between participation time and task rewards. Since the solver has chosen to participate in the contest by the participation stage, the extrusion effect of self-efficacy would not occur in this situation. Therefore, the second hypothesis is proposed as follows.

Hypothesis 2.

Task rewards positively affect solvers’ participation time.

Solvers’ participation in crowdsourcing contests is a dynamic random process. The dynamic random process is similar to a competitive auction. The longer the auction is open, the more bidders can be included in the auction [48]. In crowdsourcing contests, the longer the task is released, the more solvers will see the tasks posted and participate, thereby increasing the number of submissions [5,20]. At the same time, the longer the task duration, the more solvers will be able to participate, because most solvers use their spare time to participate in crowdsourcing contests [49]. In terms of task quality, solvers cannot solve a task in a limited time. Many solvers are more willing to participate in crowdsourcing contests with longer task duration. As such, short task duration is not conducive to seekers who desire high-quality solutions.

However, a short duration is beneficial to solvers. According to expected value theory, solvers’ participation in crowdsourcing contests is considered as trades. Solvers’ ultimate goal is to obtain the highest expected return at the lowest cost and to recover the input cost in a short period. In short-term tasks, solvers can obtain results in a short time after the submission stage and achieve the expected value with a certain probability, which could compensate for solvers’ time and labor costs. If the duration is too long, it will increase solvers’ input costs because of its longer attention and higher time costs. In addition, there is also the opportunity cost for solvers in that investing in a task requires sacrificing the opportunity to obtain other benefits. If the expected value is lower than the opportunity cost for a certain duration of time, solvers will renounce participating in the task [50]. Korpeoglu et al. (2017) argued that a longer task duration increases solvers’ workload and improves the quality of solutions, which increases seekers’ profitability. However, as the task duration increases, solvers’ participation reduces, and seekers’ returns are likely to decrease [10]. Furthermore, solvers’ trust in seekers will decrease over time, motivation for participation will drop, and solutions will not be submitted; the loss of trust occurs because, in crowdsourcing contests, there have been cases of seekers using the solutions submitted by solvers but refusing to pay the rewards. Solvers therefore have less trust in a long-term contest [27].

Accordingly, this study maintains that on the one hand, task duration can dynamically and objectively encourage more solvers to participate depending on the time allotted and obtain more task solutions. However, once the task duration reaches a certain threshold, a crowding effect causes the number of submissions to drop. Thus, Hypothesis 3 is proposed as follows:

Hypothesis 3.

Task duration has an inverted U-shaped relationship with the number of submissions.

According to expectation value theory, for solvers, the longer the task duration, the more time the solvers have to solve the task; this motivates solvers to improve the quality of their solutions to obtain a higher expected value. Thus, solvers invest more attention into a long-term task [50]. However, if the task duration is too long, solvers’ patience will decrease because of the long cost recovery period and high opportunity cost. Solvers feel that it is not necessary to invest too much time into a task with a lower expected value, thereby reducing participation time.

According to several relevant studies on self-efficacy assessment, solvers with high self-efficacy assessments have less participation time than solvers with low assessments [40]. For an individual, an increase in self-efficacy assessment can increase the level of effort invested compared to when the self-efficacy assessment is low. When self-efficacy is assessed beyond a certain limit, it reduces a solver’s effort [41,51]; this may explain why, when the task duration is short, solvers show low self-efficacy assessments because of insufficient information and high requirements to solve the task in a short time. However, this dynamic stimulates solvers to exert more effort in solving a task [27]. Nonetheless, as the task duration extends, the self-efficacy assessment to solve a task becomes higher, and the expectation of the self-efficacy assessment exceeds the actual level of self-efficacy. For example, most people think they will present a high-quality solution if given a long time, which makes it unnecessary to take too much time to solve the task [52]. Therefore, Hypothesis 4 is proposed as follows.

Hypothesis 4.

Task duration has an inverted U-shaped relationship with participation time.

5. Research Methods

5.1. Regression Models

This study constructed two regression models to verify the four hypotheses. Since both task rewards and duration affect the number of submissions and participation time, task rewards and duration were included in the same regression model. According to Hypotheses 1 and 3, the number of submissions is a function of task rewards and duration, as suggested in Equation (1); ${\alpha }_1$ represents the coefficient of the ln(Rewards) term,${\alpha }_2$ represents the coefficient of the ln(Rewards) quadratic term, and ${\alpha }_3$ and ${\alpha }_4$ represent the first term and quadratic term coefficient of ln(Duration), respectively. Ln(x) is the natural logarithm of x. The sign of the coefficient can be used to analyze the relationship between the number of submissions and task rewards and duration. In addition, X indicates the vector of control variables, X = (interaction, description length, seekers’ credit, seekers’ experience, satisfaction rate, real name authentication, and email authentication).

$$\begin{array}{ll}\ln \left(\mathrm{Submissions}\right)=\hfill & {\alpha }_0+{\alpha }_1\ln \left(\mathrm{Reward}\right)+{\alpha }_2\ln \left(\mathrm{Reward}\right)\ast \ln \left(\mathrm{Reward}\right)\\ & +{\alpha }_3\ln \left(\mathrm{Duration}\right)+{\alpha }_4\ln \left(\mathrm{Duration}\right)\ast \ln \left(\mathrm{Duration}\right)+\gamma X+{\mu }_0\end{array}$$

(1)

Similarly, according to Hypotheses 2 and 4, the article ln(Participation time) was constructed as a function of ln(Rewards), ln(Duration), and their quadratic terms. In this regression model, the meaning of ${\beta }_1$,${\beta }_2$,${\beta }_3$,${\beta }_4$ is similar to ${\alpha }_1$,${\alpha }_2$,${\alpha }_3$,${\alpha }_4$, and X is the same vector of the control variables.

$$\begin{array}{ll}\ln \left(\mathrm{Participation} \mathrm{time}\right)=\hfill & {\beta }_0+{\beta }_1\ln \left(\mathrm{Reward}\right)+{\beta }_2\ln \left(\mathrm{Reward}\right)\ast \ln \left(\mathrm{Reward}\right)\\ & +{\beta }_3\ln \left(\mathrm{Duration}\right)+{\beta }_4\ln \left(\mathrm{Duration}\right)\ast \ln \left(\mathrm{Duration}\right)+\delta X+{\rho }_0\end{array}$$

(2)

In order to unify the dimensions, all independent and dependent variables were logged, which does not affect the sign of coefficients.

5.2. Variable Definition

There were three main types of variables involved in these models. See

Table 2. Description of the definitions of the main model variables.

Variable Types	Variable Name	Definition
Dependent variables	Number of submissions	The number of solutions submitted by the solvers.
	Participation time	The simple arithmetic means of the time spent by all solvers on solving the task, that is, the average participation time, referred to as participation time in days.
Independent variables	Task rewards	The seeker sets the monetary reward for the winner of the task.
	Task duration	The number of days between the start time and end time set by seekers is used as the measurement for project duration.
Control variables	Interaction	The amount of communication between seekers and solvers during the task process.
	Description length	The amount of text used in the task description.
	Credit	The credit of the task seeker.
	Experience	The number of tasks a task seeker has released.
	Satisfaction rate	The ratings regarding the task seeker’s reliability for rewards payment upon task completion.
	Real name authentication	The real name authentication of the task seeker; 1 is for real name authentication and 0 means no real name authentication.
	Email authentication	The inbox of the task seeker is authenticated; 1 is for an authenticated inbox, and 0 indicates that the inbox is not authenticated.

for specific definitions. Regarding dependent variables, according to the existing assumptions and theoretical basis, the number of submissions and participation time were used to indicate solvers’ participation behavior. The number of submissions refers to the number of task solutions submitted by solvers, and the participation time represents the average time taken by solvers during the participation stage (see Figure 3).

Regarding independent variables, task rewards are the rewards set by the task seeker. The task duration indicates the duration set by seekers from the start to the end of the task.

As for control variables, the quantity of communication between seekers and solvers affects solvers’ participation behavior. Task description also affects solvers’ participation behavior, because a clear task description can help solvers know more about the task. Seekers’ individual characteristics—including seekers’ credit, experience, ratings, and real name certification—also affect the participation behavior of solvers. The control variables can clearly indicate the individual differences in seekers and their changes over time. Therefore, time differences and individual differences have been controlled. Therefore, there is no need to set the time fixed effect or the individual fixed effect in the regression.

5.3. Sample Selection

The data in this paper are from Taskcn.com, which is one of the biggest online crowdsourcing platforms in China. Founded in 2006, Taskcn.com hosts competition tasks involving design, writing, and programming. As of February 2018, its registered users exceeded 3.68 million, tasks exceeded 63,000, and the total amount of task rewards amounted to 40 million yuan (RMB).

The web crawler tool in the Python program was used to capture all of the tasks released under the reward module, and a total of 3833 tasks solved between 1 February 2015 and 1 January 2018 were obtained. Among them, logo design accounted for nearly 73% of the total number of tasks. The single-person winning mode, in which only a single solver can be chosen to receive the task rewards, was chosen for the research sample, yielding 2706 pieces of valid data for empirical analysis.

5.4. Descriptive Statistics

Table 3. Descriptive statistics of the main variables.

Statistics	Mean	Std	P50	Min	Max	n
Number of submissions	28.13	21.59	22	1	171	2706
Participation time	0.63	1.231	0.269	0	36.87	2678
Task rewards	527.3	695.7	300	100	20,000	2706
Task duration	38.23	98.35	17.97	0.19	1274	2706
Description length	214.3	321.6	61	0	2052	2706
Interaction	1.197	1.718	1	0	19	2706
Credit	15.86	24.4	7	0	210	2704
Experience	3.981	6.671	2	1	67	2702
Satisfaction rate	0.999	0.0295	1	0	1	2706
Email authentication	0.465	0.499	0	0	1	2706
Real name authentication	0.163	0.37	0	0	1	2706

Date resource: the data was collected from the crowdsourcing contest platform in China: www.taskcn.com.

presents the descriptive statistics for the main variables of the crowdsourcing contests. The minimum number of submissions was one and the maximum number of submissions was 171, indicating a large difference in the number of submissions. As for participation time, the value varied from 0 to 36.87 days. The average value was 0.63, which also reflects large differences between solvers’ participation time. The implication of these figures is that the number and time of solvers’ participation may be influenced by different task settings.

There was a significant difference in the value of task rewards. These values varied from CNY100 to CNY20,000, while the average value was CNY527.3. In addition, the task (task ID: 97698) with a reward value of CNY20,000 obtained just 59 submissions from solvers, implying that higher task rewards do not increase the number of submissions indefinitely. Solvers’ participation is limited by the factors of the platform used and the number of solvers online. Thus, increasing task rewards blindly does not increase the performance of the crowdsourcing contest, and is a waste of resources. In terms of task duration, the minimum value was 0.19 days (task ID: 99769); the duration of this task was from 15:25 on 4 August 2016 to 19:59 on 4 August 2016. Only one solver submitted solutions. This task duration was therefore too short to be suitable for more solvers. The maximum value for task duration was 1274 days (task ID: 79539), which was solved within the timeframe of the sample data collection. This task duration was too long, and was not conducive to solvers’ returns or the platform’s efficient operation. This task duration length implies that the platform Taskcn.com did not give reasonable guidance on task setting.

Table 4. Correlation coefficients of the main variables.

	1	2	3	4	5	6	7	8	9	10	11
(1) Submissions	1
(2) Participation Time	0.291 *	1
(3) Rewards	0.428 *	0.205 *	1
(4) Duration	0.057 *	0.151 *	0.035	1
(5) Description	0.063 *	0.058 *	0.111 *	0.042	1
(6) Interaction	0.428 *	0.228 *	0.337 *	0.050 *	0.029	1
(7) Credit	0.185 *	0.119 *	0.408 *	0.095 *	0.099 *	0.106 *	1
(8) Experience	−0.061 *	−0.012	−0.051 *	0.220 *	0.021	−0.062 *	0.623 *	1
(9) Satisfaction	0.018	0.008	0.015	−0.019	0.007	−0.001	0.004	−0.001	1
(10) Email	0.001	0.026	0.008	0.050*	0.044	0.005	0.175 *	0.199 *	0.026	1
(11) Real-name	0.021	0.01	0.04	0.037	0.035	0.071 *	0.234 *	0.151 *	0.001	0.368 *	1

Note: The asterisk (*) indicates that the correlation coefficient is significant at a confidence level of 0.01.

provides the correlation coefficients of the main variables. It suggests that correlation coefficients were significant between the dependent variables and the independent variables. Notably, there were significant correlations between task rewards and duration and the number of submissions; the coefficients were 0.428 and 0.057, respectively. It was also significant between task rewards and duration and participation time; the coefficients were 0.205 and 0.151, respectively.

6. Research Results

6.1. Regression Results of Equation (1)

This article used Stata to estimate Equations (1) and (2). The regression methods included least squares regression and stepwise regression. The regression results are shown in

Table 5. Regression results of Equation (1).

	(1)	(2)	(3)	(4)
	ln(Submissions)	ln(Submissions)	ln(Submissions)	ln(Submissions)
ln(Rewards)	0.4024 (0.000) ***	1.3360 (0.000) ***		1.1740 (0.000) ***
ln(Rewards)2		−0.0752 (0.000) ***		−0.0668 (0.000) ***
ln(Duration)	0.1676 (0.000) ***		0.7322 (0.000) ***	0.6153 (0.000) ***
ln(Duration)2			−0.0823 (0.000) ***	−0.0709 (0.000) ***
Control variable	Yes	Yes	Yes	Yes
Constant	−0.0670 (0.877)	−2.2055 (0.001) ***	1.0780 (0.016) *	−2.8537 (0.000) ***
n	2700	2700	2700	2700
R2	0.336	0.299	0.269	0.376
F	151.2896	127.2377	109.7421	147.4256

Note: The p-value is in parentheses, * p < 0.05, ** p < 0.01, *** p < 0.001, and Yes indicates that the control variables are included in the model. This table is the result of Equation (1), which is used to verify Hypotheses 1 and 3.

and

Table 6. Regression results of Equation (2).

	(1)	(2)	(3)
	ln(Participation Time)	ln(Participation Time)	ln(Participation Time)
ln(Rewards)	0.4230 (0.000) ***	−0.5386 (0.310)	0.3936 (0.000) ***
ln(Rewards)2		0.0815 (0.068)
ln(Duration)	0.6965 (0.000) ***	0.6954 (0.000) ***	1.5943 (0.000) ***
ln(Duration)2			−0.1405 (0.000) ***
Control variable	Yes	Yes	Yes
Constant	−6.5544 (0.000) ***	−3.7653 (0.069)	−7.6128 (0.000) ***
n	2655	2655	2655
R2	0.195	0.196	0.209
F	71.1884	64.4591	70.0168

Note: The p-value is in parentheses, * p < 0.05, ** p < 0.01, *** p < 0.001, and Yes indicates that the control variables are included in the model. This table is the result of Equation (2), which is used to verify Hypotheses 2 and 4.

. After dropping the abnormal value in the variable, the total sample size was 2700. Column (1) of Table 5 reports the regression of task rewards and duration in relation to the number of submissions. The quadratic terms of task rewards and duration were then added into the regression model, with the result reported in Column (4). According to these results, the coefficient of ln(Rewards) was 1.17, and the coefficient of ln(Rewards)² was (−0.0668). These were both significant (p-value < 0.001). Based on the image of the quadratic function, this indicated that the relationship between task rewards and the number of submissions had an inverted U-shape, verifying Hypothesis 1. In addition, the coefficient of ln(Duration) was 0.6153, and the coefficient of ln(Duration)² was (−0.0709). Both of these were significant (p-value < 0.001), verifying Hypothesis 3 and presenting an inverted U-shaped relationship between the task duration and number of submissions.

The relationship between task rewards and the number of submissions therefore reflects a trend of initial increases followed by decreases. If seekers increase low task rewards, this can increase solvers’ expected revenue, which attracts solvers’ participation and results in more submissions. However, task rewards can only attract more solvers until a certain level, because high task rewards will lower the probability of successful bidding. The number of submissions will decrease according to the expectation that a lower winning probability decreases solvers’ expected revenue. The relationship between task duration and the number of submissions also shows a trend of increasing first before decreasing. If the task duration is too short for solvers to complete a task in time, seekers can set a longer task duration to attract more solvers and lengthen their participation in the task, which also increases the number of submissions. However, the increasing effect of task duration has a threshold. Solvers do not like to pay too much attention to a task with an overly long duration, and they may renounce participation and reduce submissions, because a much longer task duration usually means a longer payback period, as well as higher time and opportunity costs.

Thus, these results verify Hypotheses 1 and 3. When setting task rewards, rewards that are initially set relatively low and later increased will attract more solvers to participate and submit solutions. If the level of task rewards is higher than a given threshold, increasing task rewards will strengthen competition, causing some solvers to leave the contest due to low expected value. When setting the task duration, an initially short task duration that is extended will expose the task to more solvers and result in more solutions being submitted. However, if the task duration surpasses a certain threshold, some solvers will take the time cost into consideration and choose to leave to decrease losses. As such, reward and duration thresholds are very important for seekers when setting task rewards and duration.

6.2. Regression Results of Equation (2)

The sample size was 2655 after dropping several abnormal values of the regression for Equation (2). Column (1) of Table 6 is the regression results of participation time for task rewards and duration; the coefficients were 0.432 and 0.6965, respectively. These figures suggest that task rewards and duration were both positively related to participation time, with a significant confidence level. In Hypothesis 2, it was assumed that the relationship between task rewards and participation time is positively and linearly correlated. In order to exclude a nonlinear relationship, the quadratic term of task rewards was considered in the regression equation. Column (2) of Table 6 shows that the confidence levels of the coefficients of ln(Rewards) and ln(Rewards)² were not significant. Hypothesis 2 is therefore verified.

At the same time, the quadratic term of task duration was also considered, and the result is reported in Column (3). The coefficients of ln(Duration) and ln(Duration)² were 1.5943 and −0.1405, respectively. Both coefficients were significant (p-value < 0.001). The relationship of task duration and participation time has an inverted U-shape, according to the negative coefficient sign of the quadratic term of time duration. According to these coefficients, the symmetric axis (−1.5843/(2 × (−0.1405)) $\approx$ 5.67) is on the positive real axis, meaning that there is a threshold for task duration, in which a shorter task duration that is extended will extend participation time. Conversely, if task duration exceeds the threshold, extension of the task duration will shorten participation time. Hypothesis 4 is therefore verified.

In summary, seekers should control the scope of task rewards and duration within a certain range when designing crowdsourcing contest tasks. The results show that an increasing task rewards setting will lengthen solvers’ participation time, while the relationship between task duration and participation is significant, and has an inverted U-shape. With relation to task duration, if the task duration is too short, extending participation time will allow more solvers to submit solutions to the task. However, once the task duration reaches a certain level, extending the duration will increase solvers’ time costs, prompting solvers to decrease participation time.

6.3. Robustness Checks

In this study, two regression models were used. The first model addressed the number of submissions, while the second modeled addressed participation time. To test the robustness of these two models, several dependent variables were replaced. The results are shown in

Table 7. Results of robustness checks.

	(1)	(2)
	ln(Number of Registrants)	ln(Submission Time)
ln(Rewards)	1.1274 (0.000) ***	0.0982 (0.000) ***
ln(Rewards)2	−0.0641 (0.000) ***
ln(Duration)	0.5519 (0.000) ***	1.4639 (0.000) ***
ln(Duration)2	−0.0605 (0.000) ***	−0.1336 (0.000) ***
Control variable	Yes	Yes
Constant	−2.5908 (0.000) ***	4.0986 (0.000) ***
n	2700	2700
R2	0.413	0.567
F	171.6417	352.8265

Note: The p-value is in parentheses, * p < 0.05, ** p < 0.01, *** p < 0.001, and Yes indicates that the control variables are included in the model.

. Firstly, the number of submissions in model one was replaced by the number of registrants, because solvers submitting the task must have registered for the contest. The higher the number of registrants, the more solutions solvers will submit, although not all registered solvers will submit a solution. Secondly, in place of the average participation time for Equation (2), average submission time (which is the same as submission time) was used for the test. Participation time indicates the time solvers spent on the task, and the average submission time is the arithmetic mean of the time spent by all solvers during registration and submission. These numbers are included in Figure 3. Submission time also covers the solvers’ participation time and their continuous attention on the task, which indicates the level of effort. If there is no difference in the sign and significance, the two regressions are robust. As confirmed by Table 7, the results are robust.

7. Conclusions

7.1. Research Conclusions

An in-depth understanding of solvers’ participation behavior in a crowdsourcing contest can help a company or organization to efficiently obtain high-quality solutions for related tasks. This study revisited the impact of task rewards and duration on the participation behavior of solvers in crowdsourcing contests. On the basis of existing studies and related theories, and using the latest data from the Taskcn.com website, the behavior patterns of solvers prove to be different from those measured previously, and the analyses also explain the various findings in previous research. Our research conclusions are as follows. Firstly, task rewards and duration have an inverted U-shaped relationship with the number of submissions, indicating that, although these factors play a positive role in the submission stage, they become counterproductive beyond a certain range. Secondly, task rewards positively affect solvers’ participation time. For solvers who choose to participate in a task, the task rewards encourage them to invest more time in solving the task. Thirdly, task duration has an inverted U-shaped relationship with participation time, and there is a threshold for task duration. Within the threshold range, the time expended by a solver will increase with the extension of the task duration, and the solver remains willing to invest more time and energy into solving the task; however, once the task duration exceeds the threshold range, continued extension of the task duration will reduce the solver’s time investment.

7.2. Theoretical and Practical Implications

This article addressed the effect of task rewards and duration on solvers’ participation behavior, and has significantly theoretical and practical implications.

With regard to theoretical implications, the research contributes to the literature on the effect of task rewards and duration, and enriches the applications of expectation value theory [29,33] and self-efficacy assessment theory [36]. In terms of solvers’ participation behavior, previous research has mainly analyzed solvers’ participation behavior using task results [9,13,17,21,23], but it has seldom paid attention to solvers’ participation during the task process. Moreover, many scholars have studied the effect of task rewards and duration on solvers’ participation behavior, but there are no agreed-upon findings. This article constructed a framework for the effects of task rewards and duration on solvers’ participation behavior based on expectation value theory and self-efficacy assessment theory. The findings indicate that an increase in task rewards and duration will stimulate or inhibit solvers’ participation in different stages because of the changing expectation value and self-efficacy assessment; this implies that the two basic theories are confirmed to a certain extent, and expands upon previous research.

More specifically, this article makes the following theoretical contributions: (1) it observes solvers’ participation behavior from the perspective of the task process and results for the first time. Participation time indicates that solvers’ participation efforts are used to measure solvers’ participation during the task process, while the number of submissions is used to measure solvers’ participation in task results. These two perspectives have theoretical implications for how task setting affects solvers’ participation efforts and fills the gap of previous research. (2) The inverted U-shaped relationship between task rewards and duration and the number of submissions was verified. The study revealed that only task rewards and duration within an appropriate range can prompt more solvers to submit more task solutions, elucidating the differences in existing literature on the impact of task rewards and duration on the number of submissions. (3) The research has indicated that the effects of task rewards and duration on solvers’ participation are different. Task rewards positively affect solvers’ participation time, whereas task duration has an inverted U-shaped relationship with solvers’ participation time. These findings deepen understanding of solvers’ participation behavior.

This study has practical implications for crowdsourcing contest platforms and task seekers. For task seekers, this research illustrates how to set task rewards and duration to better motivate solvers to participate in crowdsourcing contests. More specifically, for task rewards, only a certain range of task rewards can attract more solvers to submit. If task seekers want to receive more submissions, task rewards can be increased, but doing so may disallow some solvers’ participation and reduce the number of submissions. Therefore, task seekers must compare task rewards between their own and other similar tasks. For task duration, the setting of task duration should not be made as long as possible. If the duration is too long, this will have a negative effect on the number of submissions and participation time. It is also not conducive to saving task seekers’ own time. Task seekers should adjust related task settings, such as task rewards and duration, in a timely manner in order to achieve stronger crowdsourcing contest performance. In relation to online crowdsourcing contest platforms, given the demonstrated impact of task rewards and duration on solvers’ participation behavior, platforms should establish corresponding guidelines to help new task seekers in reasonably setting task rewards and duration; this will also improve the performance of online crowdsourcing contest platforms and the efficiency of the platform’s operation, optimize the platform environment, and promote the sound development of online crowdsourcing contest platforms.

7.3. Limitations and Future Research Directions

This study improved upon existing research on the influence of solvers’ participation behavior, forwarding more theoretical explanation and models and conducting an objective and scientific empirical test through a relevant research design. However, this study still has certain limitations. Firstly, the selection of study samples was limited to logo design sample data. The sample is therefore insufficient for research on different types of tasks. Although the results of the study are representative, the universality of the research conclusions requires further investigation. Secondly, in the extraction of research task characteristics, the influence of the difficulty level of the task itself on solvers’ behavior was not considered, because the difficulty level of the task itself is strongly subjective and is difficult to measure using quantitative indicators.

For future research directions, the following three aspects should be explored further. Firstly, the model in this paper should be verified using different kinds of samples of solvers. For example, the differential impact of task duration and rewards on solvers of difference experience levels should be analyzed to determine how to set tasks to attract more experienced solvers. Secondly, different types of tasks for research should be used to enrich the universality of the findings. Thirdly, subjective data on task difficulty should be collected. Finally, future research programs can study the influence of task design on solvers’ participation behavior by controlling task difficulty.