ExCrowd: A Blockchain Framework for Exploration-Based Crowdsourcing

Abstract

Because of the rise of cryptocurrencies and decentralized apps, blockchain technology has generated a lot of interest. Among these is the emergent blockchain-based crowdsourcing paradigm, which eliminates the centralized conventional mechanism servers in favor of smart contracts for task and reward allocation. However, there are a few crucial challenges that must be resolved properly. For starters, most reputation-based systems favor high-performing employees. Secondly, the crowdsourcing platform’s expensive service charges may obstruct the growth of crowdsourcing. Finally, unequal evaluation and reward allocation might lead to job dissatisfaction. As a result, the aforementioned issues will substantially impede the development of blockchain-based crowdsourcing systems. In this study, we introduce ExCrowd, a blockchain-based crowdsourcing system that employs a smart contract as a trustworthy authority to properly select workers, assess inputs, and award incentives while maintaining user privacy. Exploration-based crowdsourcing employs the hyperbolic learning curve model based on the conduct of workers and analyzes worker performance patterns using a decision tree technique. We specifically present the architecture of our framework, on which we establish a concrete scheme. Using a real-world dataset, we implement our model on the Ethereum public test network leveraging its reliability, adaptability, scalability, and rich statefulness. The results of our experiments demonstrate the efficiency, usefulness, and adaptability of our proposed system.

1. Introduction

The concept of crowdsourcing has become more prevalent as a result of its growing popularity [1]. Crowdsourcing systems are being used by an increasing number of businesses and individuals to outsource jobs and attract workers all over the globe. Crowdsourcing is a concept that involves people or organizations utilizing the skill pool available on the Web to perform a task. This is usually carred out in the form of an open call. This low-cost, quick approach has been widely utilized for data analysis tasks, including sentiment analysis, image labeling [2], as well as object recognition [3]. This has enormous implications for global resource redistribution and societal sustainability.The concept of crowdsourcing comprises three groups of responsibilities: requester, worker, and crowdsourcing platform. The majority of existing crowdsourcing platforms, such as Freelancer [4], Fiverr [5], 99designs [6], Utest [7], and others, use a centrally managed approach, which means that all operations are handled by a single centralized authority. The people or organizations in charge of the platform are referred to as the “central authorities” in this context. As part of the process of completing a task via the platform, the central authority charges a fee. There is also a presumption that the central authority is credible, which must always be correct [8]. There may be threats to the central authority by intruders or insiders who maliciously modify data [8]. Furthermore, the user may have concerns about privacy and how their information will be utilized. In rare cases, user certification authorities have been proven to be untrustworthy [9]. However, crowdsourcing systems are vulnerable to the flaws of the obsolete trust-based paradigm, which poses certain unavoidable drawbacks. To begin with, typical crowdsourcing systems are subject to DDoS, hijacking remotely, and malicious attacks, rendering the services inaccessible. The bulk of previous research has tackled this from the standpoint of a centralized crowdsourcing platform [10]. Furthermore, much attention has been drawn to reputation-based crowdsourcing, where the requester selects workers based on their reputation score. This approach has the tendency of promoting collusive behavior between the requester and worker to boost the reputation score as compared to the exploration-based approach where we explore groups of workers who show interest in the published task, study their learning behavior, estimate their progress, and further analyze their performance patterns to aid the selection process. Here, the requester selects the most suitable solution and awards the incentive to the related workers. This approach promotes open innovation where the diversity of the skills available allows for more innovative ideas to be explored and also prevents the starvation of low-reputation workers. Several works have been conducted to address some of the above-mentioned drawbacks in crowdsourcing systems. To safeguard privacy protection, cryptography and differential privacy (DP) are utilized [11,12]. To counter “falsified” and “unrestricted” behavior, reputation-based techniques have been developed [13,14,15]. The bulk of these studies, however, are based on reputation-based crowdsourcing models, which can be improved further. Until present, no existing work has addressed all the above challenges at the same time.

To tackle this topic, we offer a decentralized crowdsourcing architecture based on blockchain technology that consists of four entities: the requester who requires a task to be performed, the worker who accepts to complete the task, the evaluator who evaluates the work conducted by the workers, and the system that acts as the interface for the task poster to publish the task and workers to search for and submit the completed task, as shown in Figure 1. In summary, we have made the following contributions:

• We propose ExCrowd, a decentralized exploration-based crowdsourcing framework based on the concept of blockchain with zero reliance on a central service provider to complete the crowdsourcing workflow that will also leverage open ideas and innovation so more creative solutions can be explored.
• On the basis of the proposed concept, we give a concrete approach. The entire crowdsourcing task procedure is carried out via a smart contract, which includes task publishing, task acceptance, reward allocation, and so on. In the scheme, we discuss five basic smart contracts: User Contract (UC), Job Contract (JC), Agreement Contract (AC), Tendering Contract (TC), and Assessment Contract (AC), through which crowdsourcing components such as submitting and receiving a task can be accomplished without reliance on any centralized system.
• We use a software model based on the ethereum test network and a dataset to demonstrate the efficiency of the suggested proposed methodology. The experimental findings reveal that our proposed crowdsourcing platform is both effective and scalable. We also provide a discussion about how this method may be improved in the future.

The rest of the paper is structured as follows. Section 2 provides an overview of the related works, while Section 3 discusses our design’s preliminary stages. Section 4 presents the design formulation of our proposed solution, and Section 5 describes the implementation of the proposed system. Section 6 discusses the reward distribution computation, while Section 7 presents the implementation and performance analysis of the system. Section 8 presents the analysis of the system design, and Section 9 concludes the work.

2. Related Work

With the fast rise of the internet and mobile devices, the study of crowdsourcing has emerged as a new trend. We primarily examine some cutting-edge research in three core areas: centralized crowdsourcing systems; distributed crowdsourcing systems; and blockchain-based crowdsourcing systems. We provide a summary of the related works in Table 1.

2.1. Centralized Crowdsourcing Platform

Various crowdsourcing platforms are designed to be centralized [4,5,6,7,16]. Significant challenges of centralized crowdsourcing platforms include trust management [17,18], incentive mechanism [19,20], quality assurance [21,22], security, and privacy [12,23]. Freelancer [4] and fiverr [5], two well-known crowdsourcing sites, enable requesters to easily engage individuals to complete tasks. They did, however, need comprehensive data from users. They kept user information and job data on a centralized system, which may be vulnerable to a privacy breach [24], DDoS/Sybil assaults [25], and the risk of a single point of failure [26]. Ref. [27] presented auction-based techniques EFT and DFT, and Ref. [13] suggested a reputation-based reward system to combat free-riding and falsified attacks in crowdsourcing, but their approaches are based on the classic three parities model, which does not apply to our scheme.

2.2. Decentralized Reputation Crowdsourcing Platform

Ref. [28] reported work on a decentralized reputation system in which they offered a reputation method based on privacy-preservation. Users are given a pseudonym instead of their true identity and need validation by a certifying authority. It also continues deleting previous comments and has no approach in place to avoid or address unjust evaluation.The authors of [29] suggest reputation systems that exploit blockchain. Zacharia in [30] recommends using a more current score obtained from the assessor to defeat the attack from collusion, although it may not indicate its prior conduct and may be simpler to alter the score.

2.3. Blockchain-Based Crowdsourcing Platform

Li et al.’s publication, Ref. [31] is the most applicable to this research. They developed a reputation management method but did not consider the numerous attacks that may occur. Ref. [32] proposed a reputation management approach based on blockchain but provided an inadequate security approach for their system. Their system has difficulty detecting spam workers who coordinate with other participants to increase their reputations. Some studies analyze the collusive behaviors of spammers exclusively among workers, ignoring requester–worker collusive behavior. Therefore, dishonest workers increase their reputation by engaging in tasks posted by fraudulent requesters. The preceding works are restricted to their specific applications, whereas we envision a decentralized conceptual model based on blockchain with far reaching objectives, such as every worker is given the chance to perform a task to prevent unfairness and bias to low-reputation workers in a secured environment. Secondly, a publicly viewable and resilient reputation mechanism with minimal or no collusive behaviors on the part of users with a robust evaluation process.

Table 1. Summary of some related works.

Reference	Remarks	Limitations
[4,5]	They provide an environment that enable requesters to easily engage individuals to do task	User information and job data are kept on a centralized system, which is susceptible to privacy breach and single point of failure
[28]	Uses a reputation-based method where a user’s true identity is masked with a psudonym and validated by a certifying authority	Has no approach in place to avoid or address unjust evaluation
[30]	Recommends using a more current score obtained from the evaluators to defeat the attack from collusion	It is vulnerable to score alteration although may not indicate its prior conduct
[31]	Scheme that evaluates tasks completed via smart contract rather than a subjective third party	Lacks an efficient evaluation function where the requester has no knowledge about the solution
[32]	Proposed a reputation management approach based on blockchain	The registration scheme is susceptible to sybil attacks since workers can send multiple requests from different addresses to the server
[33]	Proposed a consensus algorithm for the confirmation of a new block	System does not explore privacy preservation, especially on anonymous authentication
[34]	Provided a scheme based on an aggregate signature on a high-performance Edwards curve to prevent nonrepudiation	This scheme falls short as far as the parallel working model is concerned
[35]	Provides a decentralized incentive and reputation scheme that is based on expert user attention	Lacks the capability to measure the authenticity of data received from workers

Table 1. Summary of some related works.

Reference	Remarks	Limitations
[4,5]	They provide an environment that enable requesters to easily engage individuals to do task	User information and job data are kept on a centralized system, which is susceptible to privacy breach and single point of failure
[28]	Uses a reputation-based method where a user’s true identity is masked with a psudonym and validated by a certifying authority	Has no approach in place to avoid or address unjust evaluation
[30]	Recommends using a more current score obtained from the evaluators to defeat the attack from collusion	It is vulnerable to score alteration although may not indicate its prior conduct
[31]	Scheme that evaluates tasks completed via smart contract rather than a subjective third party	Lacks an efficient evaluation function where the requester has no knowledge about the solution
[32]	Proposed a reputation management approach based on blockchain	The registration scheme is susceptible to sybil attacks since workers can send multiple requests from different addresses to the server
[33]	Proposed a consensus algorithm for the confirmation of a new block	System does not explore privacy preservation, especially on anonymous authentication
[34]	Provided a scheme based on an aggregate signature on a high-performance Edwards curve to prevent nonrepudiation	This scheme falls short as far as the parallel working model is concerned
[35]	Provides a decentralized incentive and reputation scheme that is based on expert user attention	Lacks the capability to measure the authenticity of data received from workers

3. Preliminaries

This section discusses the exploration-based approach used and the various tools that were employed in the ExCrowd framework.

3.1. Exploration Based Approach

For starters, reputation-based crowdsourcing, in which the requester picks workers on the basis of their reputation rating, has received a great deal of interest. Such systems have a tendency to favor workers with a good reputation. Collusive behavior between the requester and worker is encouraged by using this strategy, whereas exploration-based crowdsourcing encourages a number of workers who are interested in a job to compete and submit their responses for the reward. The reputation-based approach starts by narrowing down their search for the best applicants as soon as they receive a huge number of applications. Even while the algorithms utilized at this stage are presented as tools to help task owners make better decisions, they have the potential to exclude a large number of applicants. Discrimination in the crowdsourcing platform environment is a well-known result of comparing high and low performance and making distinctions based on these subjective judgements. When platforms utilize typical machine learning approaches to filter task applications, efforts to enhance diversification in recruiting encounter issues. Algorithms like these focus on finding the most reputable workers immediately, rather than addressing the likelihood that they may overlook talented candidates that seem to be underrepresented as a result of their low ranked reputation on the platform’s existing data. Finding a good balance between recruiting from groups with an established record of success and taking chances on low-reputation groups in order to learn about their talents is what we seek to achieve.

3.2. Blockchain

Blockchain is a distributed, unchangeable record that makes it easier to record transactions and monitor assets in a corporate network. Bitcoin [36] was the first decentralized money concept that sparked widespread interest. In files known as “blocks”, a series of time-ordered transactions are stored [37]. Each block includes the preceding block’s hash value, and they ultimately form a hash sequence known as “blockchain”, which is effectively an open, unchangeable, as well as orderly distributed ledger [38]. System users contend for the opportunity to record transactions into the blockchain by offering computer power, and the winner is paid by giving coins along with service fees. The technology of blockchain points us towards a unique path in terms of reducing the role of the intermediary in today’s society. Furthermore, while we often identify blockchain with the fintech markets, the inventive potential of blockchain systems extends well beyond this. Blockchain technology is used in a variety of applications, including micropayment schemes, identification and storage systems, and health information sharing.

3.3. Cryptography

Cryptography is essential in ensuring the validity of transactions [39], the anonymity of users and the validity of data kept in a network. The security of transactions is guaranteed by electronic signatures of participating nodes and hashes of data records. The use of anonymous addresses in a network ensures transaction privacy, while cryptographic keys—public and private keys for signing—ensure transaction validity. The private keys authorize the transaction, demonstrating the signer’s competence, while the public key confirms the signature.

4. Design Formulation

This section gives a description of the different objects that were employed in this system. We formulate a secure communication mechanism between the workers and requesters to ensure transparency and data security. Our system modeling approach is depicted in Figure 2.

4.1. User Layer

The User Layer is made up of all the entities that use the crowdsourcing platform’s services to initiate and conduct processes required for their objectives. This layer communicates directly with both the registration and authentication layers, allowing users to sign up for system access. Users can sign up by going to the portal and filling out the required information. The registration and authentication center receives and processes the information that is supplied to the blockchain network. Examples of users include individuals in their homes or offices, schools, healthcare facilities, organizations, and so on. We divide the users into workers and requesters.

4.1.1. Requesters

The requesters have tasks that are complex for computers to complete but simple for humans, such as object recognition and environmental data gathering. The requester leverages a blockchain-based crowdsourcing framework. The requester submits their job to the crowdsourcing service along with detailed specifications and a timeframe. As a commitment, an amount of incentive and a monetary fine are required, which cannot be claimed before the due time. It must be noted that job posters and requesters are the same; therefore, they will be used interchangeably.

4.1.2. Workers

They are community members with certain expertise who compete for jobs in order to earn prizes. Whenever a new task is posted, the smart contract sends a notification to the worker. The worker can choose whether or not to accept it. They search for jobs that interest them using the blockchain-based crowdsourcing system.

4.2. Blockchain Network

Several components make up the layer that helps process all the data submitted to the platform. This layer additionally performs calculations with the data and provides it with features that help monitor any activity in the system. Algorithms are created that automatically notify illegal behaviors that occur in the system and automatically prohibit access to users who do not pay their needed incentives. The reported illegal behaviors are marked with the accompanying user’s unique ID and saved safely in a database. The results of every activity transmitted to the system are broadcast across the network, ensuring trustless and unbiased auditing.

4.2.1. Registration Center

The registrar and the verifier make up this layer. The registrar first receives the data submitted by a user. The request is processed, and the user is assigned a unique ID. This ID identifies each user of the system. The information about this action is forwarded to the verifier. After receiving the data, the verifier stores the user’s ID in their system. The verifier uses this unique ID to verify a user anytime he logs into the system.

4.2.2. Smart Contract Generator

Smart contracts are pre-programmed operations that are engaged and executed when an action is received. The cryptographic keys incorporated in the smart contracts developed on this system enable the contracts to conceal the generated reports from the activity activation. The primary goal of the smart contract in this system is to flag suspicious users, requesters who do not pay their incentives, and workers who do not finish their work within the stipulated time frame and to record such behaviors on a database. The smart contract generator generates these sets of contracts. (1) UserContract: adds new participants to the system. (2) JobContract: Users use this contract to generate new tasks and view existing ones on the platform. (3) AgreementContract: This is used to establish a contract between the requester and the workers assigned to a specific task. (4) TenderingContract: When the workers have completed the task allocated to them and are ready to submit it, they call the functions of this contract. This contract’s fundamental functionality is to receive submissions from the worker and the designated evaluators for those submissions. (5) AssessmentContract: gives a variety of features to task evaluators. To carry out any operation on the platform, the user must invoke the contract’s associated function. If the function call causes a change in the Ethereum blockchain status, the call to the function is considered a transaction by the person who made the function call. To prevent non-repudiation of the origin of the transaction as well as to secure the data integrity, transactions on the Ethereum platform are signed cryptographically via an approach known as asymmetric encryption. Miners, on the other hand, can decode the transaction with the public key of the initiator. After decrypting it, they check the transaction’s legality, and if it is confirmed to be genuine, it is published into the public ledger. The user must pay ether for these transactions and interactions with the Ethereum network.

4.2.3. Certificate Authority

This CA generates signatures on the system for the various system users. The system users use the signatures to sign their requests before sending them to the system. When a requester sends a request to the system, the signature on the request is verified by the verifier from the CA to ascertain the signature’s authenticity. If the verification fails, the request will not be processed.

4.2.4. Processing Nodes

The processing nodes are responsible for processing all data sent to the blockchain network. The processing node receives data from the registration and verification center, and transactions between workers and requesters and the logs of these data are processed into blocks. For verification, the block is disseminated to the other processing nodes. The block is added to the main blockchain if it is confirmed by a majority of the network’s consensus nodes. If the verification fails, it is returned to the original node for further review.

4.2.5. Task Pool

The task pool contains a collection of tasks published on the crowdsourcing platform, which workers can select from. The task pool has tasks ranging from different domains of expertise.

5. Communication and Work Flow of the Proposed System

Figure 3 provides a workflow of the proposed architecture. Furthermore, this section discusses how each of the processes depicted in Figure 2 is carried out on our system.

5.1. User Registration

During registering, each requester or worker does not have to provide a real identity. He receives a pair of keys: a “public key” and a “private key”. The user registration smart contract creates a hash using the public key to build a user’s address. The address does not contain any information about the user, so they have far more privacy than users on conventional crowdsourcing platforms. Concurrently, a user registration smart contract will be established for the enrolled user. Users may use pseudonyms to complete transactions in order to conceal their identity. The WorkerRep model offered that users have several addresses, each with their own public–private key pair. This, however, encourages malicious users to submit multiple requests from multiple addresses, making them susceptible to Sybil attacks.

5.2. Publish and Search Task

Following registration, the requester may publish a task, defining the task’s incentive, name, and prerequisite expertise to execute the transaction. We require a down payment from the requester in order to avoid payment refusal by the requester. When the transaction is processed, the task is added as a transaction to the public ledger by the requester as well as to the contract that corresponds to the list of available jobs. This phase is included in the blockchain network so that subsequent transactions may be connected to it. It will make the verification process more efficient. When a job is published, it is added to the task pool. The task pool contains the list of tasks that are accessible for workers to do. All of the accessible tasks, as well as their pertinent details, are public and can be seen. Filters can be applied to the array of available tasks to help workers choose tasks.

5.3. Task Application

In this phase, a worker can apply for a task that has been listed on the portal. A worker can express interest in a task by applying, as long as he or she fits the requester’s criteria. As previously said, each worker must pay a fee to ensure that they complete their jobs diligently while also significantly lowering spammers and fraudulent workers.

5.4. Worker Selection and Task Assignment

Choosing the best applicant from a pool of candidates is a time-consuming exercise. Low-performing workers are not favored in the reputation-based worker selection approach since they may never be allowed to execute tasks because of their low reputation rating. This can also lead to requester–worker collusion to fraudulently boost workers’ ratings. We first categorize the workers based on their expertise, such as I.T. specialists, health workers, economists, etc. A requester states the task’s category before posting the job. This enhances task matching to the appropriate worker category on the platform. These worker matching tasks take place in the exploration-based environment based on the outcome of the worker selection decision algorithm. Once the requester has finalized the task, he/she uses the agreement contract to construct a contract that includes the details of the chosen category. The agreement in this case is a binding contract between the requester and worker for specific tasks, which may reinforce the task’s system norms, such as acceptability, submission, and reward. While producing the agreement, the requester is prompted to pay the incentive for that job towards this contract. This payment may be retrieved only when the worker has never accepted the assignment, and thus, the requester wishes to terminate the job, or it may be granted once the work is completed. Following the proper generation of the agreement, the chosen worker is prompted with updates in relation to the commencement of the task. These transaction logs are processed using the processing nodes and kept on the blockchain. Workers maintain a consistent level of performance throughout time, even if it fluctuates across activities. We then investigate whether or not employees display learning behavior when undertaking tasks and whether or not we can estimate and anticipate their progress by employing learning curves based on their conduct. As a further step, we’ll analyze existing workers’ performance patterns using a decision tree technique. To back up our claim, we’ll present evidence in this section. Consider w to be a worker with a rate of learning $l_w$ and a background knowledge $k_w$. Next, we use the hyperbolic learning curve model, which depicts that the worker w correct judgement percentage he has achieved up till the zth task is defined as:

$$H_w\left(z\right)=\frac{z+k_w}{z+k_w+l_w}$$

(1)

The quality of the worker, $h_w\left(z\right)$, is estimated given $H_w\left(z\right)$ at task z as:

$$h_w\left(z\right)=z{H}_w\left(z\right)-(z-1)H_w(z-1)$$

(2)

$H_w\left(z\right)$ and $h_w\left(z\right)$ would be simple to estimate if $l_w$ and $k_w$ were predetermined. As an alternative, we approximate $H_w\left(z\right)$ by calculating the number of right responses that each worker has submitted so far. We map a linear model onto the data in order to learn the $H_w\left(z\right)$ model.

$$M_w\left(z\right)=f_{w}z+g_w$$

(3)

where $M_w\left(z\right)=\frac{1}{1-H_w\left(z\right)}$, $f_w=\frac{1}{l_w}$, and $g_w=\frac{k_w}{l_w}+1$, and using linear regression techniques to determine the parameters of $M_w\left(z\right)$, so estimating $H_w\left(z\right)$ workers’ learning curves may be modeled in two ways. As a first step, we may utilize the projected $l_w$s to rate workers based on their learning rate and pick the most promising workers for training. This knowledge may be used in our recruiting decisions since we can calculate the likelihood that a worker will make the accurate prediction on upcoming tasks using $H_w\left(z\right)$ and $h_w\left(z\right)$.

In the second phase, we use decision tree classification algorithms to pick and categorize employees from the pool of the existing workforce based on their learning rate performance using a decision tree classifier. Classification is performed via the decision tree-based C4.5 classifier. This classifier utilizes a “divide-and-conquer” strategy from a collection of independent occurrences. Performance criteria for specified qualities are used as input variables, and the output is a recommendation for hiring status of “yes” or “no”, as seen in the worker selection decision algorithm in Algorithm 1.

Algorithm 1 Worker Selection Decision Algorithm

1: Input Worker attribute dataset D 2: Output decision tree//yes or no 3: Check for base cases such that D belongs to same class 4: for all worker attribute $wa\in D$ do 5: Estimate the normalized information gain from splitting on $wa$ 6: endfor 7: $w{a}_{best}$ = attribute with the highest normalized information gain 8: Tree = Create a decision node that splits on $w{a}_{best}$ 9: Recur on the sublists resulting from splitting $w{a}_{best}$ and add those nodes as children of node 10: return Tree

5.5. Task Reception

The worker is alerted after the job poster appoints the workers to create an agreement. The workers must send a predetermined quantity of ethers to the contract to accept the task. The ethers that the workers deposit will be reimbursed upon completing the task. When the contract code receives the ethers from the workers, it checks to see if the sender’s address of those ethers matches the address of the workers supplied by the requester during agreement formation. If they are the same, the requester is alerted, and the workers can then start working; otherwise, the gas associated with that transaction is depleted. This is carred out so that the requester does not have to wait indefinitely for the agreement to be accepted by the worker, and the requester can recover the ethers and assign them to another worker. We keep track of whether the worker has accepted the agreement so that the requester does not have to explicitly cancel it midway. If he or she has paid, the task poster cannot end the contract and will only be revoked once the due deadline has passed. Receiving ethers from the worker when he accepts the assignment ensures that he does not abandon the task in the middle of it. If he completes the task, the refund will be determined by the accuracy of the submitted output.

5.6. Task Submission

After completing the job, the worker executes the tender contract to publish the solution. Firstly, the worker creates a hash of the solution and submits it ahead of the deadline by executing a hashing algorithm. We do not treat the worker’s hashing as part of the Ethereum network. This is because the worker does it explicitly outside of the blockchain. Furthermore, as actions taken on Ethereum are made public, we would not want to make the proposal visible. The tendering contract utilizes the hash for two purposes: (1) it tells the platform that the worker has completed their assignment and is now available for evaluation. (2) The worker is prevented from changing their submission after learning who their evaluator is. Each reviewer’s public key is then used by a worker to encrypt the submission for that reviewer. The worker then encrypts the submission with their private key. Due to: (1) the prohibitive cost of keeping large quantities of data on the blockchain network, the submission will not be stored on the blockchain network. (2) The requester may be reluctant to release the entry publicly due to privacy concerns. The worker combines the hashes acquired by storing each evaluator’s encrypted submission and sends them to the tendering contract. Using first-level encryption ensures secrecy since no one but the evaluator with the corresponding private key may access it. The worker’s authentication is provided through the second level of encryption. Our system is unable to prevent evaluators from expressly sharing submissions with others. Encryption and decryption are not safe on blockchain since the keys become publicly exposed, such as in hashing the submission stated above; hence, they are conducted off-chain on the end user’s side. Having the encrypted hash stored just once for each worker consumes considerable storage capacity. As an alternative to conducting the two-level encryption mentioned above on the worker’s submission, we may save it once on a file storage system and perform it on the hash returned from the file storage system. This would significantly reduce the amount of storage capacity needed by the file storage system as the evaluator numbers grow, but it might lead to privacy and security concerns since anybody possessing the hash could access the submission.

5.7. Task Evaluation

In this section, we discuss the process of scoring the workers by the evaluators. Evaluators are miners on the blockchain network who are selected to assess submissions sent in by workers. Our suggested technique is based on the jury concept, which states that a group of less competent persons is more likely to make the right choice than an individual with more competence. As a result, rather than relying on user history to accept a worker’s contribution, we rely on the consensus of a group of workers who are miners on the blockchain network. We denote the set of evaluators on the platform as $e_1,\cdots e_n$. There are two criteria for scoring the workers. The workers are scored based on the completeness and quality of their work. The completeness score is denoted as $C_i$, and the quality score is denoted as $Q_i$. $C_i$ and $Q_i$ represent the completeness and quality scores of worker $W_i$, respectively. We represent the mean of the completeness and quality scores as $com{p}_m$ and $qua{l}_m$, respectively. Some evaluators can act maliciously and collude to score a worker unjustly. Therefore, the mean score of the completeness and quality scores cannot be the accurate score of the workers. To solve the above problem, we use the standard deviation of the completeness score and quality scores. $com{p}_s$ and $qua{l}_s$ denote the standard deviation of the completeness score and quality scores.

Furthermore, the evaluators whose scores are beyond a specific limit away from the mean are considered outliers and therefore are not regarded as part of the score. This principle helps prevent evaluators from acting maliciously and scoring the workers unfairly. If the evaluators disagree on a particular submission, they resign and repeat the whole process. After the evaluators have reached an agreement, their scores for the two factors are put together. We denote the consensus score of a submission’s completeness and quality scores as $C_{wy}$ and $Q_{wy}$, respectively, where w represents the worker and y represents the submission. Equations (4) and (5) depict the computation for the completeness and quality scores, respectively.

$$Q_{wy}=\frac{{\sum }_{i=0}^n{C}_{iy}·{\alpha }_i}{{\sum }_{i=0}^n{\alpha }_i}$$

(4)

Here, ($n=ec$) denotes the number of consensus evaluators, and $C_{wy}$ and $Q_{wy}$ are the scores related to the completeness of the work and the work quality, respectively. The evaluator’s credibility is used to buttress the scores assigned by the evaluators. The credibility of the evaluators are computed based on their reputation and denoted by ${\alpha }_i$ for the ith worker.

The final score ${\sigma }_{w}y$ assigned by an evaluator to a worker is the weighted average of the score obtained in two occurrences. Here, w represents the worker, and y represents the submission. The task poster can decide on the weights of these factors, which adds up to 1. Let ${\beta }_c$ and ${\beta }_q$ represent the weight assigned to the completeness score $C_{wy}$ and the quality score $Q_{wy}$ of the work, respectively. We compute the final score as shown in Equation (3).

$${\sigma }_{wy}=\frac{{\beta }_c\ast C_{wy}+{\beta }_q\ast Q_{wy}}{{\beta }_c+{\beta }_q}$$

(5)

After computing ${\sigma }_{wy}$, the worker is notified about the transaction. Sending the worker a transaction is a way for us to keep track of all of the reviews they’ve earned on the platform. This makes it easier for anyone to establish the worker’s previous reputation. The reputation score cannot be purposefully manipulated by a central entity in this situation, and no worker may intentionally fabricate its own reputation on the platform. The worker’s reputation is then enhanced by the score he or she obtained. The worker’s reputation is only updated if he or she makes a submission; it is not changed if the worker accepts the assignment but does not submit.

6. Computing Reward Distribution

It is imperative to appropriately compensate all employees, especially when their contributions are varied. To overcome the problem of reward distribution, certain solutions have been devised. The Shapley value is the most potent of these strategies. Shapley is used when each worker’s contributions are unequal, yet the overall task is completed collaboratively by all of the workers involved. The Shapley value is an estimate that represents the average expected marginal contribution of one participant after all possible combinations have been explored. The Shapley value has been shown to be a reasonable method of assigning value.

The worker set is denoted by K, and the number of employees is given by $\left|K\right|$. $\omega$ denotes a worker group and is a non-empty subset of K. The group’s worth function K is given as $u\left(\omega \right)$, indicating the group’s input. We use ${\vartheta }_i$ to represent the i-th worker who does not belong to group $\omega$. As a result, the marginal contribution ${\vartheta }_i$ of the group K is defined as:

$$u(\omega ,i)=u(\omega \cup \{i\left\}\right)-u\left(\omega \right)$$

(6)

Equation (5) defines the Shapley value.

$$\rho \left({\vartheta }_i\right)=u(\omega ,i)\sum _{\omega \in {\omega }_i}\frac{\left(\right|K|-|\omega |-1)!\left|\omega \right|!}{\left|K\right|}u(\omega ,i)$$

(7)

${\omega }_i$ contains all subsets of $\vartheta$ except for ${\vartheta }_i$. It is critical that incentives are distributed properly based on the efforts of the workers. We will use data collection and picture annotation as examples to assess each worker’s participation. The more data samples collected in a data collection process activity, the more accurate the task’s ultimate outcome. As a result, samples obtained after the first are less significant. Consequently, we use Equation (6) function to calculate the contribution.

$$H\left(\gamma \right)=1-e^{-{\upsilon }_1\gamma }$$

(8)

$\gamma$ denotes the number of samples submitted by a worker. For instance, ${\vartheta }_1$ submits six samples initially, followed by ${\vartheta }_2$ uploading three samples. To make the problem easier to understand, we define ${\upsilon }_0$ as 1. The input of ${\vartheta }_1$ is $1-e^{-6}$. The overall input of ${\vartheta }_1$ and ${\vartheta }_2$ is $1-e^{-9}$. Therefore, the input of ${\vartheta }_2$ is $1-e^{-9}-(1-e^{-6})$. If ${\vartheta }_2$ first submits their samples, the input of ${\vartheta }_2$ is $1-e^{-3}$. Furthermore, the input of ${\vartheta }_1$ whose samples is 5 is $1-e^{-9}-(1-e^{-3})$. Equation (5) can be used to compute the contribution of each worker. We show the algorithm for distributing rewards in Algorithm 2.

Algorithm 2 Algorithm for distributing rewards

1: INPUT: Workers, TaskAddress(TA) 2: OUTPUT:$[m_1\cdots m_n]$//Allocate reward to respective workers 3: $N_w\leftarrow$ Number of workers 4: $id\leftarrow$ Index of workers(TA) 5: $P_c\leftarrow$ Possible Coalitions(TA) 6: $P_p\leftarrow$ Possible number of workers in coalition(TA) 7: $T_c\leftarrow$ The total of possible number of workers in coalition 8: $V\left(P_c\right)\leftarrow$ Value of coalition; 9: Determine $N_w,id,P_c,P_p,V\left(P_c\right)$; 10: Create Matrix[]; 11: for$q=1$ to $T_c$ do 12:    for $r=1$ to $N_w$ do 13:      Estimate the submission of workers to coalitions with $P_p$ workers 14:    end for 15:    Estimate the probability of coalition with workers $P_p$ workers 16:    Accumulate total reward of the task(TA) 17: end for 18: $\mathit{Shapely\; value}\leftarrow$ accumulated reward of the task 19: return $m\leftarrow reward$

7. Implementation and Performance Analysis

This section details the implementation details of our system and evaluates its performance. Experiments have been devised and key parameters have been measured. We evaluate the framework’s viability by defining node connections to specific components across an Ethereum network, utilizing a proof of stake consensus technique provided by the system. The dataset used for the conduction of the experiment is CALTECH-101, an object recognition dataset provided by Fei-Fei Li et al. in Learning generative visual models from a few training instances (https://data.caltech.edu/records/20086, accessed on 11 March 2022). Users (including requesters and workers) are enrolled on the blockchain, which requires aggregating information about a given user. Next, a public and private key pair are generated for each user, which may be used to publish jobs and solutions on the system.

Our solution was built on top of an Ethereum Ropsten network, which has more than 5000 nodes. Ethereum is a decentralized application platform built on the Solidity programming language. To replicate the blockchain, a 2.3 GHz Intel i7 PC with 32 GB of RAM is utilized, as shown in

Table 2. Simulation Environment Setup.

Parameter	Value
System Specification	Intel®CoreTM i7 6700 HQ [email protected] GHz (8 CPUs) 2.3 GHz Memory: 32 GB RAM
Ethereum Client	Ropsten
Nodes	50–1000

. We created an application in Python that allowed users to post or submit a request on the platform. A JavaScript Object Notation (Remote Procedure Calls) library was used to synchronize the Python application with the blockchain. Queries are submitted to the system and filtered (into tasks and solutions) and transmitted to the blockchain.

7.1. User Registration

On the blockchain network, we timed the duration it takes to register a user. To register, the person transmits their information to the blockchain and he/she is issued membership keys. We estimated the time required for this transaction to be mined. The average registration latency was calculated after simulations of 40 repetitions of this scenario. The average latency in the experiments was 13.75 s, which is close to the 13 s required for block production in Ethereum networks as depicted in Figure 4.

7.2. Time Complexity

Caltech-101 is an object recognition dataset provided by Fei-Fei Li et al. in Learning generative visual models from few training instances (https://data.caltech.edu/records/20086, accessed on 11 March 2022). It features images from 101 distinct object categories with varying sizes and spans. For each object category, there are about 40 to 800 images, but then most classes contain about 50 images. The images have a resolution of around 300 × 200 pixels.

We ran 25 sets of trials to process picture tagging tasks using the CALTECH-101 dataset. We chose 10 categories of images such as elephants, bicycles, soccer balls, human brains, crocodiles, and airplanes to analyze the utility, security, and efficiency of the system. Each of the 800 images in the CALTECH-101 dataset is split into five training batches and one test batch.For each set of experiments, we choose images at random from the training batch.

Furthermore, five employees and one requester have been added to the system, with each obtaining 20 ETH coins. Three workers complete each activity. We chose 500, 1000, 1500, 2000, and 2500 photos randomly from the training samples based on task 500, 1000, 1500, 2000, and 2500. The solutions are also secured with the public key of the requester and preserved in data storage, making them inaccessible to malevolent users.

We deployed the UserContract, JobContract, AgreementContract, TenderingContract, and AssessmentContract in the Ropsten Network. In Ropsten, the average mining block time is 10 s. A total of 800 transactions were used in the experiment. Each transaction is mined together into single blocks with a block of numbers, which ranges from 4,324,649 to 4,330,703, as depicted in Figure 5, Figure 6 and Figure 7.

On task publishing, receiving of tasks, as well as submission of solutions, the average transaction confirmation time was about 39.24 s, 34.17 s, and 38.71 s, respectively. Each transaction was confirmed in around 35 block time. Furthermore, we provide a network performance metric in

Table 3. Network Performance Metric.

State	Value
Hash rate	381,710.11 GH/s
Average block time	0.200 s
Blockchain size	600.79 GB
Block time	12.2 s
Average block size	41.097 KB
Block count	10,751,967
Average transaction per second	13.5 TPS

, gas consumption in

Table 4. Gas consumption and the cost of carrying out diverse functionalities of the smart contract.

Functions	Required Gas	Fee Estimate ($)
Create requester	248,310	0.0775
Create worker	250,786	0.0777
Post task with a cost	290,544	0.0854
Create agreement	220,134	0.0637
Accept agreement	780,829	0.0129
Hash submission	190,028	0.0459
Designate evaluators	550,708	0.0100

and feature comparison in

Table 5. Comparison of system features with existing crowdsourcing systems.

References	Privacy	Decentralization	Smart Contracts	Encryption	Exploration Based	Shapely Value
[40]	YES	YES	NO	YES	NO	NO
[41]	YES	YES	NO	YES	NO	NO
[35]	YES	YES	NO	YES	NO	NO
Ours	YES	YES	YES	YES	YES	YES

.

To have a clearer perspective of the transaction costs associated with implementing the primary functions of our application, we first gather gas needs before calculating the cost per gas at 1 Giga-Wei. It should be emphasized that one Gwei equals 0.000000001 ETH, and the value of one ether during this experimentation (February 2022) is USD 2935.89. Observing the information in Table 4, we determine the entire task cost from the uploaded time to the time it undergoes system review. The entire amount of the needed gas is 2,531,339, which equates to USD 0.3731. Since the expenses of registering as a worker or posting a job are only incurred once, they have not been included in this calculation. Various ways of testing the smart contracts were investigated throughout the design of our smart contracts. One approach is to utilize software such as ganache-cli, which provides a localized replica of the Ethereum blockchain for testing purposes. A common alternative is to employ test networks, which include Ropsten or rinkeby. The test networks function similarly to the Ethereum main net, with the exception that ether in these networks has no value and is primarily used for testing.

We conducted a segment of a range of image tagging investigations and documented the image tagging accuracy by the workers in each task using varied image quantities. On 8500 images, the average accuracy was 96.52%; amongst the erroneous findings, some images were extremely difficult to deduce. This demonstrated ExCrowd’s effectiveness. Furthermore, we discovered that certain miners were offline during the tests, but this had no effect on the process, and users were able to post or receive tasks as usual, demonstrating the decentralized aspect of our system.

We further looked at the latency and processing time of the network with and without blockchain to assess the quality of the provided solution. In Figure 8, we demonstrate the latency in the quality assessment of 50 images given different worker sizes and values of e for e-repeated assessment. In Figure 9, we illustrate how long it takes to evaluate (processing time) Caltech images employing five evaluators of different sizes. Evaluators receive 50 photos every time a worker presents them for e-repeated review. Submissions are made till the total number of images reaches 1000, 2000, 3000, 4000, and 5000. The results demonstrate that ExCrowd can evaluate 5000 photos in under one minute. In addition, the time for processing does not change substantially even if the number of workers rises.

We further repeat this scenario without the blockchain architecture, as indicated in Figure 10 and Figure 11. The outcome shows that the latency and processing time increase. This may be a result of the centralized nature of the system. Thus, putting a lot of demand on a central server is compared to the distributed nature of blockchain, which reduces the areas of vulnerability and also improves the distribution of resources.

7.3. Comparison

Figure 12 and Figure 13 illustrate a comparison of the proposed system in terms of registration time and processing overhead with three existing ones, namely PrivCrowd: A Secure Blockchain-Based Crowdsourcing Framework with Fine-Grained Worker Selection [40], CrowdR-FBC: A Distributed Fog-Blockchains for Mobile Crowdsourcing Reputation Management [41] and An Incentive and Reputation Mechanism Based on Blockchain for Crowd Sensing Network [35]. For starters, the proposed system is analyzed for its typical registration times. By altering the number of blocks, it has been shown that the proposed task takes less time to execute than the existing systems. Thus, the performance is evaluated depending on the amount of time it takes to process the data.

It has been possible to use a variety of crowdsourcing systems up to this point. Table 5 compares these systems on the basis of a variety of important factors.The qualities of privacy, decentralization, smart contracts, encryption, exploration based, and Shapley value are all taken into account in this comparison. The first factor is privacy. As a result of the sensitive data stored in these systems, it is essential to protect the privacy of users.

Certain crowdsourcing systems do not require participants to use their actual names, which helps protect their anonymity. The most critical aspect affecting a crowdsourcing system’s acceptability is its ability to boost both the quality and quantity of user participation. The second parameter considered is decentralization, where the system is not controlled by a central authority, thereby lowering the risk of system failure. The third parameter used in this comparison is the adoption of smart contracts, which helps to check suspicious behaviors of users in the system, especially with requesters who refuse to pay their incentives.

Furthermore, encryption is performed to secure any form of private information, eliminating the exploitation of privacy. The fifth factor used in this comparison is the exploration-based approach, which ensures that the system is not biased towards high-reputation workers as is conducted in reputation-based systems, where certain systems take workers’ reputations into consideration when allocating tasks. This kind of approach plays a critical role in reputation-based threats as a result of some unscrupulous workers engaging in requester–worker collusion, resulting in the system storing false data impressions.

Finally, we adopt the “Shapley value” approach in our reward distribution. Workers will only stay engaged in crowdsourcing if they feel that their efforts are being compensated fairly. As a cooperative game, crowdsourcing may be modeled using Shapley values, one of which is the best approach for distributing prizes fairly. Each worker’s participation may be evaluated via the smart contract and their incentives distributed evenly using our recommended Shapley value-based distribution method. There is no waste in the distribution of advantages from collaboration. Equal payoffs are given to those who contribute equally.

8. Discussion

This section explains how our system’s goals are accomplished.

8.1. Decentralization

No centralized body is responsible for administering the platform or for data maintenance. The miners on the blockchain network verify all functions performed by users on the platform. There are incentives for the user to behave well and disincentives for bad activity.

8.2. Resistance to Security Threats

This section discusses how our approach aids in the prevention or avoidance of the aforementioned attacks.

• Stuffing of ballots and collusion attack:

These are avoided by using a rigorous evaluation technique, as detailed in the evaluation section.

• Sybil Attack:

This can be avoided simply by charging an initial one-time access fee. When a worker quits the platform, the amount refunded to them is determined by their reputation at the moment.

• Reciprocity:

Given that there are numerous evaluators, the odds of being chosen as an evaluator for that worker are slim, and given that anomalies are taken into account when estimating the worker’s final score, any worker will be unlikely to reciprocate.

• Unfair rating attack/defamation

A decision is being made based on a consensus among many evaluators. The removal of outliers is part of the consensus method. The outlier in this case is the evaluator whose assessment score differs from the rest of the evaluators. The outlier/s’ result is not taken into account in the final ranking. If the evaluator is proven to be an outlier, their reputation decreases or he/she is assigned another assignment for review. This works as a significant barrier to giving a rating that is inconsistent with the job carried out by the peer worker.

8.3. Anonymity and Privacy

Using asymmetric-key encryption methods, our system ensures the anonymity of job submissions. Submissions are only accessible by the evaluators and the requester and will not be shared with anyone else. On the other hand, our platform ensures users’ pseudo-anonymity by disassociating their account from their individually identifiable information. The blockchain has left a legacy of pseudo-anonymity. For privacy, Ref. [33] avoids associating transactions from different activities. However, this would result in a lack of confidence in the platform. Because a user’s previous behaviors are unknown to other users, it is possible for them to mislead others on the site.

9. Conclusions

With the massive progress and spread of technology, blockchain has emerged as the best solution for delivering a decentralized yet collaborative environment while protecting the anonymity of all application participants. A decentralized blockchain-based crowdsourcing network called ExCrowd is presented in this study. The typical centralized approach of crowdsourcing has a number of disadvantages, including privacy breach, single point of failure, as well as hefty service prices. According to our analysis, we established ExCrowd to deal with these centralization issues and high reputation worker-selection biases. The suggested system decentrally caters to the needs of all entities, resulting in consistent data, secure communication, enhanced collaboration rate, and legitimate evaluations. Meanwhile, we improved crowdsourcing adaptability by using smart contracts to express sophisticated crowdsourcing logic, which also enables traceability, avoids unverified data alteration, and delivers a fair amount of incentive and remuneration to the task poster. To build an actual scheme inside the framework, a number of design algorithms equipped with smart contracts were presented. In addition, we put our method to the test on Ethereum by developing modules that provide decentralized crowdsourcing solutions. We further validate our approach by implementing the system on the Ethereum test net.

However, considerable discussions on several of the concerns are necessary, and we will reserve them as future work. To begin with, the requester had no involvement in the evaluation process. Possible changes include giving the task poster a reputation score and giving them a bigger say in the evaluation process in general. Furthermore, executing calculations on the Ethereum blockchain is quite costly, which may be decreased by utilizing computational oracles that execute computations off blockchain rather than on blockchain, but this comes with the expense of introducing centralization. We may also consider exploring other classification algorithms as far as the worker selection decision is concerned.