Verification of a labor market domain using an academic crowdsourcing system

Full Text

1. Introduction Assessment of the job market is an important practice in the field of academic curriculum development and in the election of a career path for prospective professionals before becoming part of the educational system [1]. Nowadays, however, students are more than ever concerned about correctly selecting a professional path in which they won’t lose time in learning skills that they are not likely going to apply, even if it means the possibility of not obtaining a diploma [2]. As one of the popular topics in today’s sharing economy, crowdsourcing [3], [4] has become a feasible tool for people to obtain knowledge at reasonable costs from a set of experts in a field. Such experts, with varying levels of expertise, can be found in platforms like Amazon’s Mechanical Turk [5]. In an academic environment, the roles of teacher and student (expert and requester) can also be integrated in a crowdsourcing system, and if a single agent is allowed to perform both roles interchangeably, then the agent can be further benefitted by an academic platform such as VUZ [6]. Entering an environment like VUZ requires a certain level of previous knowledge of the labor market [7], either compiled by the student or requested to experts. A platform like this, however, can also aid in gathering feedback [8] from the job market in order to actualize itself and optimize the benefits for the requesters and workers. Such feedback provides a way to verify [9] the previous knowledge about the market and points to possible improvements to the system. In particular, the verification of a domain theory [10], in this case the labor market, allows to tackle all the possible issues that could cause a failure of a forecast, which in this work we take as the projected salary of a new professional in its field of expertise in the job market [11]. We structure this paper as follows: the basic concepts are detailed in the second section, before we explain the concept of labor market in the next part. The fourth section deepens into the concepts of verification and contradiction, while the fifth section focuses on the crowdsourcing system’s ability to generate feedback and sustain itself in time. 1. Labor market The labor market can be thought of as an open multiagent system (MAS) with limited resources [12] where agents seek to maximize their revenues by performing a balance between exploration [13] and working in exchange for a retribution. In the simplest case, requesters can be thought of as being separate from the workers [14] in the sense that their interrelations are not necessary evident to the workers. 2. Academic crowdsourcing Agents can enter the open MAS freely, but often the process is channeled by an academic institution. In such cases, students agree to pay the institution more depending on its level of prestige in certain field of study [15]. Nowadays, however, prospective professionals are growingly looking for ways to land in the desired position in the market in a simplified way, often bypassing the obtainment of a diploma [16]. Personalized courses designed by a set of experts conform the core functioning of the academic platform VUZ and they correlate and complement the skills already mastered by the student, so that all excessive costs are avoided. 3. Verification We take into account a formal system =< , , , > [17], where is a finite set of basic characters known as the alphabet of the theory, corresponds to a collection of formulas, is referred to as a set of axioms and is a set of inference rules. Then, the objective of the verification process is to find all the discrepancies [18] among the elements of the formal system, starting with the set ′ ⊆ that contains the object types, the properties and values of the types and the logical and functional relations between them. 4. Contradictions Contradictions [19] arise in the formal system when the process of recognition of objects in the real environment [20] results in discrepancies with the entities and relations in the respective domain theory under which the recognition is performed. In particular, contradictions are reflected by the presence of ’impossible’ elements of the alphabet [21] in the real environment as well as the absence of expected elements. On the other hand, they point to conflicts in the axioms [22] or contrasting states of the system caused by defects in the inference rules. 5. Retribution A key point in the VUZ framework is the treatment of students as experts, along with the teachers [23]. This means that every level of expertise from the agents of the system is taken into account, and is utilized in order to teach other students with a lower level of knowledge in a determined field of study. As such, a student is required to pay a certain cost to obtain a crowdsourced assessment of his/her skills, as well as a personalized curriculum [24] for the obtainment of the desired new skills. The cost, however, is mitigated by giving the student the opportunity to share his/her knowledge with others in exchange for a retribution [25], [26]. This is an internal benefit from the academic crowdsourcing system, obtained prior to entering the labor market. Once in the market, the new professionals will obtain a real salary which will become the source of feedback for the system. 6. Profiles The profiles of the agents in the crowdsourced system consist in a set of skills that gradually grows while the agents develops themselves as workers in the system [27]. The description of a skill is maximally general in order for the skill to be used as a variable in different kinds of tasks. In particular, the agent performs a task by adopting a certain role where a set of key skills are necessary and actively used. On the other hand, roles and task performing are valid and can be evaluated and priced only when observed from the point of view of a certain context [28]. This implies that the same skills can have a different monetary value in different situations: contexts induce different partitions of the set of skills [29]. Like roles, contexts are described by means of a profile, albeit not of skills but of locators: they function as dimensions and can take different values ordered in a scale, giving the context a defined structure to answer to questions involving the place, time and other situational aspects of task performing. 7. Feedback Gaining direct knowledge from the labor market is crucial in the development and functioning of the academic crowdsourcing system. Once a student leaves the system and joins the job market, he/she is nevertheless asked to communicate his/her salary and to provide a picture of the context in which he/she works, which can include a set of indicators such as weighted coefficients associated to quantitative and qualitative attributes [30]. Such picture is then introduced into the system and helps determine possible contradictions. 2. Model description Let (, <) be a set of salaries ordered in ascending order, a set of roles, a set of contexts and ∶ × → ′ ⊆ a function assigning a collection of retributions to agent performing a certain role. Then the system is in presence of a contradiction when max(′)- min(′) > , where is an arbitrarily defined constant that we call the classification threshold [31]. It is assumed that further refinement of the set of skills and locators for the corresponding role and context will lead to a reduction in the constant , ultimately pointing to the skills and location of a unique agent. This, however, is implausible in real environments, and for this reason we introduced the arbitrary constant into the model. Such contradictions can be traced back to a single source, provided sufficient resources for analysis. 1. Incomplete role profile, correct context profile The set of skills or locators associated to a role or context corresponds to a subset of the total collection of skills or locators known to the system, which in turn makes possible the overall classification of roles and contexts. Contradictions can be mitigated and the classification threshold attained by adding skills to a role known or suspected to be incomplete. This additional dimension can be partitioned into different values according to its scale, and a reduced role can be obtained by selecting the corresponding element from the partition. 2. Complete role profile, incorrect context profile The same procedure can be applied in the case of incorrectly constructed context profiles. In particular, dimensions can be added to a locator in the context in order to refine the description of the time, place and other situational aspects. 3. Complete role profile, correct context profile Contradictions can also arise when both the role profile and the context profile are guaranteed to be correctly constructed. Assuming that every possible skill and locator is known to the system and available to be used, this scenario points to issues in the definition of the parameters of the system. In other words, the assessment of the agent’s skill levels and/or the locator value in the scale is not accurate and needs to be revised. On the other hand, there is the possibility that not all the skills or locators are known to the system, case in which it is necessary to engage in exploration of the entirety of the state space [32] in order to add the missing elements to the system. 3. Algorithmic implementation 1. A verification algorithm We now propose an algorithm destined to fix the profiles after the detection of a contradiction. Let be a set of (role and context) profiles and ∶ × → [0, +∞[ a semimetric [33] among the profiles. On the other hand, let ⊆ and ⊆ be a set of skills and locators respectively, both belonging to a set of elements and yielding = {(′, ′)}′⊆,′⊆. Then a set can be constructed such that each skill or locator will be labelled with the distance of its profile to a determined central profile. () = {(, (, ))| ∈ , ∉ ′, ∉ ′∀ = (′, ′) ∧ ≠ }. (1) Equation (1) allows to build an algorithm for gradually adding elements to profiles. Algorithm 1 The addElements function Input: a profile set , a set of profile elements, and a chosen ∈ Output: a modified profile ← (); a set of tuples of elements with distances ∗ ← (); the set in ascending order ′ ← (); the set of skills of profile ; the set of locators of profile ′ ← () for all ∈ do if ∉ ′ and ∉ ′ then .() if checkVerify(p) = true then return end if end if end for The importance of the function addElements (Algorithm 1) is due to the property of the academic crowdsourcing system of continually receiving feedback from new professionals. In particular, each added element in the function corresponds to a new skill or locator which needs to be asked to experts in the labor market in order to refine the system. Their continual contribution will allow to perform the algorithm repeatedly, not forgetting to check for the validity of the profile on each iteration by means of the function checkVerify (which yields if the desired classification threshold is attained). 2. An optimization algorithm An optional but recommended continuation of addElements in Algorithm 1 corresponds to a function to clean (optimize) the resulting profiles. Due to the continual addition of elements to a specific profile in order to mitigate a contradiction, an excess of skills or locators is generated, which in turn calls for an optimization. Algorithm 2 The cleanOptimize function Input: a profile set , a set of profile elements, and a chosen ∈ Output: a modified profile ′ ← (); the set of elements of profile ∗ ← (′); the set of combinations of elements of profile ← (); the set of combinations of elements of all profiles for all ∈ ∗ do if ∉ - ′ then = 0 for all ∈ do .() if checkVerify(p) = false then .() = + 1 end if end for if = || then set . = true end if end if end for return Concretely, the proposed function cleanOptimize (Algorithm 2) takes all the skills and/or locators and subdivides them in groups different from the element sets of other profiles. Afterwards, the function checks by subtraction (by means of the function checkVerify) if the eliminated elements are necessary skills or locators, leaving only the key elements in the profile. 4. Discussion 1. A cost balance equation We calculate the cost of performing the verification algorithm for a determined number of profiles in a time lapse, and compare it to the expected revenue from students’ tuition. Let = |′| + |′| + (|| - (|′| + |′|)) ∗ | - 1| the cost of obtaining the set () and = (|()| log(|()|)) the cost of ordering the set stud >= ∗ ∗ ( + + 4 ∗ |()| + 2|′| ∗ 2|′| - || ∗ (2 + 2||)). (2) Then the verification algorithm and its subsequent optimization can be performed along iterations as long as students’ tuition (stud) is equal or greater than the expression in Equation (2), where corresponds to a cost constant for ordering and is a defined constant for performing common operations. 2. Implementation outlook This paper is incorporated into a body of work related to the processes of recognition and verification under uncertainty. We plan to integrate such concepts into an expanded notion of academic life cycle management, where crowdsourcing will be utilized not only to individually assist students in achieving a certain goal, but also coordinate them collectively so that a certain institution can attain its own objectives with their help. We also study the implementation of means to store data and multimedia related to the academic platform, so that every bit of information needed by the students could be one day provided by the system without delay. In addition, we have also studied new ways to incorporate knowledge related to Big Data to correctly manage uncertainty from the recollection of the primary data. On the other hand, we plan to extend the knowledge gained in the academic crowdsourcing environment to other fields such as the legal one. In particular, we hope to implement a crowdsourcing management system capable of registering normative events and linking them in a network of judicial decisions assessed by a set of experts with varying degrees of expertise. 5. Conclusion In this work we presented an example of verification using a labor market as a domain theory. We outlined our verification approach based on the application of an academic crowdsourcing system, which serves as an introduction of new professionals into the open MAS of the job market. Students in the crowdsourcing system not only receive knowledge from experts, but also become experts according to their level of experience and knowledge. Once outside of the system, students become professionals and are expected to collaborate with the functioning of the system by providing feedback from the labor market. In particular, information about the work salary is recollected from professionals in different contexts of the job market, and later processed in the system in order to identify contradictions in the understanding of the domain. Such contradictions arise in the form of salaries unable to be classified due to high inconsistency and variation. In our model, we characterized agent roles by means of skills and context by means of locators. We proposed a method to tackle salarial contradictions, which is based in the gradual addition and/or removal of skills or locators in order to achieve a reduced salarial gap and attain the desired classification threshold. Based on our proposed algorithms, we found that the verification process can be repeated iteratively as long as the students’ overall tuition is equal or greater than a quantity defined by the sizes of the profile set and the element set, as well as the values of certain operational constants.

About the authors

Esteban J. Azofeifa

Peoples’ Friendship University of Russia (RUDN University)

Author for correspondence.
Email: azofeyfa-gomez-e@rudn.ru

PhD student at the Department of Information Technologies

6, Miklukho-Maklaya St., Moscow, 117198, Russian Federation

Galina M. Novikova

Peoples’ Friendship University of Russia (RUDN University)

Email: novikova-gm@rudn.ru

Candidate of Engineering Sciences, director of the Center of Development of Digital Technologies for Educational Processes and assistant professor at the Department of Information Technologies

6, Miklukho-Maklaya St., Moscow, 117198, Russian Federation

References