Keywords

1 Connecting Vulnerable Learners with Technology Jobs

More and more, communities leverage information and communication technologies (ICT) and infrastructure to empower themselves. Technology jobs support this, and are becoming ever more foundational to the modern job market [1]. However, vulnerable constituents (like the homeless) often lack access, adequate services, and institutional support to foster participation in the processes of attaining (fully) integrated communities. Homeless populations are rarely able to (re-)train for the skills they need to succeed in an increasingly science, technology, engineering, and mathematics (STEM)-driven job market. Although online education resources and tools such as MOOCs are available, some communities that could – under the right circumstances – benefit from this content are not yet connected to the tools in a meaningful way [2].

Disparities in accessing and completing online education are an example of how disadvantaged people, such as those experiencing homelessness, do not benefit from the opportunities of the digital age to the same degree as better-resourced individuals. Economic, social, and educational disparities reinforce – and are reinforced by – a lack of access to and knowledge about new ICTs. In addition to the more obvious challenges of their precarious situation, those experiencing homelessness may face a lack of connectivity [3], and lack of agency in daily scheduling while they work with various social and emergency services; this contributes to an atypical resume, which is a hurdle for job applicants [4]. This situation is a byproduct of the missing social capital that comes with not being firmly rooted in a community or work environment [4,5,6]. This research presents a model to address that gap for individuals experiencing homelessness.

This work proposes patching this missing social capital with sustainable education in STEM skills, through a work-learn model of micro-internships [7, 8]. ParticipantsFootnote 1 complete small real-world tasks and are paid as they learn, simultaneously supporting community connectivity and self-sufficiency. A key aspect of this model is the interaction of participants with online labor markets, and the ensuing transfer of credentials from online to offline scenarios. In our model, participants gain an in-demand skill set to bolster career prospects, and by working on real-world freelance tasks, they gain financial rewards. We hypothesize that remuneration incentivizes sustained participation [2] and decreases the time of dependency on emergency services. By connecting learning and assessments with well-known portals like UpWork and LinkedIn (which are linked), learners (re-)build credentials and digital resumes.

A fundamental research task (and indeed the purpose of the work at hand) is the operationalization of metrics that are both specific to homeless communities and generalizable to under-represented learners. Both types of metrics will be grounded in expert knowledge from community partners, data gathered from the community, and prior work in the field. Early iterations of the research will focus on qualitative research: descriptive metrics observing participants’ experiences in the program and their mastery of program material, interviews with and observations of participants as they proceed through the phases of the project, and information gathered from community partners about their experiences with participants and the project, and their perceptions of the program’s success. Additionally, metrics from the learning sciences will be used to gather information on attitudinal change such as increased interest and confidence in pursuing computing careers and on learning gains in the program.

This work is structure as follows: Sect. 2 illustrates the problem at hand and discusses and validates the proposed solution, the Mooqita learning platform. Section 3 describes how the program will be researched and evaluated. Section 4 addresses limitations and future research and project deliverables.

2 Problem-Solution Validation

Meaningful and sustainable employment is key to creating and maintaining financial stability [9], high well-being [10], and social connectivity [11]. Unemployment, underemployment, and low wages relative to cost of living are frequent causes of homelessness [12]. At the same time, vulnerable individuals face obstacles far more substantial than the general public at finding and maintaining meaningful employment and education. Moreover, not all employment is created equal: an estimated 25% of urban homeless are currently working or have worked in the recent past.Footnote 2 ICT careers have the potential to be more remunerative than the low-income jobs that are associated with the working homeless.

Underprivileged students, those who are not already highly resourced and well-educated, rarely succeed in MOOCs and often fail to complete courses. While MOOCs have the potential to provide valuable skills and knowledge, they require a consistent effort, time investment, and access to computing resources. Further, the assignments can be theoretical and divorced from authentic practices Recent research in online courses has shown that real world tasks increase retention and resilience even if no monetary incentive is involved, and that the quality of work was comparable to expert online freelancers [7]. Connecting learners with job training and a mechanism for demonstrable skills is critical to ensure they have the tools they need for long-term stability and success.

2.1 Barriers to Meaningful Employment

There are a number of pathways an individual may follow to employment. A standard pathway to employment generally starts with the credentialing process [13]. An individual is credentialed externally (i.e., by completing a school degree or certificate) or in-house (i.e., on the job learning), and in some cases like technical training schools, that process is combined. While in some number of career paths a combination of experience and education are proxied for ability, in the knowledge sector a degree of certificate is generally a minimum requirement. These credentials vouch that an individual can be expected to have a certain set of skills (content-oriented and social skills) [1]. The individual applies these skills, sometimes paired with others, in order to maintain continuous employment. In the optimal case this combination is sufficient for maintaining employment and financial security such that monthly salary is higher than monthly expenses. This is in fact the case of many knowledge sector careers. Underemployment occurs in the cases of over qualification (credential higher than requirement for position) and involuntary part-time work when full-time is desired (credential match but not employer/employee capacities). Underemployment can generally be understood as a contributing factor to those experiencing homelessness while currently in active the labor market [14, 15].

People experiencing homelessness often struggle with (untreated) mental illnesses, substance abuse, and other life traumas that contribute to their circumstances [14, 15]. Any individual factor is a known contributor to joblessness. Moreover, poor health outcomes and joblessness are reinforcing: when one occurs, the other supports its continuation [16]. Vulnerable constituents (e.g.: experiencing homelessness, former convicts, those with chronic health problems or disabilities) already combat various employment barriers [9]. In a competitive job market these barriers can become almost insurmountable, thus making it difficult to exit homelessness [4, 12]. Combining skill mismatch and/or low educational attainment with any of these factors is also a contributing factor to homelessness. In addition, individuals have a generally chaotic schedule while interacting with emergency or social services [12, 14], leaving many people experiencing homelessness without the ability to invest in long-term high-skill training opportunities. Such training is often time-intensive and requires a regular time commitment. The training opportunities which fit in well with a frequently-changing schedule are generally for service sector or labor-intensive jobs which tend to pay less than living wages.

The requirements to lowering barriers and increasing participation in knowledge sector work can be understood as the following:

  • The materials closely match in-demand skills of (local) industry such that a pathway to sustainable employment exists.

  • The learning materials are organized in a way that can flexibly meet individuals’ scheduling requirements.

  • The materials are broadly accessible in terms of time commitments, resources required for completion, and previous educational attainment.

  • Some sort of recognized credential is granted with successful completion.

We specifically focus on creating a solution addressing these points. The learning platform hosts content on the most up to date technological skills in demand, and is extendible to match niche market demands. Other than basic personal login data for credentialing purposes, Mooqita is open to any interested learner. The content is written in such a way that the minimum requirement is basic language and computer literacy. Finally, Mooqita links with internationally recognized services like UpWork and LinkedIn to provide learners with a digital portfolio. The platform aspects which fulfill the requirements are discussed in the coming sections.

2.2 Mooqita: A Tool to Support Online Learning on a Job

We lay out a four-phase plan to support participants in overcoming the aforementioned barriers to sustainable employment. The process helps participants gain self-efficacy, learn in-demand skill sets, engage with groups, and make initial steps in building a demonstrable work portfolio. Details about the Mooqita system can be found here [17].

The participants progress is documented online that allows tracking of learning success and display work samples [17]. Based on participant’s successes, Mooqita compiles a digital resume for participants highlighting and validating their abilities and achievements. This resume will ultimately be helpful in gaining job opportunities. The system also provides a peer review system to improve learning outcomes and assess the quality of individual assignments. An essential benefit of peer reviews is that students learn by providing feedback to peers [18]. Learners practicing revision skills strengthen their ability to identify and solve problems [19]. Feedback and practice are key elements in developing new skills [20] and gaining insight to better understand how one’s work is perceived by others [21]. This additional practice helps learners build their resume as the solutions and reviews are persisted and evaluated.

The first phase of the course focuses on igniting participants’ interest in the relevant topics while laying the foundation for learning computer science skills. Activities will provide opportunities for participants to establish self-efficacy and mastery orientation. Broadly concentrating on computational thinking skills, learners also gain the ability to analyze and problem solve in the future. In the second phase, we introduce technical skills such as the Python programming language. In the third phase, participants work on simulated real-world tasks from UpWork, such as implementing a provided design in HTML. Thus they will practice the gained skills within an online freelancing platform and with structuring a work schedule appropriate to such a platform, while continuing to learn. During these simulations, participants will be paid for their work. The specifics of the payment schemes are structured to match as closely as possible other career training or internship programs offered by our partners, to avoid providing an unfair incentive to participate.

In the last phase, participants will work independently on UpWork, receiving support from the project in finding and applying for the jobs. They can thus gain validated credentials from UpWork, improving their chances of getting future freelance jobs, and start building a resume that will be valuable in traditional job markets. Participants will also be encouraged to import their credentials to professional social networks as digital resumes; for example, LinkedIn and UpWork profiles can be linked directly.

The model is iterative, integrating feedback from our partners and learning community into necessary modifications for learning pathways. The Mooqita platform provides participants with an online environment that is able to persist and publish learning outcomes, achievements, and work experience. Every homework assignment is available in the system and participants can decide individually which assignment will be shown to employers. Together with the assignments and solutions, the system also stores peer reviews. The review process can be seen in Fig. 1.

Fig. 1.
figure 1

Mooqita’s review process

Full size image

In the current job market certain social skills are of grave importance. A core component of Mooqita is online team work. This helps participants to develop and demonstrate required social skills e.g., reliability, cooperation, or mentoring abilities. In contrast to other peer review systems Mooqita also allows to assess the quality of reviews. The system asks participants to provide feedback and ratings of reviews they received. As participants will write peer reviews for each other and rate the reviews they receive the system can aggregate indicators for these social skills. Additionally, participants perform group tasks in teams. For these group tasks the system provides an interface to collect feedback on the group experience of each participant.

2.3 Simulating Work

In the second phase, we select learning material from existing, validated online content and courseware, for example, Khan Academy for the basics, and edX and Coursera when participants are ready for more advanced training. This will allow participants to establish recognizable credentials, and will provide a known benchmark for learning outcomes. While learning relevant skills is important we want to provide a more holistic experience to participants. For that reason, we simulate the situation of online work environments. In the third phase, participants will start to work on real-world tasks on UpWork, to get practice with online freelancing, and with structuring a work schedule appropriate to such a platform.

Participants are using the Mooqita platform to post and work on tasks as described above. In the group setting participants will still provide reviews individually but submit solutions as a team. After the group assignment participants will reflect on their experience in the group assignment. They are asked to fill out a questionnaire on the platform.

Our team will identify typical freelance tasks that correspond to the course material. The team will select these tasks from online labor markets such as UpWork. Such tasks tend to be underspecified and require more detailed explanations to be useful as learning material. The onsite team will therefore provide additional information on how to solve tasks in this phase. Participants will work individually on their tasks and submit solutions to the Mooqita platform. They also provide reviews for their peers and rate the reviews they received (Fig. 2).

Fig. 2.
figure 2

Real-work tasks for learning in-demand skills

Full size image

2.4 Linking Learning to Careers

In the last phase, participants will work on their own on UpWork, receiving support from the project in finding and applying for the jobs. They can utilize the resume they created over the course to find jobs on UpWork. They can thus gain additional experience with actual work on online labor markets, improving their chances of getting future freelance jobs, and also start building a resume that will be valuable in more traditional job markets. Participants will also be encouraged to port their credentials to other professional social networks; for example, LinkedIn profiles can link directly to UpWork. While participants still continue building their skill set taking online courses using the Mooqita platform to persist their achievements. They also have an income through online freelance work. Participants now have the chance to transition between education and work as required.

3 Evaluation Criteria

A variety of tools will be implemented to evaluate the success of the program, particularly focusing on the challenges and progress of learners at each stage of the project. The effectiveness of each phase will be evaluated separately, collecting information from relevant stakeholders as to the effectiveness of the program and participants’ mastery of the material.

Each stage will include platform-internal metrics of participants’ progress through the program. In this way, time on task, success in completing in-platform activities and tasks, and overall progress and completion can be measured, as well as compared between participants in order to understand how differences in prior experience, attitudes, and demographics might impact learner success. Further, these metrics will allow revision and refinement to the curricular materials and supports. For example, if certain topics or assignments are proving challenging, extra help can be provided in the system, or the materials may be revised to address learner misconceptions and confusion. Overall success will be evaluated by participants’ overall completion of the modules and phases of the program. While students do not have to perform perfectly, minimum standards of mastery will be required for program success. Metrics of learner performance on the performance task will be the greatest measures of success – are students persisting to learn the material and to what extent are they successful in learning the material? Does success or failure in some elements affect persistence and mastery of other material? For example, success at the computational thinking skill of problem decomposition is likely to positively predict success in certain programming tasks. However, success in learning one language or environment, such as HTML or COBOL would not necessarily predict success in another such as Python. Understanding the paths and performance of learners will inform future work on the system.

Surveys will be used to assess participants’ interest, confidence, and attitudes towards computing, programming, and computing careers [22]. These will be distributed at the beginning of the project, and at the completion of each phase. In this way, the impact of the activities and project can be determined. In particular, it is expected that participants will have a greater increase in positive affect following the initial phase, designed to increase their self-efficacy. As participants continue through phase two and encounter challenges in learning the material, it is expected that their positive attitudes may decrease, though perseverance and success at tasks should mediate the impact of learning challenges.

A more in-depth form of data collection will be in the form of observations and interviews with participants. In-person observation will provide information about how participants interact with each other as well as non-verbal and other information about their interaction with the learning modules. For example, as a person proceeds through the materials, they may have times when they do not appear to be actively interacting with the system. This can be for many reasons. On-topic reasons include switching to a different window out of the system to look up additional information in more depth or discussion with other participants about the work. Off-topic reasons include distractions in or out of the room such as off-task discussions with other people or off-topic web browsing. Even off-topic distractions may serve useful if, for example, they serve to create a sense of community between learners and promote greater engagement. This can be measured through observation by researchers. Interviews will focus on participants’ expectations of the program and how well the program met their needs, as well as what they perceive as strengths and weaknesses and recommendations for change as well as elements of the program to keep the same. Understanding how participants experience the program, particularly what motivates them and what challenges they perceive will help inform future iterations.

In addition to participant evaluation, it is crucial to understand and evaluate the program in terms of outside stakeholders, specifically the site at which the program runs and the job-placement partners. Metrics will be collected on job placement for participants in the program – when and where they are placed and how successful they are at filling roles into which they are hired. The success metrics will be measured through post-program interviews with participants and also interviews or surveys of the job partners, to determine their impressions of the program, the qualifications of participants, and how the participants perform in the job. Surveys and interviews will also be developed and used with the service providers who host the trainings, to understand their perspective on challenges and successes, and inform future iterations of the project in how to collaborate with community partners to ensure the best outcomes for vulnerable populations.

Establishing appropriate metrics will allow the research to expand to other underserved populations. The metrics can also deliver insights into structural differences in skill development between vulnerable youth and adult learners. Understanding and contextualizing the data we collect via the platform, interviews and feedback, and job-placement tracking draws on multiple fields, including educational psychology, HCI, and social and cognitive science, along with community expertise harvested via participatory action research.

4 Limitations and Future Work

Recent research in online courses has shown that real world tasks increase retention even if no monetary incentive is involved [2]. Those results were obtained in an uncontrolled online setting without pay. We hypothesize that with monetary incentives effects will likely be much stronger. As we gain a better understanding of possible designs and parameters for future programs, we will explore approaches to making the model cost neutral, so it can scale effectively.