1 Introduction

Universities in the United Arab Emirates (UAE) are facing a challenging problem in terms of lack of effort, rigor and punctuality from the students, specifically within the architecture program as it is a creative degree which has less structure than other engineering processes. Additionally, Business Information Modelling (BIM) is an innovative development in the AEC industry that is increasingly in demand and more gradually being implemented in AEC curricula, which adds a layer of complexity to the problem. The usual result in such a circumstance is that the faculty are forced to lower expectations within their courses in order to create an environment where these expectations are in sync with the amount of work produced by the students.

In order to improve this situation and create a more competitive environment while improving student motivation, the authors changed grading and project submission policies in order to encourage students to take on the responsibility and ownership of creating their own schedule tailored to their abilities and interests. In this way the students will develop their knowledge at their own speed, allowing advanced students to progress faster and those needing more time to build confidence by giving them the tools to complete the course at their own pace.

In this experiment we created two different approaches, one linear approach where the students are required to follow a single path sequentially, with each step more challenging than the last; and another adaptive approach which has two paths – a linear one which encompasses the basics of the course and must be completed by every student, and optional assignments which will deepen student knowledge of each course segment and will allow for more credit to improve overall grades.

2 Related Work

2.1 Building Information Modelling (BIM)

Building Information Modelling (BIM) has been recognized as a pivotal point in the development of the Architecture, Engineering and Construction (AEC) industry. Whereas traditionally the various stakeholders involved in the design, construction and management of a building worked largely independently, BIM allows these parties to have real-time access to shared information in order to develop and produce a project cooperatively. It is not a single tool, rather it is a centralized database where information on a single model/project is uploaded, used, and downloaded using different tools. Common factors in different definitions of BIM include a common database; interoperable information; a whole-life informational model (from a building’s design to its demolition); and demonstrable advantages, namely increased efficiency and productivity [1].

In 2009 Farid stated, “Twenty years ago, AutoCAD pushed designers into a new era; BIM represents a new generation of virtual model already widely accepted by the industry” [2]. The adoption of BIM within the AEC industry has expanded, such that now many governments globally require the use of BIM, recognizing the clear benefits to costs and workflow [3]. Consequently focus has turned to academia, as AEC graduates and professionals need BIM skills and experience, however most curricula have not fully adopted the BIM methodology [4, 5].

One challenge for institutions in implementing BIM (and the new BIM Information Technology (IT) tools) in their departments has been logistical – whether it should be introduced in an individual course or across the entire curriculum. There is limited data on the practical implementation of BIM within the curriculum; [6,7,8,9,10] however some have suggested a tiered approach where BIM is introduced in a single subject, then across disciplines with subsequent emphasis on the collaboration and teamwork that BIM demands, culminating in actual construction projects in conjunction with AEC professionals in the final year [11, 12].

The demand for BIM in the AEC industry, the unique and innovative nature of BIM, and the lack of concrete data as to its implementation within the AEC curricula make this study of critical importance for further integration of BIM in academia.

2.2 Mixed Methods Research

A mixed method methodology is ideal for gathering and analyzing data from a smaller pool of subjects, as encountered in this study. As the name suggests, this method combines quantitative and qualitative aspects to accomplish a deeper understanding. Quantitative methods use objective questions to generate “hard” data that is useful on a large scale to generate statistics and create graphs or tables of the resulting analysis. Traditionally these methods are more associated with the natural sciences, statistics and other related studies.

The data from qualitative methods is more subjective, generated by observation, case studies, interviews, etc. which are commonly associated with the social sciences. The collection of the data is more labor-intensive, both in formulating the questions and collecting and analyzing the information generated. Because of this complexity, qualitative methods are usually conducted on a smaller scale. The advantages of using a mixed method over either method individually is that the two methods can work together to fill any gaps and produce more refined results [13]. Quantitative methods can enable the researcher to determine the critical elements that warrant further investigation, allowing the qualitative research to focus on those significant areas and provide richer data [14].

2.3 Adaptive Learning

The focus of adaptive learning is a customized learning experience that is adapted to each individual student, in order to give students a central role in their learning experience to boost motivation and maximize learning potential [15]. The student and instructor cooperate and interact to create the ideal learning environment, individually personalized. There is a range of possible approaches, but all are focused on tailoring the learning experience to each student in order to “stimulate his learning process and to encourage his involvement in this process [15,16,17,18]”.

Importantly, while latitude is given to students to shape the course to maximize their experience, the role of the instructor/tutor is still necessary for setting the criteria and configuring the course elements that are required; modification can include a range of elements, such as content covered, time needed to complete assignments, order in which topics are learned, assessment methodologies, interface options for introducing course material, etc. [15].

The interest in adaptive learning includes motivation of the student. Pintrich and De Groot propose three components to self-regulated learning – cognition (planning, monitoring and modifying), effort (management and control of focus), and cognitive strategies (methods students use to learn, remember and understand) [19]; however they likewise acknowledge the need for student motivation in order for a self-regulated, or otherwise individually-adapted, learning environment to be successful. They identify three motivational elements: expectancy, a student believes himself capable of completing a task (self-efficacy); value, a student understands and believes in the importance of, or has interest in, a task; and affective, or emotional reaction (e.g. test anxiety) [19].

Milosevic et al. addressed adaptive learning and motivation directly in a study where course material was designed to be delivered online and adapted to individual student’s learning style preferences and level of motivation. Motivation was determined by similar elements – intrinsic motivation, self-efficacy, engagement and test anxiety. Students with greater motivation and interest were designed lessons with more content covering multiple learning outcomes at greater depth at once, whereas those with less motivation were given core content material only, with additional in-depth material optional.

An adaptive learning system developed at the college level was shown to be effective in improving learning achievements and performance by Tseng et al. [20]. They designed a course in two manners – a traditional, sequential framework and an adaptive modular framework consisting of individualized sections of 38 learning objectives in total. Sixty-four students were given a pre-test to establish equivalency, then they were divided into two groups, experimental (adaptive, modular course) and control (traditional course). After completing the course a post-test was given, in which the experimental group showed a “significantly better” academic performance, and 91% of whom found the adaptive course material “suitable”. After completing the course analysis, and conducting a qualitative interview, results “concluded that the adaptive system is innovative, helpful, and well-developed enough to foster students’ learning” [20].

3 Data Analysis Methodology

The main objective of this study is to create, compare and analyze two similar adaptive learning methodologies, and compare and contrast them with previous traditional methodologies used in prior semesters. Two BIM construction courses (Construction II and IV) from the Architecture curriculum are redesigned and serve as the subjects of this study. Each course was changed from a BIM Project-Based Learning (PBL) format with a traditional methodology, to an adaptive learning process where the student can work their own path at their own speed, depending on their skills and interests, to complete the different assignments.

The analysis and comparison will be defined using a well-established mixed method (as more fully explained in the journal articles of Fonseca and ourselves [7, 13, 14, 21, 22]), using two surveys, grades, graded samples from course files and student interviews. All data gathered will be combined in a quantitative and qualitative methodology. The quantitative statistical analysis for short samples will be complemented by a qualitative analysis which supplies supporting and explanatory data. The combined analysis measures student motivation, satisfaction and performance in the courses, comparing and noting any improvements from previous semesters. It will also provide the student point of view throughout each process in order to highlight any strengths and/or weaknesses from the adaptive learning method.

3.1 Statistical Analysis Tools for This Research

The quantitative data obtained in this research will come from the surveys and grades, mentioned previously and explained in the referenced journals [7, 14]. In order to analyse the data, we need first to understand the type of data we will collect. The size of the classes is fewer than 25 people in each; such small samples cannot be tested correctly for normality, so it cannot be assumed. If normality of the sample is not proven, we are not able to use the P-test, T-test or ANOVA, as all these statistical tests are only accurate when used with normal distribution samples. Therefore non-parametric tests are utilized, which are more accurate for non-normal samples.

It must be noted that two kinds of data will be gathered in this study for analysis:

  • Different groups of students in the same course, re-designed with a different (non-traditional) methodology, in order to understand the pros and cons of adaptive learning in the course.

  • Different student groups at different points in their construction courses, to compare both of the adaptive learning methods proposed, in order to understand which one would be more successful in raising student motivation and performance.

This kind of data should be considered as independent samples, as we are testing the results in different study subjects. It will be tested two-by-two (as pairs), using the two samples comparison test developed by Wilcoxon (1945) [23] and Mann-Whitney (1947) [24], and as recommended by Fay, Proschan, Depuy and Neuhaser [25,26,27,28]. The described test will be used for analysis of the day-to-day study throughout the course.

For documents, requiring the analysis of a larger amount of data and comparing it in a common margin, we will follow a multiple pairwise comparison using the Steel-Dwass-Critchlow-Fligner procedure/two-tailed test. This test compares the median/means of all pairs of groups using the Steel-Dwass-Critchlow-Fligner pairwise ranking nonparametric method, and controls the error rate simultaneously for all k(k + 1)/2 contrasts [29].

In this manner we can test all the values for one variable at the same time for all the courses, thereby achieving an understanding of the overall evolution of the variable.

3.2 Qualitative Data

There are three kinds of qualitative data which we collect in this research: first, feedback extracted from the questionnaires; second, interviews after all students have finished their construction courses; and third, the graded samples.

In both pre-test and post-test questionnaires we introduce one final qualitative section where we will receive varying inputs from the students. In the pre-test this section will inquire about the different tools students think should be used to teach construction, design and other courses. In the post-test, it was initially decided that the students would be given the opportunity to highlight the best and worst elements in the course and how they could be (further) improved; however, ultimately we removed this section from the post-test. The reality is that most students are reluctant to answer these questions, and if included as mandatory in the online questionnaire most of them would write either very few things or comments that are not relevant; truly relevant issues will likely be the same as would appear during student interviews. This consideration prompted us to delete the qualitative questions on the post-test, and to focus the post-course qualitative data mainly in the interviews where there is a more accurate, genuine and relevant interaction with the students [30, 31].

It is important to highlight that the post-test questionnaires will also be administered after students finish the course, so that the students can be more sincere, honest, and open to talking about any issues encountered. The questions are directed to discover and understand: student opinions about the course; suggested improvements to the course; ideas for how to coordinate the use of different Information Technology (IT) tools within the course; their satisfaction towards BIM tools; their motivation and intention to use Revit in their future studies and work; and the need to implement Revit as a BIM software in the university.

Student interviews will be performed after finishing the course, so that the students can be more sincere, honest, and open to talking about any issues encountered. The interviews will be conducted in a relaxed environment at the AURAK campus under the direction of researcher Jose Ferrandiz. In these interviews the students will have the opportunity to explain their feelings and opinions about the course, their performance, and the new IT and why they believe they were successful (or not) in using it. The responses collected in the quantitative tests are on a scale of 1 to 5, without the possibility to explain (qualitative data); so after finishing the construction courses, when students have a comprehensive view and experience of the entire program, they are then given the opportunity to provide feedback and their sincere opinion about the process, the course, BIM and related new IT tools, how they dealt with these tools, and whether they will continue using them after this stage.

During the interviews the interviewer will note any comments, and immediately afterwards read them back to the interviewee to confirm accuracy, allowing for rectifications or modifications in the notes to ensure that all opinions are accurately represented. Following the interviews, all statements are categorized and organized in a way to generate meaningful conclusions and calculate percentages of the students interviewed who held one opinion or another.

Student graded samples will be collected and organized by course, group, grade and assignment. These samples are to be used to double check the grading, but most importantly it will provide a database allowing the comparison of completed courses and the introduction of BIM in other courses which were not a subject of this study.

The qualitative data is a critical element in a mixed method research because it provides a measure of depth to the consequences of our study that a purely quantitative data approach could not provide.

4 The Case Studies

As mentioned above two courses were re-developed for this study – Construction II and Construction IV, changing from traditional methodology BIM Project Based Learning (PBL), to an adaptive learning process. In actuality we created two different methodologies of adaptive learning in order to compare the data from both and determine which is more effective.

  1. 1.

    The first design has two paths. Two types of assignments are created for each chapter: type A are basic assignments providing concepts, fundamental understanding of the elements and basic skills; type B will provide advanced knowledge and skills in the same subject area.

    • a. When a student completes and passes a type A assignment, a type B assignment will be released from the same chapter in addition to the next type A (related to the next lecture).

  2. 2.

    The second design is linear. Course chapters will be arranged by complexity and each assignment will lead to the next one on a linear basis.

With the first design the student is able to develop basic skills for all the chapters studied, and obtain advanced knowledge on those topics where he has the interest and/or motivation to improve; in the second the student can use as much time as needed to complete each task but they must all be finished in order, therefore if a student has difficulty with one assignment it can negatively impact the rest of the course. Each method has its benefits, and their individual effect on student performance, motivation and satisfaction will be measured in order to understand which one would be more effective, and to determine the relative benefits and disadvantages of each.

4.1 Changing the Grading Criteria

In order to improve student motivation and effort, the grading criteria will be changed from the traditional way, where the students are not fully informed of their status and feel that they are losing critical points with each assignment, to a simplified grading system where students increase their grades depending on the knowledge acquired. This is stated in the syllabus as follows:

  • This course will be evaluated on a basis that is different from the rest of the courses. We will be using an adaptive learning environment, where the assignments will not have a specific due date. Every assignment will be evaluated from 1–10, and the student will need to earn at least 7 points in order to be able to continue and open the next assignment.

  • Each one of you can use the time you need to properly complete each task, depending on its requirements and your skills. You will need to learn and understand each chapter before advancing to the next task, assessed by the assignments. You won’t be able to advance to the next level until you complete the current one.

  • You must fulfil at least 60% of the assignments in order to pass the course, constituting a minimum of six assignments and the portfolio; after assignment 6, any completed assignment will increase your grade. The grading criteria will consider all grades, but it will always align with the following Table 1:

    Table 1. Comparison of the phases of the two frameworks
    Full size table

  • For those who seek to earn an A grade, all the assignments but one should be finished and graded with a minimum score of 9. There will be other optional assignments which will help to improve your grade, but these will not count as one of main ten.

Using this grading criteria the students will know their course standing and grade at all times throughout the course, and will know what they need to do in order to get a higher grade; it is anticipated that this will improve the organizational and self-responsibility skills of the students, who will have more ownership over their course performance and grading and a clear awareness of how to earn the desired grade.

The students will need to reach a minimum level at each stage in order to continue with the next concept; if they have not reached that level, they can review extra material that will be provided or ask for help from the instructor until they reach it. By this method it is suggested that student motivation will increase, while assuring that any student who passes the course has at a minimum a clear understanding of the basic concepts.

There are two possible benefits from these changes: first, this process will provide the student time to fully understand each concept at his or her own pace, as explained; second, and importantly, instead of potentially losing points with each assignment, which can frustrate some, students move forward and their grades progress as they advance in their studies, which should encourage them to work harder. While in the traditional system a student who earned a poor grade on initial assignments may find it difficult to get good grade in the course, by this adaptive process we will help them until they complete each assignment properly and earn the required marks, while ensuring they have a solid foundation in the course material.

4.2 Works Submission, Lectures, Examples and Extra Information

It is expected that the students will attend all the course lectures, and cover the same chapters. In order to accomplish this, although the assignments are related to the lectures these elements will be disconnected. Everyone in the class will experience the same lectures at the same time, but each student will work on the assignments to cement the understanding of each concept on their own path. This process doesn’t evaluate the student by their assignments, but by how much knowledge they have acquired during the semester.

In order to help in this process, the assignments are released only when the student has acquired the knowledge from the prior lesson, which will help them to advance. We have also prepared a complete package to help the student on the learning process, comprised of several items:

  • Lecture pdf, which allows the students to review the concepts explained during the class;

  • Examples of previous students’ work, which we also review and explain during the lecture, including the strengths and the weakness of each;

  • Checklist of the minimum requirements for the assignment to be completed;

  • Video tutorials of additional skills which could be helpful for the assignment;

  • Additional material – such as construction process videos, real examples, codes related to the assignment or any other relevant information available.

All this material is shown as a package at the same time as the assignment. Most of it is not new, as it will already have been discussed during the lectures and will be reviewed again during the lab sessions as one-to-one training with each student. It is very important to provide the material only for the current and previous assignments, so the student can use it without distractions or overload of materials/information. A lack of information can be a problem, but too much information without filters is likewise problematic and can overwhelm and/or distract the student from the target.

By this methodology the student can focus on one concept at a time until they reach mastery and then advance to the next step, while providing each student with their own time to take in the information and cement the knowledge as the foundation for the next step.

5 Conclusions

In furtherance of this research, we have created two different adaptive methodologies to be introduced in construction BIM courses at the degree of architecture, in order to improve student motivation, effort and performance/results. These methodologies provide the students the opportunity to learn and work at their own speed, allowing more advanced students to gain further knowledge while affording every student enough time to properly learn and internalize each concept.

These methodologies are analyzed using a mixed-method methodology which uses both quantitative and qualitative data in order to provide statistical analysis and greater understanding of the underlying causes.

The application of adaptive learning to the architectural degree, and mainly in a BIM project-based learning experience, is unique; therefore this research can lead to the improvement of the learning process at the Architectural program.