Skip to main content

Advertisement

SpringerLink
Log in

Performance-based testing for ICT skills assessing: a case study of students and teachers’ ICT skills in Finnish schools

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Skills enabling and ensuring universal access to information have been investigated intensively during the past few years. The research results provide knowledge on the differences and digital divides. When examining ICT skill levels, the accuracy of assessment is one of the key issues to address the results and gain the applicable data for appropriate interventions to enhance digital inclusion. To date, ICT skill assessment is based mainly on self-reports and subjective evaluations. However, as previous studies have shown, people tend to overrate or underrate their own levels of competence. Thus, novel performance-based approach for assessing ICT skills is presented in this paper. The ICT skill test contains 42 tasks grouped into 17 ICT fields. The study was conducted with upper comprehensive and upper secondary level school students (n = 3159) and their teachers (n = 626) during years 2014–2016 in Finland. Using factor analysis, three ICT skills factors were created: basic digital skills, advanced technical skills, and professional ICT skills. The performance in the ICT skill test was also divided by gender, as the male students and teachers outperformed the female students and teacher. Outperformance also occurred by educational level, as both upper secondary level students and teachers were seen as possessing higher-level ICT skills than students and teachers at the comprehensive level. We thus argue that to compare the ICT skills and the validity of the assessments, we needed to ensure consistent assessment for both students and teachers. In addition, in order to diminish the ICT skill gap among students, interventions using formal education are urgently needed, and in particular, more attention should be given to both teacher training and in-service training.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. van Deursen, A.J.A.M., Courtois, C., van Dijk, J.A.G.M.: Internet skills, sources of support, and benefiting from internet use. Int. J. Hum.-Comput. Interact. 30(4), 278–290 (2014)

    Article  Google Scholar 

  2. Livingstone, S., Helsper, E.J.: Gradations in digital inclusion: children, young people, and the digital divide. New Media Soc. 9(4), 671–696 (2007)

    Article  Google Scholar 

  3. Kivinen, O., Piiroinen, T., Saikkonen, L.: Two viewpoints on the challenges of ICT in education: knowledge-building theory vs. a pragmatist conception of learning in social action. Oxf. Rev. Educ. (2016). doi:10.1080/03054985.2016.1194263

    Google Scholar 

  4. Hargittai, E., Walejko, G.: The participation divide: content creation and sharing in the digital age. Inf. Commun. Soc. 11(2), 239–256 (2008)

    Article  Google Scholar 

  5. Ainley, J., Fraillon, J., Schulz, W., Gebhardt, E.: Conceptualizing and measuring computer and information literacy in cross-national contexts. Appl. Meas. Educ. (2016). doi:10.1080/08957347.2016.1209205

    Google Scholar 

  6. Ferrari, A.: DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe. Publications Office of the European Union, 2013, European Union, 2013, Luxembourg (2013). http://jrc.es/EURdoc/JRC83167.pdf

  7. Binkley, M., Erstad, E., Herman, J., Raizen, S., Ripley, M., Miller-Ricci, M., Rumble, M.: Defining 21st century skills. In: Griffin, P., McGaw, B., Care, E. (eds.) Assessment and Teaching of 21st Century Skills, pp. 17–66. Springer, Dordrecht (2012). doi:10.1007/978-94-007-2324-5

    Chapter  Google Scholar 

  8. Fraillon, J., Schulz, W., Ainley, J.: International Computer and Information Literacy Study: Assessment Framework. IEA, Amsterdam (2013)

    Google Scholar 

  9. van Deursen, A.J.A.M., Helsper, E.J.: The third level digital divide: who benefits most from being online? Commun. Inf. Technol. Annu. 10, 29–52 (2015). doi:10.1108/S2050-206020150000010002

    Article  Google Scholar 

  10. van Dijk, J.A.G.M.: The Deepening Divide: Inequality in the Information Society. Sage, London (2005)

    Google Scholar 

  11. Siddiq, F., Hatlevik, O., Olsen, E., Throndsen, R.V., Scherer, R.: Taking a future perspective by learning from the past—a systematic review of assessment instruments that aim to measure primary and secondary school students’ ICT literacy. Educ. Res. Rev. 19, 58–84 (2016)

    Article  Google Scholar 

  12. Merrit, K., Smith, D., Renzo, J.C.D.: An investigation of self-reported computer literacy: is it reliable. Issues Inf. Syst. 6(1), 289–295 (2005)

    Google Scholar 

  13. Umar, I., Yusoff, T.: A study of Malaysian teacher’s level of ICT skills and practices, and its impact on teaching and learning. Soc. Behav. Sci. 116, 979–984 (2014)

    Article  Google Scholar 

  14. van Deursen, A.J.A.M., Helsper, E.J., Eynon, R.: Development and validation of the Internet Skill Scale (IIS). Inf. Commun. Soc. 19(6), 804–823 (2016). doi:10.1080/1369118X.2015.1078834

    Article  Google Scholar 

  15. FNBE: National Core Curriculum 2004. Finnish National Board of Education, Helsinki (2004). http://www.oph.fi/english/curricula_and_qualifications/basic_education/curricula_2004

  16. van Dijk, J.A.G.M., Hacker, K.:The digital divide as a complex and dynamic phenomenon. In: Annual Conference of the International Communication Association, Acapulco, 1–5 June 2000

  17. Hargittai, E., Hsieh, Y.P.: Digital inequality. In: Dutton, W.H. (ed.) Oxford Handbook of Internet Studies, pp. 129–150. Oxford University Press, Oxford (2013)

    Google Scholar 

  18. van Deursen, A.J.A.M., van Dijk, J.A.G.M.: Internet skills and the digital divide. New Media Soc. 13(6), 893–911 (2011)

    Article  Google Scholar 

  19. Hatlevik, O.E., Guomundsdóttir, G.B., Loi, M.: Digital diversity among upper secondary students. Comput. Educ. 81(February 2015), 345–353 (2015). doi:10.1016/j.compedu.2014.10.019

    Article  Google Scholar 

  20. Goode, J.: The digital identity divide: how technology knowledge impacts college students. New Media Soc. 12(3), 497–513 (2010)

    Article  Google Scholar 

  21. Helsper, E., Eynon, R.: Digital natives: where is the evidence? Br. Educ. Res. J. 36(3), 503–520 (2010)

    Article  Google Scholar 

  22. Claro, M., Preiss, D., San Martin, E., Jara, I., Hinostroza, J.E., Valenzuela, S., Cortes, F., Nussbaum, M.: Assessment of 21st century ICT skills in Chile: test design and results from high school level students. Comput. Educ. 59, 1042–1053 (2012)

    Article  Google Scholar 

  23. Gui, M., Argentin, G.: Digital skills of internet natives: different forms of internet literacy in a random sample of northern Italian high school students. New Media Soc. 13(6), 963–980 (2011)

    Article  Google Scholar 

  24. Thompson, P.: The digital natives as learners: technology use patterns and approaches to learning. Comput. Educ. 65(1), 12–33 (2013)

    Article  Google Scholar 

  25. Litt, E.: Measuring users’ internet skills: a review of past assessments and a look toward the future. New Media Soc. 15(4), 612–630 (2013)

    Article  Google Scholar 

  26. Livingstone, S., Helsper, E.: Balancing opportunities and risks in teenagers’ use of the internet: the role of online skills and internet self-efficacy. New media Soc. 12(2), 309–329 (2010). doi:10.1177/1461444809342697

    Article  Google Scholar 

  27. Hargittai, E.: Beyond logs and surveys: in-depth measures of people’s web use skills. J. Am. Soc. Inf. Sci. Technol. 23(14), 1239–1244 (2002)

    Article  Google Scholar 

  28. van Deursen, A.J.A.M., van Dijk, J.A.G.M.: Improving digital skills for the use of online public information and services. Gov. Inf. Q. 26(2), 333–340 (2009)

    Article  Google Scholar 

  29. Eshet-Alkalai, Y., Amichai-Hamburger, Y.: Experiments in digital literacy. CyberPsychol. Behav. 7(4), 421–429 (2004)

    Article  Google Scholar 

  30. Hakkarainen, K., Ilomäki, L., Lipponen, L., Muukkonen, H., Rahikainen, M., Tuominen, T., Lakkala, M., Lehtinen, E.: Students’ skills and practices of using ICT: results of a national assessment in Finland. Comput. Educ. 34(2), 103–117 (2000). doi:10.1016/S0360-1315(00)00007-5

    Article  Google Scholar 

  31. Viteli, J.: Teachers and use of ICT in education: pilot study and testing of the Opeka system. In: Herrington, J., et al. (eds.) Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2013, pp. 2326–2346. AACE, Chesapeake (2013)

    Google Scholar 

  32. FNBE: Finnish Education in a Nutshell. Education in Finland. FNBE, Helsinki (2012). http://www.oph.fi/download/146428_Finnish_Education_in_a_Nutshell.pdf

  33. Christmann, A., Van Aelst, S.: Robust estimation of Cronbach’s alpha. J. Multivar. Analysis 97, 1660–1674 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  34. Calvani, A., Fini, A., Ranieri, M., Picci, P.: Are young generations in secondary school digitally competent? A study on Italian teenagers. Comput. Educ. 58, 797–807 (2012)

    Article  Google Scholar 

  35. OECD: Pisa 2009 Results: Students On Line. Digital Technologies and Performance, vol. VI. OECD, Paris (2011)

    Google Scholar 

  36. Tømte, C., Hatlevik, O.E.: Gender-differences in Self-efficacy ICT related to various ICT-user profiles in Finland and Norway. How to self-efficacy, gender and ICT-user profiles relate to findings from PISA 2006. Comput. Educ. 57(1), 1416–1424 (2011)

    Article  Google Scholar 

  37. Ilomäki, L.: Does gender have a role in ICT among Finnish teachers and students? Scand. J. Educ. Res. 55(3), 325–340 (2011)

    Article  Google Scholar 

  38. Hargittai, E.: Digital na(t)ives? Variation in internet skills and uses among members of the ‘net generation’. Sociol. Inq. 80(1), 92–113 (2010)

    Article  Google Scholar 

  39. OECD: The 2012 Programme for International Student Assessment (PISA), Computers, Education and Skills. OECD Publishing, Paris (2012). doi:10.1787/9789264239555-en

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank all the students and teachers who participated the ICT assessments. The work was supported by the University of Turku.

Author information

Authors and Affiliations

  1. Research Unit for the Sociology of Education (RUSE), University of Turku, Turun Yliopisto, 20014, Turku, Finland

    Meri-Tuulia Kaarakainen, Osmo Kivinen & Teija Vainio

  2. School of Management, University of Tampere, Tampereen yliopisto, 33014, Tampere, Finland

    Teija Vainio

Authors
  1. Meri-Tuulia Kaarakainen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Osmo Kivinen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Teija Vainio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meri-Tuulia Kaarakainen.

Appendix 1

Appendix 1

Field

No. of items

Type of item

Description of items

Basic operational skills

2

Multiple choice

There were two tasks. First, the participants had to choose between seven alternatives, namely the right answers (2) to the question: ‘How do you type special characters, which are not included in the QWERTY keyboard (i.e. Θ, ¾, ¥, ∉)’. In the second task, participants had to choose, between seven alternatives, namely which items were the correct answers to the question: ‘Which two statements about the clipboard are correct?’

Information seeking

5

Multiple choice and open-ended questions

Participants had to choose between four alternatives, namely the best information sources for the given situations (all together three tasks), from a search query for a ‘web search engine’ (Task 1), and how to evaluate and choose relevant and reliable outcomes for a given problem from the ‘search engine results page’ (Task 2)

Word processing

2

Multiple choice (matching)

In the first task, participants had to choose between five alternatives for what modifications (paragraph and page formatting, header and footer) were presented for the text documents. Then, similarly in the second task, they had to choose, between five alternatives for the desired modifications to implement (indexing, page numbering, and page break)

Spreadsheets

3

Multiple choice (matching)

Participants had to choose between four alternatives for the right formula for the spreadsheet cell, and similarly, between four alternatives, for the right function to solve the presented task. In addition, they needed to choose between five alternatives for the appropriate formatting actions to implement (formatting and ordering of cell content)

Presentations

2

Multiple choice (matching)

Participants had to choose between four alternatives for the best implemented actions to take on the presented slide shows (by inserting background and bullets/numbering, formatting charts and graphics) (two tasks total)

Image processing

2

Multiple choice (matching)

Participants had to choose between five alternatives for the best implemented formatting actions to present images (for formatting brightness and colours, cropping the picture, and/or removing elements from the image)

Social networking

4

Multiple choice (case examples)

Participants had to choose, between four alternatives for the most appropriate and secure option for the social networking case (four tasks)

Web content creation

2

Multiple choice (matching)

In the first task, participants had to choose from five html outputs the correct match for the given html code (a simple example containing text, link, input field, and font colours). In the second task, participants had to choose between seven alternatives for the correct answer to the question: ‘Which two statements about the (Finnish) exercise of freedom of expression in the mass media are correct?’

Software installation and initialisation

2

Multiple choice

Participants had to choose between seven alternatives for the right answers to two questions: ‘Which two statements about (options for installation, and installation of media in) software installation are correct?’ and between the four alternatives for the question ‘Which statement about (operations needed during the) software installation is correct?’

Operating system installation and initialisation

2

Multiple choice

Participants had to choose between four alternatives for the right answers to the questions: ‘Which statement about operating system installation and initialisation is correct?’ (two tasks)

Maintenance and updating

2

Multiple choice

Participants had to choose between seven and four alternatives for the right answers to the question: ‘Which statement(s) about maintenance and updating are/is correct?’ (two tasks)

Information security

2

Multiple choice

In the first task, participants had to choose between four alternatives for the correct action/conclusion in the case where web service store user passwords in a clear text format. In the second task, they had to choose, between seven alternatives for which two options were not proper information security methods

Programming

4

Multiple choice

In the programming tasks, participants needed to choose between four alternatives to answer what the given code examples (2 pseudocode examples) does, and what were the values of these variables after running (2 questions)

Database operations

2

Multiple choice (matching)

Participants had to choose between four alternatives for the correct SQL query for a given situation. In addition, they had to choose between four alternatives for the correct description for the presented database schema

Information networks

2

Multiple choice (matching)

In the first task, participants had to choose between four alternatives for the right answers to the question: ‘Which statement about denial-of-service attack is correct?’ In the second task, participants had to recognise information network techniques and then choose the correct match from four alternatives for the presented network graph

Server environments

2

Multiple choice

Participants had to choose between five and four alternatives for the correct statements regarding logical volume management (Task 1) and hot-swapping (Task 2)

Digital technology

2

Multiple choice (matching)

In the first task, participants had to choose between six alternatives for the best match for the presented graph on logic gates. In the second task, they had to choose between five alternatives for the correct answer to the question: ‘On which branch of mathematics is digital technology based?’

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kaarakainen, MT., Kivinen, O. & Vainio, T. Performance-based testing for ICT skills assessing: a case study of students and teachers’ ICT skills in Finnish schools. Univ Access Inf Soc 17, 349–360 (2018). https://doi.org/10.1007/s10209-017-0553-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-017-0553-9

Keywords

Search

Navigation