Norsk IKT-konferanse for forskning og utdanning

Forord til UDIT 2024

Per Lauvås — 2024-11-24

Verdien av et tverrfaglig semester, slik studentene ser det

Tengel Sandtrø — 2024-11-24

På politisk nivå, og innen privat næringsliv, omtales tverrfaglighet som en del av løsningen på gjenstridige problemer. Et tiltak kan være å tilby tverr-faglige utdanninger, men disse byr på noen utfordringer; dybdeforståelsen kan måtte ofres når tiden må brukes på å få oversikt over et mye bredere felt av fag eller disipliner. Vinteren 2024 blusset det opp en diskusjon om tverrfaglige utdanninger, da det ble antydet at noen handlet mer om student-rekruttering enn egenverdi. Denne artikkelen tar for seg et tverrfaglig semes-teremne, og dokumenterer hvordan prosjektbasert læring i denne konteksten fungerer som pedagogisk verktøy for å oppnå høy opplevd verdi for tekno-logistudentene som deltar. Målsetningen for semesteret er i stor grad sam-menfallende med målene for tverrfaglige utdanninger, med den ulikheten at i det aktuelle emnet er studentene allerede i sitt tredje eller fjerde år i en inge-niør- eller teknologiutdanning. Prosjektbasert læring (PjBL) har både ut-fordringer og styrker, og studentenes egen opplevelse av verdi er ikke et per-fekt mål. Likevel argumenterer artikkelen for at PjBL er et godt pedagogisk verktøy og at studentene har en realistisk og relevant forventning til denne typen emner.

Role Models as an Intervention for Gender Diversity in Computing Education

Anna Szlavi — 2024-11-24

Context: The lack of gender diversity has been a persistent challenge for computing fields for decades now. Both in education and in the workforce, men are in a significant majority, which poses a threat not only to social fairness and gender equality, but also to the quality and inclusiveness of new technology produced by computing teams. Efforts have been made to improve diversity, equity, and inclusion in the computing field, especially in computing education. Studies suggest that role models would be a helpful intervention for improved gender diversity and equality, but little research has been done to explore the concept more in-depth and specifically within the computing education context.
Objective: The research objective of this paper is to explore the impact of role models on gender inclusion in computing education. The lack of thorough research on role models in the field has motivated the following research questions (RQs):
RQ1: What recent research exists that explores the impact of role models on computing students?
RQ2: Who functions as role models in computing education?
Method: The method used to answer the RQs is a systematic literature review (SLR).
Results: The SLR included 16 primary papers published between 2015 and 2023. The results, presenting what current literature exists regarding the impact of role models on computing students, reveal a lack of focus on non-US-based contexts and on the retention, not only the recruitment, of non-male identifying students to computing. The paper also provides insight into who are considered to be role models in the context of computing education, what attributes appear important, and why they are considered role models.
Conclusion: The results indicated a scarcity of global research on the usefulness of role models for gender diversity within computing education, which calls for further explorations. Teachers and older students are the most commonly identified role models, and they work as role models by breaking down stereotypes and other barriers faced by girls in computing, but certain features must be present in order for them to fulfill such roles. Relatability in gender, age, and passion was identified as significant attributes of role models.

Using Microsoft Copilot Chat in the Work of IT Educators: Pilot Study

Kateryna Osadcha — 2024-11-24

Artificial intelligence (AI) chatbots have recently emerged in nearly all aspects of life, including education, where educators are exploring their potential. However, achieving proficiency remains a challenge due to numerous hurdles. This article aims to identify the problems IT educators face when using AI chatbots and propose solutions. We used Scopus, Web of Science, and Scopus AI to explore how IT educators use AI chatbots and the problems they face. Additionally, we developed and implemented the Microsoft Copilot Chat user guide, and conducted a workshop at our institution on using Microsoft Copilot Chat for professional activities of IT educators. Findings from the literature and the pre- and post-workshop surveys provided firsthand insights into educators' knowledge, usage patterns, and the drawbacks of using Microsoft Copilot Chat. The literature review shows that educators mainly use AI to generate ideas, grade assignments, and automate tasks, while also valuing AI chatbots for personalized learning. However, problems include a lack of guidelines, identifying AI use in student work, and potential impacts on critical thinking. Additionally, a workshop on the use of Copilot for professional activities was conducted, accompanied by pre- and post-surveys with questions inspired by findings from the literature. One of the important findings of the surveys is that most educators use AI chatbots and are knowledgeable about their capabilities, except for areas like image quality assessment and emotion detection. Lastly, McNemar test was conducted to assess the workshop's impact, revealing significant improvements in participants' knowledge of AI chatbots capabilities and suggesting a positive influence on their future use of these tools. Conclusions were made regarding the improvement of the Copilot guide content and method of conducting the workshop for future research.

Trenger vi læringsassistenter? Hvordan kan hverandre-vurdering og/eller egen-vurdering gi god læringsutbytte.

Omid Mirmotahari — 2024-11-24

Denne studien undersøkes effekten av lærervurdering (LV), egen-vurdering (EV) og hverandrevurdering (HV) på læringsutbyttet til 528 studenter i et introduksjonskurs i informatikk. Studentene ble tilfeldig fordelt i tre grupper, hvor hver gruppe mottok en annen type tilbakemelding på en obligatorisk oppgave. Resultatene viste at alle intervensjonene førte til en signifikant forbedring i studentenes prestasjoner fra første utkast til endelig innlevering (p < 0.05). HV-gruppen hadde den største gjennomsnittlige økningen i poeng (M = 0.52), etterfulgt av LV (M = 0.45) og EV (M = 0.36). På eksamen oppnådde HV-gruppen den høyeste gjennomsnittlige poengsummen (67.08%), sammenlignet med EV (62.85%) og LV (62.59%). Disse funnene fremhever potensialet til alternative vurderingsmetoder, spesielt hverandrevurdering, som effektive verktøy for å fremme læring og prestasjoner. Studien bidrar til forskningen på vurdering i høyere utdanning og gir innsikt i fordelene og utfordringene ved hver tilnærming. Resultatene har viktige implikasjoner for vurderingspraksis og understreker behovet for å kombinere egenvurdering og hverandrevurdering med tradisjonell lærervurdering for å skape et mer helhetlig og studentsentrert vurderingsmiljø.

Applying Liljedahl’s Thinking Classrooms in a Higher Education Digital Technology Course

Terje Idland — 2024-11-24

This paper explores student learning experiences in the ICT course Digital Technology using one specific pedagogical approach, Pe- ter Liljedahl’s ”Thinking classrom”. We assigned 25 students to random groups to solve a network problem on whiteboards and conducted in- depth interviews with 10 participants. Thematic analysis revealed im- proved communication and academic focus, though knowledgeable stu- dents still took more active roles. While most students felt engaged, some noted drawbacks such as increased energy demands, awkwardness, and reduced autonomy, and expressed concerns about using this method for summative assessments.

Programmeringsparadokset etter Sfard og Leron

Oline Sæverud Aksnes — 2024-11-24

Sfard og Leron (1996) observerte at studentar arbeider flittigare og oftare lukkast når dei programmerer enn dei gjer med analytisk matematikk, sjølv om programmeringsoppgåva løyser det same matematiske problemet i ein meir generell form. Her går me gjennom relevant litteratur og teori for å drøfta kvifor det kan vera slik. Dette reiser interessante spørsmål både om korleis me kan bruka programmering for å styrka matematikkforståinga og korleis ein best lærer programmering.

Students' perceptions toward essential functionalities and qualities of peer code assessment tools

Somayeh Bayat Esfandani — 2024-11-24

Peer code assessment (PCA) empowers computer science students, enhancing their learning and equipping them with practical skills for industry work. However, instructors often face a scarcity of well-designed tools customized for the learning environment. Additionally, there is a knowledge gap concerning the latest generation of peer assessment tools. So, the question is, how could a PCA tool be designed to support students’ programming learning experience? A case study was conducted using the PeerGrade tool, employing interviews and observation as data generation methods to answer this question. Research aimed to identify features that enhance student learning and uncover essential qualities of peer code assessment tools from students’ perspectives. Informants considered inline comments, general comments, rubrics, threads, and Code editor functionalities essential. They also highlighted the importance of utilizing a customized, user-friendly tool with a step-by-step process. The Self-Regulated Learning conceptual theory has been used as a theoretical lens. Based on SRL, it has been found that implementing identified tool qualities and features will improve the learning experience by increasing students’ motivation and enabling them to follow their learning strategies. The findings can be used as a design principle for developing peer code assessment tools.

A Case Study on Student Perspective of Peer Code Review (PCR)

Attiqa Rehman — 2024-11-24

Peer Code Review (PCR) is a professional practice and a learning method. A case study on PCR was conducted in a “Programming Languages” course in the fall semester of 2023 at the Norwegian University of Science and Tech-nology (NTNU). A new protocol for peer code review was implemented where the students received a suggested solution and instructor feedback on their solutions before reviewing their peers’ solutions. The motivation for the protocol was to reduce the students’ cognitive load, allowing them to focus on assessing the code produced by their peers. A survey among the students showed that they engaged in the PCR and found the workload rea-sonable. The survey also indicates that students found the review work an opportunity for learning even under the new protocol.

remarks - Machinery for Marking Student Work

Oleks Shturmov — 2024-11-24

remarks is an open-source suite of tools for marking student work. It has been in moderate use for assessing semi-structured submissions in a handful of Computer Science courses with hundreds of students since 2016. The contemporary approach to systematically collaborate on such assessments is to use general-purpose administrative software (e.g., spreadsheets, text files or documents, Emacs Org-mode). Since this software has not been expressly designed for the purpose, it tends to require non-trivial technical skill to achieve but mediocre technical support for the assessment process. In particular, it is hard to achieve (1) support for criteria-based analytic marking, without precluding (2) holistic, free-form justifications for the given marks, while enabling (3) decentralized collaboration with a reliable and transparent synchronization mechanism. remarks has been expressly designed to meet these criteria. Guided by the theory of assessment and evaluation in higher education, it has grown from a series of scripts surrounding general-purpose administrative software to a dedicated tool suite for marking student work. With remarks, assessing a unit of work constitutes filling in a document whose structure is guided by, but not limited to a chosen marking scheme. Assessors can use a contemporary source code revision control system (e.g., Git) to collaborate on these documents; making conflict resolution explicit and unsurprising. Using one document per unit of work, further reduces conflicting edits, in general. The remarks tool can then derive summative and descriptive assessments from such documents. We describe and justify the design of remarks, report on our experiences with the approach, and invite you to help develop the ideas further.

Students as developers of educational tools - What educational challenges do they address and which learning approaches do they implement?

Line Kolås — 2024-11-24

Although game development students typically lack a formal pedagogical education, they have clear opinions and concrete ideas on improving higher education learning practice. In Excited, the Centre for Excellent IT Education, game development students were invited to pitch ideas of educational tools to improve higher education. Then, some of the student teams were offered summer jobs to implement the ideas. The study has investigated their products, and through an explorative qualitative artifact analysis, a sample of 6 student-created educational tools have been analyzed. The thematic analysis answered the following research questions; 1) What educational challenges do the students address when creating tools to improve higher education? 2) What learning approaches do students implement when they develop educational tools? The findings show that the student-created educational tools address both soft and hard skills, and exemplifies a complex learning reality for students with a necessity of learning both industry-relevant technologies, and also addressing the necessity of learning soft skills like team management. The findings also reveal a wide variation in the complexity of the educational challenges students address, evident in the scope and depth of the learning objectives embedded within the tools, including vocabulary acquisition and understanding simple commands as well as simulating intricate processes navigating a multifaceted environment. Concerning learning approaches, the analysis shows that the students have implemented tools focusing on content creation, process management, exploration, competition, collaboration, articulation and communication. The study shows how student involvement can be implemented through a role as developers of learning tools, not only as information providers, actors, experts, and partners.

OnlineProver: First Experience with Teaching Formal Proofs

Joachim Kristensen — 2024-11-24

OnlineProver is an interactive proof checker, tailored for the educational setting. The main features are a user-friendly interface for editing and checking proofs, and a DSL for specifying deduction systems and exercises about them. The user interface
provides feedback directly in the derivation, based on error messages provided by a proof checking web-service. A basic philosophy of the tool, is that it should aid the student, while still maintain that the students construct the proofs as if it was on paper. We gathered feedback on the tool through a questionnaire, and we provide a comprehensive evaluation of its effectiveness, assessing its impact on teaching and learning approaches, along with technical aspects.
The evaluation showed that the students were satisfied with using OnlineProver as part of the teaching and gave a first conformation of the learning approach behind. This gives clear directions for future developments.

Improving Fairness and Quality in Master's Thesis Assessment in Norwegian Higher Education

Mohsen Toorani — 2024-11-24

Assessment of master's theses is a pivotal aspect of higher education, significantly influencing students' academic success and future career opportunities. In Norway, discussions on fairness, objectivity, and the integrity of thesis evaluations have highlighted challenges in ensuring impartial assessment practices, especially as universities are legally mandated to maintain transparency and impartiality. This paper examines factors that may affect the objectivity of thesis evaluations in the Norwegian context, drawing on research and case studies to highlight challenges and propose solutions. Addressing these issues is essential for upholding the credibility and fairness of higher education. By implementing strategies such as standardized assessment criteria, comprehensive examiner training, public defenses, and technology-driven approaches, Norwegian universities have the potential to set a global standard for equitable thesis evaluations. The insights and recommendations presented aim to guide policymakers, educators, and academic institutions in enhancing assessment practices to ensure fairer outcomes for all students.

Rettferd i variantoppgåver med tilfeldig trekking

Guttorm Sindre — 2024-11-24

I automatiserte prøver som studentar skal kunne ta om att mange gonger, er det behov for å lage store mengder oppgåver som blir trekte tilfeldig frå gong til gong. Ved formativ bruk av prøvene trengst variantar for å unngå at stu-dentar etter kvart svarer rett på alt berre ved å pugge svar, og ved summativ bruk av prøvene trengst variantar for å hindre at gjentak gir ein urettvis for-del framfor å ta prøva på første forsøk. Eit mogeleg problem med tilfeldig trekking mellom mange variantar, er dersom variantane har ulik vanskegrad, slik at studentar kan vere meir eller mindre heldige med kva oppgåver som blir trekte for dei. Denne artikkelen ser på nokre variantoppgåver som blei brukte både i formative treningstestar og summative meistringstestar i eit innleiande programmeringskurs i Python for førsteårs universitetsstudentar, meir spesifikt oppgåver som skulle teste forståing av if-setningar i Python. Spørsmåla vi ser på er følgjande: 1) Kva analyser kan faglærar gjere for å sjekke om oppgåvevarianter er rettferdige? 2) Var oppgåvene rettferdige ved dette konkrete høvet? 2) I den grad det blei funne variasjon i vanskegrad mellom variantar, kva var årsaka til denne? Spørsmåla blei undersøkte ved statistisk analyse av anonymiserte studentprestasjonar oppgåvene. Funn ty-der på at jamvel om kodekompleksiteten var lik i alle variantane, var det noko variasjon i vanskegrad. Mogeleg forklaring kan liggje i samanhengen mellom koden og den tekstlege utforminga av oppgåveteksten eller svaral-ternativa. Artikkelen avsluttar med idear til korleis oppgåvene kunne ha vore gitt på ein meir rettferdig måte.

Automated adaptive testing vs. linear testing in undergraduate mathematics

Øyvind Aas — 2024-11-24

We conduct a lab-based randomized controlled trial with 47 undergraduate students in mathematics, comparing an automated adaptive testing system, which adjusts difficulty based on performance, to traditional linear tests. Results shows that students using the adaptive test scored 26.2 percentage points higher on a subsequent exam than those in the linear test group (p < 0.05). Feedback indicate that participants found the system user-friendly, believed it could improve their performance, and valued the tailored feedback, particularly guidance on focus areas.

Insights from the Perspective of Examiners on the Justification of Grades in Higher Education in Norway

Hanne Sørum — 2024-11-24

Students in higher education in Norway have the right to ask for justifica-tion for and/or complain about the grades they receive. Drawing on an online survey of examiners (n = 54) at a Norwegian university college, we report in terms of numbers and open-ended comments about justifications given to students. Guided by Winston and Boud (2020), we contribute insights that fill a gap in the literature. Our results show that the examiners experienced many requests for justification, which they perceived as time-consuming. They suspected that the information system for exams and grades (WISEflow, in our case) made it easy for students to request justifications. At the same time, they were positive that feedback should be given to stu-dents. We conclude that there is a need for more research on the usefulness of the justification of a given grade in higher education. Our findings reveal that the examiners requested better guidelines for the purpose and content of justifications (descriptive feedback only or advice on how to improve grades) and how to motivate students to be receptive and learn from justifications. Thus, there is great potential for future studies pertaining to this topic.

The Nordic Prior Knowledge Test in Programming: Motivation, Development and Preliminary Results

Sondre Bolland — 2024-11-24

With recent updates to Norway's national curriculum, computational thinking and programming has become a core part of the K-12 education, leading to an influx of students entering higher education with prior programming experience. This shift has the potential to impact the teaching of ICT at universities, as foundational knowledge could allow for the introduction of more advanced topics earlier. However, the quality of programming education varies, making it essential to assess students' prior knowledge.

To address this need, the Nordic Prior Knowledge Test in Programming was designed to assess incoming students' programming proficiency in the basic elements of the introductory programming course (CS1). This study details the rationale for assessing incoming students, the development and content of the test, and the preliminary results from its 2024 administration.

The test was completed by 3,038 students (2,661 after data pruning) across eight higher education institutions in Norway. Results indicate a mean score of 39.9%, with a significantly higher performance among students exposed to the new curricular model (50%). Despite these gains, a substantial proportion of students scored at the lower end of the scale, highlighting the ongoing need for foundational programming instruction.

Although most students will benefit from completing the standard CS1 course, a notable subset of students achieved high scores (14.7% scoring above 90%), suggesting the potential value of accelerated or alternative learning pathways, such as an advanced CS1 course or direct progression to CS2.

Fundamentals of Norwegian CS1

Sondre Bolland — 2024-11-24

The introductory programming course, known as CS1, has evolved considerably since its inception, with diverse opinions on the essential concepts that should be included. This study aims to identify the fundamental concepts taught in Norwegian CS1 courses in order to develop a validated assessment tool: a concept inventory. This tool will be utilized in the Nordic Prior Knowledge Test in Programming, which is designed to assess the pre-existing programming knowledge of students entering higher education. This test uses Python, the dominant programming language in K-12 and higher education in Norway.

To identify the fundamentals of CS1 we employed a triangulation approach that included three perspectives: the intended curriculum, the assessed curriculum, and the experienced curriculum. Our methodology involved a Delphi process with Norwegian CS1 educators, an analysis of final exams from various Norwegian institutions, and surveys of computer science students regarding the difficulty and importance of programming concepts.

Our findings reveal that concepts related to looping, functions, conditionals and error interpreting are central to Norwegian CS1 courses, aligning with existing literature. However, we also identified notable discrepancies compared to older CS1 concept studies developed in other countries, particularly in concepts like recursion, data structures beyond arrays/lists and maps, and test design. These results underscore both the dynamic nature of computer science education and the enduring importance of foundational topics that students are expected to master.

Shaping a Modern Programming Paradigms Course for Advanced University Students

Leonardo Montecchi — 2024-11-24

Programming is one of the core disciplines in Computer Science (CS) and Computer Engineering (CE) courses, and it is increasingly permeating the curricula of other study programs. After undergoing an introduction to programming and a course on object-oriented programming, some students will attend an advanced course on programming languages, where features of different programming paradigms are discussed, with an emphasis on the semantics of execution. Due to the emphasis on theory aspects, such courses often employ old or experimental languages that have little practical application, resulting in low engagement of students. Unfortunately, most of the research work focused on introductory programming, which has a more established syllabus and larger possibilities for interventions. In this paper we analyze the current status of the programming languages course at our university, and we investigate possibilities for renewing the syllabus with modern languages and tools. We first review the current topics addressed by the course, and then we discuss possible content changes with the aim to shape a more engaging course. The paper ends with a plan for implementing and evaluating the new version of the course starting from the next academic year.

Students' Visualisations of Programming Concepts: An Exploratory Analysis

Rolando Gonzalez — 2024-11-24

This study explores the intuitive visualisations of code created by novice programming students with the aim of uncovering how these reflect their understanding of fundamental programming concepts. Through a thematic analysis of student drawings, this exploratory research identifies common themes and categorises different modes of visual expression. The findings reveal great variations in how students can visualise code, and suggest that drawings can serve as an effective tool for both diagnosing student misconceptions and supporting their learning of programming. This study highlights the potential of incorporating visual exercises into programming education to improve concept comprehension.