Research Article
Edmund De Leon Evangelista
CONT ED TECHNOLOGY, Volume 17, Issue 1, Article No: ep559
ABSTRACT
The rapid advancement of artificial intelligence (AI) technologies, particularly OpenAI’s ChatGPT, has significantly impacted higher education institutions (HEIs), offering opportunities and challenges. While these tools enhance personalized learning and content generation, they threaten academic integrity, especially in assessment environments. This study systematically examines the impact of ChatGPT on academic integrity in HEIs, focusing on exam design, assessment strategies, AI detection tools, and policy frameworks. This research draws from current literature and expert recommendations to identify practical approaches for developing assessments that foster critical thinking and deep cognitive engagement, making them less susceptible to AI-generated content. Key areas explored include the creation of complex, analytical exam formats, deploying advanced AI detection software to counter AI-assisted cheating, and formulating institutional policies that promote the ethical use of AI. This comprehensive framework aims to equip educators and administrators with practical strategies to preserve academic standards while harnessing the potential benefits of AI, ensuring the continued validity of assessments in the AI-driven educational landscape.
Keywords: ChatGPT, academic integrity, higher education, exam design, AI detection, assessment strategies, AI policies
Review Article
Johan Syahbrudin, Edi Istiyono, Moh. Khairudin, Anita Anggraini, Indah Urwatin Wusqo, Metta Mariam, Yenni Muflihan
CONT ED TECHNOLOGY, Volume 17, Issue 1, Article No: ep554
ABSTRACT
Computer-based assessment (CBA) is a top-rated tool for conducting assessments, mapping learning outcomes, and selecting new candidates. Research that examines the development and use of CBA is also increasing from year to year, so without bibliometric analysis, it would be quite challenging to keep up with all of these studies. This study aims to clarify the status and world trends in CBA research around the world. Bibliometric analysis was performed using bibliometrix based on 2,020 articles published in Scopus from 1982 to 2023 (data update time on 5 January 2024). This bibliometric analysis identified the state of research publications, scientific productivity, the network state of research publications, and CBA research priorities and directions going forward. Results show that research on CBA has grown rapidly, with the highest number of publications in 2022, with the USA being the most productive country. Greiff is the most prolific and Wise is the most impactful author. The journal with the highest impact is “Computers and Education”. These findings also indicate that research themes that show research potential due to their high relevance to CBA but are still little studied are those related to “regression analysis”, “automation”, “information technology”, “assessment”, “motivation”, “quality control”, “problem solving”, “online systems”, “decision making”, and “teaching”.
Keywords: bibliometric, bibliometrix, computer-based assessment, assessment
Research Article
Carmen Köhler, Johannes Hartig
CONT ED TECHNOLOGY, Volume 16, Issue 4, Article No: ep528
ABSTRACT
Since ChatGPT-3.5 has been available to the public, the potentials and challenges regarding chatbot usage in education have been widely discussed. However, little evidence exists whether and for which purposes students even apply generative AI tools. The first main purpose of the present study was to develop and test scales that assess students’ (1) knowledge about ChatGPT, (2) actual ChatGPT usage and perceived value of use, and (3) attitude towards ChatGPT. Our second aim was to examine the intercorrelations between these scales, and to investigate differences (a) across five academic fields (i.e., human sciences, social sciences, teaching profession, health sciences, and law and economics) and (b) between stages of education (i.e., number of semesters). N = 693 students from various German universities participated in our online survey. Quality checks (Cronbach’s alpha, MacDonald’s omega, and confirmatory factor analyses) show satisfactory results for all scales. The scales all positively relate to each other, except for the knowledge and attitude scales. This means that more knowledge about ChatGPT is connected to a less favorable attitude regarding the generative AI tool. Lastly, MANOVA and subsequent Bonferroni corrected ANOVA tests show that ChatGPT is mostly used by law and economics students, and most frequently by students in the third year of higher education.
Keywords: ChatGPT in higher education, student knowledge, student use, student attitude, scale development, assessment, large language models (LLMs)
Research Article
Lu Ding, Sohee Kim, R. Allan Allday
CONT ED TECHNOLOGY, Volume 16, Issue 3, Article No: ep512
ABSTRACT
With the exponential development and vast interest in artificial intelligence (AI), the global economic impact of AI is expected to reach $15.7 trillion by 2030. While AI has infiltrated everyday life, a lack of knowledge of what AI is and how AI works is ubiquitous across all ages and professions. Teaching AI literacy to non-technical individuals has become imperative and requires immediate attention, however, assessing AI literacy has heavily relied on subjective measurements such as qualitative assessment and self-reported surveys, which may lead to biased results. This study contributes to the field by developing and validating an assessment created based on a well-established AI literacy framework. A total of 196 responses were collected from pre-and in-service teachers in the United States, and 186 responses were included in the analysis to validate the assessment. The final assessment comprises 25 objective-based items reduced from an originally 31-item assessment. Both experts’ insights were sought, and statistical methodology was employed to ensure the validity of the assessment. The results indicate that pre-and in-service teachers have a moderate level of AI literacy and in-service teachers performed slightly better than pre-service teachers on our assessment. Inconsistent answers across various AI concepts indicate that teachers may possess an even more ambiguous understanding of certain AI concepts.
Keywords: AI literacy, assessment, non-technical, teachers, objective-based
Research Article
Oraboot Wuttikamonchai, Paitoon Pimdee, Jaruwan Ployduangrat, Aukkapong Sukkamart
CONT ED TECHNOLOGY, Volume 16, Issue 1, Article No: ep494
ABSTRACT
In this study, we present a comprehensive needs assessment (NA) aimed at identifying key factors that contribute to enhancing instructor skills in mobile website design (MWDS). In the realm of professional development, the significance of NAs for user experience (UX) designers has gained attention, with scholars emphasizing the importance of competency modeling to elucidate the intricate knowledge, skills, and abilities inherent in UX design. This perspective aligns with the essential nature of learning NAs for continuous professional development. Therefore, the objective of our investigation was to evaluate the current state of instructor MWDS and their real-world user experience through a sample of 41 individuals, including information technology (IT) instructors and IT entrepreneurs. Employing a simple random sampling method in August 2023, our study utilized descriptive statistics to analyze the data, revealing a moderate proficiency level (mean [M]=3.40). Interestingly, the highest rankings in our findings were attributed to planning mobile website design from a user experience perspective (M=3.81) and developing or creating a website for mobile devices user experience (M=3.81). These skills demonstrated consistent high perceived importance (M=4.78). Moreover, specific aspects of MWDS stood out in terms of significance, with developing or creating mobile websites’ user experience holding the top priority (M=4.87), followed closely by designing mobile websites’ user experience (M=4.79) and developing or prototyping mobile websites’ user experience (M=4.77). Our findings underscore the urgent need to prioritize the promotion of MWDS, particularly those focused on user experience. Using priority needs index modified (PNImodified) method, we identified that user research holds the highest priority (PNImodified=0.56), followed closely by mobile website design user experience (PNImodified=0.41), and defining website content for mobile devices user experience (PNImodified=0.40). This information provides valuable insights for educational institutions and industry stakeholders, guiding efforts to enhance instructor skills and align curricula with the evolving demands of mobile web design. In summary, our study sheds light on the current state of instructor MWDS, emphasizing the importance of user experience-related competencies. The identified priority needs offer valuable guidance for educational institutions and industry stakeholders, facilitating efforts to enhance instructor skills and align curricula with the dynamic requirements of mobile web design. Notably, our results align with the hypothesis testing, where paired sample t-test statistics found that both overall and individual aspects indicated a statistically significant preference for desired over actual outcomes at the .01 level.
Keywords: mobile web design, needs assessment, priority needs index, Thailand, UX designer
Research Article
Katherina Gallardo, Leonardo Glasserman, Nohemi Rivera, Lizette Martínez-Cardiel
CONT ED TECHNOLOGY, Volume 15, Issue 2, Article No: ep415
ABSTRACT
A mixed-method study about the perception of students and faculty around learning assessment practices was conducted in a multicampus competency-based approached Mexican private university. The objective was getting new knowledge about higher education community perception around learning assessment practices, learning outcomes production and how authentic assessment was driven while working on different modalities during the pandemic: remote learning, synchronous hybrid learning, alternate hybrid learning, and on-site modality. A questionnaire, and an interview for faculty and focus groups for students were designed, validated, and conducted. A total of 281 faculty and 908 students participated in this study. The main quantitative results indicate that students who took on-site classes evaluate the variables better than those who took hybrid classes. In addition, students who took remote classes evaluate the authenticity variable better than those who took it in alternate hybrid modality. In the case of the faculty, only the equality variable showed significant differences between the reported modalities. Besides, the main qualitative results indicate that perceptions are quite different between faculty and students while getting their perceptions from an evaluation standards scope. Thus, their perspectives around equality, authenticity, feasibility, reliability and ethics were obtained and discussed against literature. The main conclusions of the study remarked that students and faculty were positive towards most of the execution of learning assessment and learning outcomes practices during the pandemic. Nevertheless, students’ preference of on-site modality was determined as the way they have more opportunities for interacting and learning more from assessment and feedback.
Keywords: innovation in education, competency-based education, higher education, learning outcomes, performance assessment
Research Article
Thuy Thi Tang, Thuy Nga Nguyen, Huong Thi Thu Tran
CONT ED TECHNOLOGY, Volume 14, Issue 3, Article No: ep375
ABSTRACT
The purpose of this study is to examine factors that influence teachers’ intentions to use technology in assessments using the technology acceptance model (TAM) as a framework. An online survey was utilized to collect data, and 360 teachers participated in the survey. This study used partial least squares-structural equation modelling (PLS-SEM) to test the hypotheses to verify the effects of variables on teachers’ intention of e-assessment use. The model consists of four constructs: computer self-efficacy (CE), perceived ease of use (PEOU), perceived usefulness (PU), and frequent use of e-assessment tools (FoUAT). The findings revealed a significant influence path from CE to PEOU, FoUAT, and behavior intention. In addition, PEOU is a critical factor that positively impacts both PU and teachers’ behavior intentions. In contrast to our expectation, frequency of use was statistically insignificant and had no impact on teachers’ intention to use (IU) e-assessment tools. The total of these four variables corresponded to 71.4% of the variance of user intention. These results confirm that TAM is an effective model to explain teachers’ technology acceptance to use e-assessment tools for their teaching.
Keywords: computer self-efficacy, e-assessment tools, PLS-SEM, technology acceptance model
Research Article
Christye Majuddin, Mas Nida Md. Khambari, Su Luan Wong, Norliza Ghazali, Noris Mohd. Norowi
CONT ED TECHNOLOGY, Volume 14, Issue 2, Article No: ep358
ABSTRACT
While educators worldwide are moving towards the universal design for learning, it is also essential to ensure learners are suitably assessed. Assessments and learning profoundly reciprocate one another as assessment may inform the learning practices, and vice versa. Resonating the aforesaid view, PutraPacer was developed as a customizable tool to empower instructors in embracing differentiated assessment. The objective of this pilot study is to elicit feedback on the use of PutraPacer as a differentiated assessment tool among a group of undergraduate students at a higher education institution. Drawing on the UTAUT model, this study employs an explanatory sequential mixed-method design to gather both quantitative and qualitative data. The quantitative findings show that the mean values for performance expectancy, effort expectancy, social influences, and behavioural intention to use PutraPacer are ranged between 3.56 and 3.67. Based on the Pearson’s correlation coefficient, there are strong association between performance expectancy, effort expectancy, social influences with behavioural intention to use PutraPacer as a differentiated assessment tool. The qualitative findings reveal that the students perceived PutraPacer as a user-friendly and a learning tool that promotes individualized learning experience and supports students with different abilities, and iii) a good platform for practices, quizzes, and revision.
Keywords: alternative assessment, differentiated assessment, learners’ diversity, explanatory sequential mixed method
Research Article
Mirian Agus, Maria Lidia Mascia, Maria Assunta Zanetti, Simona Perrone, Dolores Rollo, Maria Pietronilla Penna
CONT ED TECHNOLOGY, Volume 13, Issue 3, Article No: ep310
ABSTRACT
In the literature there are more and more works dealing with cyberbullying and adolescents’ perception of this phenomenon. In particular, we are interested in cyberbullying and the effects of cyberbullying on adolescent life. What is more, in 2020 the impact of COVID-19 has increased the digital presence of cyberbullying behaviours. This phenomenon is extremely complex, since it is multicomponential and multifactorial, and many components act on it. Starting with an analysis of the literature, this work offers preliminary data towards the validation of a self-reporting questionnaire; it was administered to 650 adolescents, to evaluate their perceptions of the victims of cyberbullying. The questionnaire consisted of 33 items; Principal Component Analysis was applied, which identified the dimensions resuming the items’ variability. The findings confirmed the multidimensional nature of the issue covered by the questionnaire and highlighted three psychological dimensions: Internalisation, Counterbalance and Marginalisation. The results support the structure of the questionnaire, useful to quickly collect information about adolescents’ perception of cybervictimisation. The assessment of this information might help teachers, educators, and tutors to formulate targeted interventions to combat the spread of cyberbullying aimed for example at improving emotional intelligence, starting from the construction of emotional contagion.
Keywords: cyberbullying, adolescent representations, cybervictims, assessment
Research Article
Jale Balaban-Sali
CONT ED TECHNOLOGY, Volume 3, Issue 4, pp. 265-277
ABSTRACT
This study examined new media literacy skills of university students based on Jenkins and his colleagues’ classification. Toward this purpose, an online Likert scale was administered to a sample (n=170). This scale included a multi-component understanding of media literacy such as tackling the consumption of media messages and the original creation of multimedia material. The Cronbach’s Alpha reliability coefficient of the scale was 0,93. The instrument was structured around three main sections; demographics, media use characteristics, and new media literacies (NMLs). The third section aimed to assess participants’ new media literacy skills by presenting them 60 items about their social and cultural modes of engagement, online interaction, and media consumption and creation patterns. The statements were conceptually built around the 12 NMLs skills identified by Jenkins and his colleagues. These skills are: Play, appropriation, distributed cognition, collective intelligence, judgment, transmedia navigation, networking, negotiation, and visualization. The results of the study showed that individuals who spent more time on Internet, social media, and blogging had the highest NMLs levels. Furthermore, young participants’ NMLs levels were higher than those over the age of 32.
Keywords: New media literacies, Social media skills, New media literacy assessment, Digital literacies, Measuring new literacies