Technology Integration Through Evidence-Based Multimodal Reflective Professional Training

For technology integration to achieve intended instructional objectives, professional development is of paramount importance for in-service instructors. Technology enhanced reflection providing multimodality and evidence-based data has the potential of helping instructors achieve technology integration in their teaching. Thus, this study addressed the lack of professional development programs for in-service English language instructors aiming at technology integration through evidence-based multimodal reflective practices. By following the developmental research methodology, an instrumental case study was conducted using mixed-methods design including both qualitative and quantitative data. The professional training was carried out with eight English language instructors at tertiary level who aimed to integrate technology into their English classes by being involved in evidence-based multimodal reflective practice. In this study, there were five different data sources (instructors’ opinions, recorded lessons, video annotations, face-to-face interaction notes and pages in the learning management system) and seven data collection tools (interviews, tutor reflection template, self-reflection template, peer-reflection template, technology integration questionnaire, online discussion forums and evaluation criteria form). As a result, this professional training particularly designed and implemented for the in-service English language instructors at tertiary level did make a change in their both technology integration and evidence-based multimodal reflective practice. Apart from the change in the instructors’ technology integration, this study had an impact on their understanding, practice and quality of reflective practice involving evidence-based multimodal reflection tools as it helped the instructors to increase depth in their evidence-based multimodal reflective practice.


INTRODUCTION
Technology integration, one of the key issues in today's teaching and learning settings, can be simply defined as making technology an available, accessible, and integral constituent of instruction so as to foster the learning and teaching process (Wheeler et al., 2000). It has been the research topic of many studies, presenting challenges regarding its true nature, its stakeholders, the ways to achieve it, and issues regarding evaluation of the outcomes. Technology integration has become more significant in higher education (Sahin & Thompson, 2006); however, there are fewer studies on technology integration at tertiary level (Instefjord & Munthe, 2017). Most studies focus on pre-service teachers (Liu & Kleinsasser, 2015;Park & Son, 2020;Turgut, 2017), rather than in-service instructors. Particularly in English language learning and teaching, there 2 / 22 is lack of focus on teachers' use of technology (Gönen, 2019). There needs to be a greater focus on the significance and power of technology integration (Baker et al., 2012) and on building a theoretical base (Jung et al., 2019).
Technology integration is a highly intricate process that involves numerous factors and variables (Kimmons & Hall, 2016). Since instructors are viewed as having a key role when integrating technology (Ruggiero & Mong, 2015), many training programs of various lengths and scope have been designed (Alemdag et al., 2020). Most place the emphasis on how to use a particular technology, turning the innovation into an addins, rather than integrating technology with content and pedagogy (Bebell & O'Dwyer, 2010). Often instructors are unable to integrate technologies that they have learned as a part of their professional development into their classes (Kwangsawad, 2017).
Being aware of the real benefits of instructional technologies, many colleges and universities allocate a generous budget for the newest technologies (Li, 2014). Even though it is essential eliminating first-order barriers (Ertmer, 1999), technocentric (Papert, 1988) attempts do not guarantee the effective and widespread use of technology (Hixon & Buckenmeyer, 2009). It is illogical to regard instructional technologies separate from teaching and learning (Okoje et al., 2016); however, the main hurdle for in-service teachers to integrate technologies into their teaching is inadequate or inappropriate professional trainings (Healey et al., 2011). Workshops focusing on the technical capabilities and functions of the new technologies form the backbone of many teacher-training programs. However, it is argued that such trainings do not help to achieve intended objectives due to the lack of focus on integrating technology into teaching (Koehler et al., 2007). In particular, there is an absence of best practices or sample lessons demonstrating how to achieve technology integration (U.S. Congress, Office of Technology Assessment, 1995). Furthermore, there is no way to check if the participants will change their teaching after the training (Glazer et al., 2005). Thus, Dockstader (1999), Keengwe et al. (2009) argue that high quality teacher training is a necessity for successful technology integration. Studies point out that such professional programs are effective when they: • are based on the identified instructional needs and objectives (Keengwe et al., 2009), • are prepared according to the participants' characteristics (Arnold & Ducate, 2015), • promote collaboration (Darling-Hammond & Richardson, 2009), • are reflective (Harris, 2016), • take place in authentic contexts (McNeil, 2013), • are given with timely feedback and guidance (Glazer et al., 2005).

Reflection in Professional Development
Owing to the vague mechanism of reflection, there are a wide variety of perceptions (Clara, 2015). For example, Mathew et al. (2017) describe it as thinking about one's practices. Farrell (2012) uses the metaphor of a compass; teachers pause, analyze and decide the current position and desired destination. Leijen et al. (2012) describe four levels of reflection considering their quality: description, justification, critique and discussion. Descriptive information, as the superficial level, provides statements depicting what happened. The justification stage explains the rationale behind the actions and beliefs. The critique stage involves both explanation and evaluation. The discussion stage, as the highest level, involves all the previous stages, and presents alternative ideas and solutions to make changes in practice.
Teaching has such an intricate and complex nature that it calls for teachers to analyze their own professional beliefs and practices (Mathew et al., 2017). This has made reflection a frequently used method in many teacher-development programs (Ottesen, 2007), especially for second language teachers (Avalos, 2011). The related studies clearly show that there are such various methods of reflection as narrative inquiry (Johnson & Golombek, 2002), critical incident analysis , coaching (Mathew et al., 2017), and case analysis (Olaya Mesa, 2018).

/ 22
Surprisingly, instructors are rarely encouraged to reflect on how successful they are in integrating technology, even though "reflection is central to all learning" (Bruner, 1960, as cited in Ray & Coulter, 2008. However recently, a small number of researchers underscored the significance and function of reflective practice in achieving technology integration (Avcı et al., 2020;Burger, 2019).

Evidence-based Multimodal Reflective Practice
Technology fosters teachers' reflective practices (Burhan-Horasanlı & Ortaçtepe, 2016). Traditionally, reflection is carried out in written format rather than exploiting the advantages of multimedia instruments. However, it is essential that reflective practice become more data-led, and less dependent on written forms . Instructors need to be presented with evidence-based and concrete data to improve their teaching skills via reflection . This can be done via multimodal feedback and reflection involving audio and visual media (Campbell & Feldman, 2017).
Technology-enhanced reflection provides instructors with an online platform (i.e., blog, wiki, LMS, etc.) to establish a dialogic interaction (van Braak et al., 2018). This allows them to receive various perspectives from multiple observers (Sherin & van Es, 2005). This feedback accompanies direct evidence from the video clips created from their recorded lesson (Schmid, 2011). Such extracts enable them to re-live certain teaching episodes (O'Leary, 2017) and for observers to make relevant comments or recommendations (Howard & Myers, 2010). The evidence-based multimodal reflection activates both the visual and auditory channels. Making a link between verbal and visual information allows a deeper understanding compared a single channel (Sweller et al., 1998). To this effect, particular technology enhanced tools which could systematically provide multimodal descriptions based on the evidence-based data have been used in reflective practices such as videos, video annotations and e-portfolios.
Videos have been used in teacher training since the 1960s (Sherin & van Es, 2005) but there has been an increasing recent interest due to digital technology (Tripp & Rich, 2012). Teachers can record lessons using their phones and store or share it via the cloud. Many scholars agree that recordings of lessons provide multimodal, qualitative, rich and thick descriptions (Schmid, 2011) for reflections. Video stimulated recall could be considered as cognitivist, while video-stimulated peer reflection is socio-constructivist (van Braak, et al, 2018) because it is a form of collaborative inquiry (Powell, 2005), bringing a deeper understanding of their pedagogical beliefs, values and practices. It brings the understanding that other instructors have similar challenges and allows them to compare their teaching to their colleagues' (Arya et al., 2015).
Video annotation refers to the software which enables taking notes on any particular moment of a video (Perez-Torregrosa et al., 2017). This allows asynchronous computer-mediated communication (Howard & Myers, 2010). Many studies have demonstrated that video annotations enrich reflection, allowing teachers to make connections between different parts of a lesson, and give responses or make comments on specific sections (Rich & Hannafin, 2009). Some researchers emphasize the ease of use, usefulness and appeal of video annotations in teacher training programs (Picci et al., 2012). E-portfolios or digital portfolios are digital repositories of verbal and visual data including text, image, sound and video documents. They aim to encourage reflection in order to demonstrate to the instructors and others the progress made (Strudler & Wetzel, 2008). Bartell et al. (1998) also state that e-portfolios cultivate selfregulated growth, gain effective teaching skills and promote collaborative work.
This study aims to find answers to the following questions: to what extent does the 'Technology Integration Through Evidence-based Multimodal Reflective Professional Training' change English language instructors' technology integration into their classes? (RQ1) and how does it change their evidence-based multimodal reflective practice? (RQ2). 'Technology Integration Through Evidence-based Multimodal Reflective Professional Training' which lasted 12 weeks was a professional training designed to help English language instructors both broaden their repository of instructional technologies and also learn how to implement evidence-based multimodal reflection tools. In this research, technology integration was not restricted to technology (Papert, 1988); the conceptual framework was based on technological, pedagogical and content knowledge (TPACK) as seen in many other studies (Hsu, 2016;Jaipal & Figg, 2010;Kurt et al., 2014;Lehiste, 4 / 22 2015). Evidence-based reflection was data-led reflection composed of various sources, includes valuable data and was based on concrete information . Multimodality refers to the use of audio and visual media (Campbell & Feldman, 2017). Evidence-based multimodal reflective practice refers to making use of digital and multimedia instruments to assist instructors to reflect using specific data from their classes both in visual and auditory modes.

METHODOLOGY
This study consisted of two phases: (1) the design and development of the 'Technology Integration Through Evidence-based Multimodal Reflective Professional Training' and (2) the implementation and evaluation. This is a qualitative case study aided with quantitative data in order to find the answers to the two research questions.

Phase I
This study followed a developmental research methodology in this phase, which aims to "create knowledge from instructional design and development practice" (Richey & Klein, 2005, p. 24). Rather than suggesting a new instructional design model, the MRK Instructional Design Model (Morrison et al., 2011) was used as a framework. The designer conducted a 'needs assessment', which refers to assessing the gap between the target and present situation to determine the required type of instruction (Dick et al., 2014). The needs assessment was carried out via classroom observations and interviews in this study. Participants were asked to complete a 'Participant Information Form', consisting of two sections: general characteristics and specific entry competencies. The general characteristics involved information such as the instructors' age, nationality and professional experience. The specific entry competencies aimed to check whether the instructors could use the institution's LMS and lecture capture system. Although there was no prerequisite cognitive level for the participants, the potential instructors were expected to circle 'yes' in the 'Entry Competencies Section' to fulfill the requirements of the professional training. They also needed to be familiar with the institution's learning management system (Blackboard) and lecture capture system (Panopto). The participants in the two phases of the study were the same, so the information for each instructor was given in the second phase.
The task for the participants was as follows: how to create an interactive learning environment via technology, manage the class, foster collaboration among students and their own reflect on technologyintegrated ELT classes. As shown in Table 1, the ten objectives for this study were divided into two categories as 'evidence-based multimodal reflection objectives' and 'objectives of technology integration in an ELT class'. The sequence of the content of this study was identified according to the suggestions made by Posner and Strike (1976, as cited in Morrison et al., 2011). These are learning-related, world-related and conceptrelated sequencing. The participants were provided with various activities throughout the professional training. Each of the following activities below is explained, with the underlying reasons and links to the research questions, in the data collection section, rather than in this section, in order to avoid repetition. h. interviews with the researcher both at the beginning and at the end of the professional training An associate professor with more than twenty years of experience in the field of instructional design and English language teaching evaluated the design of this professional training and gave detailed feedback. This expert stated that the objectives were clear, since a detailed needs assessment had been conducted, via observations and interviews with the in-house trainers who were familiar with the instructors. The expert was also content with the data regarding the participants' general characteristics, entry competencies and previous experiences. As for the learning and performance contexts, she noted that the descriptions were quite clear. The expert also underscored the benefit of co-operation between the training designer and a senior member in the TDU in determining the content of the instruction. She stated that the learning objectives were satisfactorily based on the task analysis, but she recommended moving certain objectives between categories, so "… participate in online discussions to be engaged in collaborative feedback environments" was moved to the reflection category, and "… integrate Blackboard into their teaching to promote collaborative learning", to the technology integration category.
'Technology Integration Through Evidence-based Multimodal Reflective Professional Training' was planned as twelve weeks. Figure 1 displays the flow of the training with assessment and evaluation components. This also presents the output of Phase I in the research. Phase II

Sampling and participants
The researcher worked with a convenience sample of eight instructors in this study. The information of the instructors is shown in Table 2. The research was carried out at the school of foreign languages in the preparatory program of an Englishmedium foundation university in the west of Turkey. The English preparatory program of the university follows a modular system. There are four modules in an academic year, each of which lasts eight weeks. The students move to the following level when successful in each module. Classes size varies from 17 to 20 students. The students attend six 45-minute lessons a day. Breaks between lessons last 15 minutes. The classrooms are equipped with a touchpad desktop computer, two cameras (one for the instructor and one for students), a microphone, an interactive whiteboard, and a smart projector. Blackboard is the online learning-management system used. LMS used. The university also uses a lecture capture system which enables instructors to record their lessons and share it with their learners or colleagues. They can also make video annotations on their recorded lessons. Table 3 displays the data sources, data collection instruments and data analysis aligned with the research questions. The technology integration questionnaire and the evaluation criteria form were the quantitative data collection tools. The qualitative data collection tools were interviews, discussion forums (online), and peer, tutor and self-reflection templates.

Data collection
Technology integration questionnaire. The instructors filled in this Likert scale questionnaire at the beginning and at the end of the research in order to self-reflect on their acceptance and integration of technology. They indicated a score from 1 (the lowest) to 5 (the highest) to such statements as "Using the ICT in my classes would enable me to accomplish tasks more quickly". The data allowed identification of any changes in their technology integration (RQ 1) as an impact of this training. The questionnaire was originally developed for use with university academic staff at a higher education institution by Oye et al. (2014), highlighting the alignment with the setting and participants of this research. The reliability analysis was originally conducted for the 34 items using Cronbach's Alpha, and showed a good degree of reliability of .704 (Oye et al., 2014). The Cronbach's Alpha in this study showed high internal consistency (α > 0.8).

/ 22
Evaluation criteria form. The researcher and the external trainer evaluated the participant instructors' technology integration and evidence-based multimodal reflective practice by using this form. They evaluated the instructors' lessons with video annotations, and their online reflections with a score between 1 and 5, according to the specially prepared rubrics. There are two main sections in this form: 'Section A: Technology Integration' involving the subsections of technological, pedagogical and content, and 'Section B: Technology Enhanced Reflection on a Technology Integrated Lesson'. Two sample statements are as follows: "The technological tool(s) diversified the interaction patterns" and "In the self-reflection template, the instructor proved the achievement of the language learning objectives". When generating the rubrics, the researcher consulted two academicians at the institution and a member of the preparatory program testing unit.
Interviews. The researcher held individual semi-structured interviews at the very beginning and at the end of the study to understand participants' perception, knowledge and practice of 'evidence-based multimodal reflection' (RQ 2) and "technology integration" (RQ 1). The first four questions are related to reflection and evidence-based multimodal reflective practice (RQ 2). To illustrate, question 3 is "Have you ever done reflection in digital platforms? If yes, can you tell me about this experience? If not, how do you think you can do reflection in these platforms?". Questions 5 to 8 aim to reveal the instructors' understanding and practice of instructional technologies integrated into their teaching (RQ 1). For instance, question 5 is "Do you use any technological tools or software in your ELT classes? If yes, can you please tell me which ones? How often?" In the process of preparing this interview, the researcher worked with two experts, one in instructional technologies and the other in English language teaching. This helped to create an interview that blended evidence-based multimodal reflection with instructional technologies in English language teaching. Discussion forums (online). The participants joined the online discussion forums in the weeks five, seven and eight during the professional training to share their perceptions on 'technology integration into their ELT classes' and 'evidence-based multimodal reflective practice'.
Self-reflection template. The instructors completed the 'Self-reflection Template' in order to display their self-reflection on their technology integration (RQ 1) and how effectively they used evidence-based multimodal reflection tools (RQ 2). They wrote their reflections using the template for their second, third and fourth lessons and uploaded these to their Blackboard page. The first part of this template encouraged instructors to observe their recorded lessons in order to deliberate on instructional objectives, selection of the technology, rationale behind this selection, the strategies used in employing that technology, and assessing the achievement of learning objectives and technology integration. Two sample questions are as follows: "Which instructional technology was required?" and "How satisfactory was the technology integration?". For this part in this template, the conceptual framework derived from Wang and Woo's model (2007).
Tutor reflection template. The tutor reflection template was the same as the self-reflection template but was completed by researcher. The aim was to set a model for the instructors and give them feedback on their technology integration (RQ 1) and how effectively they used evidence-based multimodal reflection tools (RQ 2). The researcher uploaded the template for each recorded lesson of each instructor to the instructors' Blackboard pages after they had uploaded their own self-reflection templates. The tutor reflection templates were used to triangulate the data gathered from other sources.
Peer reflection template. The instructors used the 'Peer Reflection Template' in order to give feedback to their peers on their technology integration (RQ 1) and use of evidence-based multimodal reflection tools (RQ 2). The instructors shared their feedback on their Blackboard pages. The template consists of eight openended questions. Questions 1 to 4 encouraged a critical lens toward their peer's technology integration (Research Question 1). Questions 5 to 8 encouraged evaluation of their peer's evidence-based multimodal self-reflection (RQ 2). To illustrate, Question 7 is "Do you think the teacher benefitted from the evidencebased multimedia self-reflection?". When completing the peer reflection template, the instructors both observed their colleagues' recorded lessons with annotations and analyzed the colleagues' feedback on their Blackboard pages. The self-reflection template was the basis for the flow of the questions in peer reflection template. The peer reflection template enriched the feedback available to the instructor and created a nonthreatening socio-constructivist learning environment.
The content of the training sessions is given in Table 4. The technologies introduced in the 'Technology Integration into an ELT Class' sessions set weekly targets in terms of the target technologies that the instructors aimed to use in their lessons.

Data analysis
The quantitative data was gathered through the 'evaluation criteria form' and the 'technology integration questionnaire'. Two raters used the criteria form when evaluating the instructors' LMS pages, recorded lessons and video annotations. To evaluate the interrater reliability, Spearman's rho correlation testing was applied; and a strong correlation between the two raters was observed, as the coefficient value was 0.874 (p < .01). In the analysis of the data from the 'evaluation criteria form', using Friedman test, the test statistic value (chi square) and the significance level (Asymp. Sig.) were reported to determine whether there was a statistically significant difference between the mean ranks of the related groups (p <0 .05). In order to notice where the differences occurred, a post hoc test was also run using SPSS. Since making multiple comparisons increased the risk of Type I error on the post hoc test, the Bonferroni adjustment was also administered to set the adjusted p values, as shown in pairwise comparisons.
The Wilcoxon signed-rank test was used to analyze the data from the technology integration questionnaire. As the Wilxocon signed-rank test is a non-parametric test, in the descriptive statistics, the quartiles information was given to describe the groups. Asymp. Sig. (2-tailed) value was considered whether this study had a meaningful impact on the instructors' readiness to accept and integrate technologies (p < 0.05). The Zstatistic was also used when reporting the Wilcoxon signed-rank test. Since the questionnaire consisted of five constructs in addition to the whole test result, each construct was analyzed separately to determine which constructs had a statistically significant impact on the instructors' technology acceptance and integration.
For the qualitative data, content analysis was employed, using open, axial and selective coding. The researcher conducted the qualitative data collection and analysis concurrently during the course of the study (Morrison et al., 2002). He unveiled the themes that emerged from the qualitative data, as well as triangulating findings by referring to the relevant quantitative data. Member checks were conducted throughout the research. For peer debriefing, the data and researcher's interpretations were discussed with a professor in ELT. All these procedures increased the credibility of the findings.
Like other studies, this research has limitations. Firstly, the validity depends on the reliability of the instruments used. Secondly, generalizability of the results is restricted to the honesty of the instructors' answers to the questions and reflection templates. In addition, all the instructors were in the same institution, which means that results may not be generalizable. The class size, learners' age, the syllabus, the books and the available technologies are context specific. Finally, there was no control group in this research; this could be a focus of future studies in order to compare the results.

Qualitative Findings
This study presented comprehensive and richly descriptive findings. The emerging themes which were aligned with their sources and the two research questions of this research are given in Table 5.  Questioning their understanding and practice of instructional technology integration into ELT. The qualitative data show an increase in all the instructors' understanding, knowledge and skills in instructional technology integration. The five most frequent words used in their definition of instructional technology integration at the beginning of the professional training were: lesson, classroom, like, use and teaching. Interestingly, at the end of the professional training, the most frequently used words were: learning, learners, pedagogical, teaching and process. This shows a great difference between the definitions they gave at the beginning and at the end of the study. For instance, when Ayberk was asked about the aim of instructional technology integration into the ELT class, he gave a rather broad and vague explanation: "It is making use of technology in your class so that your students benefit from technology" (Ayberk-Pre-interview). There was no reference to pedagogical or content-wise implications but focused on technology only. There was also no mention of to what extent or how to do so. At the end of the professional training, he reached a more accurate understanding, stating: "The use of technology in a learning environment both by the teacher and learners to achieve the objectives in a lesson. It is not only related to the teacher; the students are also involved in this" (Ayberk-Post-interview).
More than half of the participants underlined the importance of the multimodal reflection as the leading factor in their improved understanding and practice of instructional technology integration during the professional training. For example, Fulin wrote the following discussion forum post "More importantly, reflection is used as a unique method of expanding our knowledge and skills in integrating new technologies in our classes". Three instructors pointed out the tailored feedback they had received, and three referred to the reflection templates. They also referred to the design of the professional training, online resources, tutor's guidance, collaboration with colleagues, assigned articles, training sessions, students' engagement and exploratory approach.

Increasing variety in interaction patterns set by the technologies selected.
The participants integrated technologies so as to vary the interaction patterns in their lessons. There was evidence of a greater variety of communication patterns, enabled by the use of technologies towards the end of the professional training. The students were given more chance to interact with each other, either in groups or in pairs, or both. None of the students had previously experienced interacting with a virtual character, so this was a new interaction pattern for them. The time allocated for the student-to-student interaction was also much higher in the lessons towards the end of the course. This shows that the instructors were successful in creating more student-centered lessons by integrating technologies in their teaching.
Giving more tangible evidence of the achievement of language learning objectives by integrating technology. All the participants showed progress in making a link between the technologies and their lesson aims. By using the evidence-based multimodal reflection tools, they also provided more concrete and multimodal evidence for the achievement of the objectives. The evidence-based multimodal reflection tools, such as images, short videos and video annotations added multimodality and concrete examples from the lessons, thereby providing evidence regarding the achievement of lesson objectives. The greater use of images, short videos, and video annotations towards the end of the professional training reflected their increased confidence, knowledge and practice of various technologies, and the effectiveness of evidencebased multimodal reflection tools.
Increasing their understanding of reflection. The data displayed that this professional training helped the instructors increase their understanding of reflection, and practice of reflective skills via multimodal reflection tools. The biggest progress was in the use of e-tools for reflection. As also seen in the quantitative data, all the instructors learned and improved their use of evidence-based multimodal reflection tools using the following methods: e. creating short video extracts from his/her recorded lesson to reflect on certain points.

Increasing depth in their reflections.
Even in their first reflections, Fulin and Pera made evaluative statements "Compared to WhatsApp desktop that I had used the previous day with the same students, MM worked more successfully, grabbed students' attention and motivated them to respond immediately" (Fulin-Self-reflection1). They also demonstrated the high quality of reflections on their peers' recorded lessons. Ayberk and Jenny presented some high-quality reflective statements at the beginning, but the number and quality of their in-depth reflections increased towards the end of the professional training, which shows progress in their reflective abilities. Devran, Mert, Mateo and Frank, on the other hand, at the beginning described what had happened in their classes in a rather unreflective way. Their later reflections, especially in their third and fourth lessons, showed signs of deeper reflection: "Using the class management tool togroup generator to group the students but seems impossible. It also turns out some students on the list are absent. But the show must go on!" (Frank-VideoAnnotation3).

Quantitative Findings
The evaluation criteria form. Each of the eight participants was evaluated four times according to the 18itemed criteria. A non-parametric Friedman test of differences among repeated measures was conducted to understand whether there was a statistically significant difference between the mean ranks of the related groups (lessons) in each construct (technological, pedagogical, content and evidence-based multimodal reflective practice). The results rendered a Chi-square value of 24.00 for the pedagogical, content and evidence-based multimodal reflection components and a Chi-square value of 23.734 for the technological component and p < .05 for each construct. The results of the four Friedman tests conducted for each construct showed a statistically significant difference between the instructors' lessons, i.e., there was meaningful progress in the instructors' technological, pedagogical and content knowledge (RQ 1) and evidence-based multimodal reflection (RQ 2). Table 6 displays the scores of all instructors from each construct in the evaluation form. The descriptive statistics and Chi-square values of each construct are also given in Table 7. Dunn-Bonferroni post-hoc tests were also conducted for each construct (technological, pedagogical, technological and evidence-based multimodal reflection) in this 'evaluation criteria form'; and the adjusted p values displayed 13 / 22 that there were significant differences between Lesson 1 and Lesson 3, Lesson 1 and Lesson 4, and Lesson 2 and Lesson 4 in each construct. The technology integration questionnaire. The questionnaire involves five constructs: performance expectancy, effort expectancy, social influence, facilitating conditions and behavioral intention. Table 8 displays the participants' scores from each construct at the beginning and at the end of the professional training.  -performance  28  28  29  26  26  27  30  31  Post-performance  33  30  30  32  30  30  32  33  Pre-effort expectancy  14  17  31  22  20  23  30  27  Post-effort expectancy  27  23  31  27  26  28  33  33  Pre-social influence  26  23  24  28  22  23  29  28  Post-social influence  27  25  24  28  25  25  29  29  Pre-facilitating  10  11  14  19  12  13  19  18  Post-facilitating  18  16  17  19  18  18  20  20  Pre-behavioural  9  10  15  15  13  12  16  18  Post-behavioural  17  15  18  16  18  18  17  20 The Wilcoxon signed-rank test indicated that this difference was statistically significant z=-2.52, p=.012. The effect size (r=Z/√n) for this analysis (r = .89) was found to exceed Cohen's (1988) convention for a large effect (d = 3.9). The statistical values of each construct which show a statistically significant difference are also given in Table 9.

DISCUSSION
The data gathered demonstrated a meaningful change in the in-service English language instructors' technology integration and evidence-based multimodal reflective practice. One important point to note is that this study involved technology in two dimensions: (a) technology integrated into the ELT class (b) technology integrated into reflective practice.

The change in the instructors' technology integration.
This professional training changed the way the English language instructors integrated technology into their teaching. One interesting finding is that it is challenging to overcome negative feelings or emotions (Hannafin, 1999). For example, Mateo first compared technology integration "to high altitude acclimatization designed for a mountain climb" (Self-reflection template 2). Although he seemed more positive at the end of the professional training, with the analogy he selected: "You wanted a bicycle? Now you have to pedal! ... dealing with all the positive and negative aspects this thing entails" (Self-reflection template 4), it can be understood that the change in his feelings was not as great as the change in his knowledge.
The findings also showed that believing in the value of technologies is more important than high confidence levels. That was obvious in Mert's case. He explicitly stated, "I can handle any kind of technology, but I do not believe that many new technologies help students learn better but add fun to lessons only" (Int1). He had full confidence in using technologies, but his negative attitude at the beginning stemmed from a lack of awareness of the potential benefits of technologies in his teaching. Obviously, having a high level of confidence did not automatically result in technology integration. After a change of beliefs regarding technologies in an ELT class, he gained a more positive attitude, in line with Vongkulluksn et al.'s (2018) research, which compared value and ability beliefs among teachers.
The instructors were asked to give the definition of instructional technology integration, since definitions determine the scope, purpose, methods, tools and criteria for assessment. This upholds Dockstader's (1999) argument that the ability to define technology integration is the starting point in order to achieve it. Therefore, it could be claimed that after re-defining instructional technology integration, the instructors gained a sounder understanding, which helped them achieve technology integration. The change in their understanding of technology integration could also be regarded as a factor influencing how they teach (Fabry & Higgs, 1997).
Instructors are expected to have sufficient level of TPACK, and to develop this throughout their professional life (Swallow & Olofson, 2017). The reason why this professional training had an impact on the instructors' skill of integrating technologies into their teaching could be attributed to its design. In line with Hew and Brush (2007), the training involved the knowledge and skills of technology, technology-supported pedagogy and technology-related content and classroom management. Another point is that learning how to use technology within ELT functioned like a potential accelerator in the process of technology integration because instructors could more easily implement what they had learned in their training (Snoeyink & Ertmer, 2001/2002. The technologies integrated into their teaching were aligned with their existing pedagogical and content knowledge, further contributing to their technological, pedagogical and content knowledge and practice (Ertmer & Ottenbreit-Leftwich, 2010). The results of the 'technology integration questionnaire' demonstrated that this professional training contributed to the instructors' readiness to accept and integrate technology regarding the constructs identified by the unified theory of acceptance and use of technology (UTAUT) model (Venkatesh et al., 2003). The analysis of the instructors' initial and final responses (p = 0.012) clearly showed the meaningful impact of this professional training on each of the UTAUT constructs. Thus, it denotes that this professional training made the instructors more inclined to embrace innovations in their teaching.
The findings of this study presented further insight into the interrelation between integrated technologies and the effectiveness and quality of ELT classes. The first point is that the integrated technologies allowed the students to become more autonomous (Hennessy et al., 2005) taking the responsibility of their own learning. Another reason was that the learners were able to master basic language skills (Young, 2003). The learning environment took on a more enjoyable and exciting aura (Young, 2003). In their self-reflection templates and video annotations, almost all instructors referred to their learners' excitement and joy in their technology integrated lessons. Another point to note is related to the selected technologies which provided the learners with activities that required them to collaborate with their peers. This is also supported by previous research arguing that the integration of technologies varies the interaction patterns between students and the instructor (Kaya, 2015;Kern, 1995). The selected technologies also helped the learners use their higher-order thinking skills (Yang, 2012) and receive individualized feedback (Eröz-Tuğa, 2013).
The most significant reason for the improvement in the instructors' effectiveness could be attributed to the close link between technology integration and evidence-based multimodal reflection. Instead of explaining or describing how they integrated technologies, the instructors were able to share their recorded lessons on their Blackboard page. Their peers could click on the hyperlinks to witness how that technology made a change in the lesson. The instructors also shared their video annotated recordings with each other, i.e., asynchronous dynamic interaction. Thus, it can be deduced that the effectiveness of the technologyintegrated ELT lessons was due not so much to the individual activities, but more to the collaboratively conducted evidence-based multimodal reflection through hyperlinks, full lesson recordings, short video clips, Blackboard pages, and interactive video annotations (Sherin, 2007).
The change in the instructors' evidence-based multimodal reflective practice. Apart from the change in the instructors' technology integration, the findings also demonstrated that this training had an impact on their understanding, practice and quality of reflection involving evidence-based multimodal reflection tools. The instructors reframed their understanding of reflection throughout the study. When asked to express what they understood reflection to be at the beginning, the instructors gave a variety of definitions, reflecting Black and Plowright's (2010) point that there is no standard meaning. However, what is interesting is that the less experienced (e.g., Mateo) and those never previously involved in reflective practice (Frank, Mert) regarded it as diary keeping or mere description of what happened, and could not give a proper definition, although they had heard the term quite often. This finding conforms with Gün's (2011) view that instructors often hear of 'reflection' during their professional career and are continually encouraged to reflect; however, few can successfully do so since they are not clear about the meaning.
The increasing depth in the instructors' evidence-based multimodal reflections is in line with the categorization of reflections created by Leijen et al. (2012): description, justification, critique and discussion. The majority of the instructors gradually gained depth in their reflections throughout the study. One of the main reasons for the higher quality of their reflections was the multimodal dimension of their reflective practice. When their reflections were analyzed, it was noticed that the more multimodality involved in their reflections, the better the quality of their writing. Thus, it can be inferred that the multimodal nature of their reflections contributed to a deeper understanding via dual channels (Sweller et al., 1998). Devran said "Thanks to the multimodal dimension they [evidence-based multimodal reflection tools] add, we do not need to describe what we experienced in pen and paper" (Devran -Interview2). This remark is echoed by other instructors, denoting that they were less dependent on written forms . Their reflections enriched with multimodality presented rich and thick descriptions (Schmid, 2011). To illustrate, Ayberk said "You live those moments again and again" (Interview2). Similarities in their comments help to drive an understanding that, thanks to the multimodality, the instructors had access to realistic and vivid 16 / 22 descriptions of their class through video recordings and video clips and were able to relive specific teaching episodes (Hiebert et al., 2002). Their reflections showed that they also became more observant of the key moments that they may otherwise have missed, i.e., improved noticing skills (Marsh & Mitchell, 2014).
Among all the evidence-based multimodal reflection tools, all the instructors openly stated that video annotations were the most effective, as supported by Nilsson and Carlsson (2019). The instructors referred to the ease of use, effectiveness, interactivity, multimodality and evidence-based nature of the video annotations. They highlighted that this interactive evidence-based multimodal reflection tool helped them in the following five respects: • take notes on any certain moment of a video (Perez-Torregrosa et al., 2017): "We could just write our reflections on the video" (Mert -Interview2).
• save time by making to-the-point reflections instead of peripheral descriptions (Howard & Myers, 2010): "…thanks to this technology, we do not need to describe any moment…" (Fulin -Interview2); "…one can easily find the relevant moment" (Fulin -DiscussionForum).
• give detailed insight into their strengths and weaknesses (Picci et al., 2012): "…minute by minute panopto discussion is also critical in specifically identifying the parts of the lesson that went well and which parts can be improved" (Frank-DiscussionForum).
• witness undeniable evidence from their class : "provide such concrete evidence that you cannot miss or turn a blind eye" (Fulin -Interview2).
Apart from the video annotations, the instructors benefited from the hyperlinks and short video clips. They not only freely accessed the relevant moments but also achieved a sense of ownership, supporting the argument that people learn better when they create something meaningful for themselves (Calandra et al., 2009). The instructors' willingness to edit and create short videos also supports Yerrick et al. 's (2005) argument that teachers prefer the video-enriched reflection process to written reflections.
The 'Technology Integration Through Evidence-based Multimodal Reflective Professional Training' made a significant impact on the instructors' technology integration and evidence-based multimodal reflective practice. It provides evidence against Hubbard's (2008) claims that in-service trainings are unable to bring a change in teacher behaviors, and that they usually follow the methods learned as students or trainee teachers. As the main contribution of this study, the evidence-based multimodal reflection tools fostered the process of technology integration and increased the quality of reflection.

CONCLUSIONS AND IMPLICATIONS OF THE STUDY
The 'Technology Integration Through Evidence-based Multimodal Reflective Professional Training' successfully changed the instructors' technology integration and their reflective practice. The results of the present study have contributed to the literature, providing implications for the professional trainings aiming at technology integration through reflective practice for in-service English language instructors at tertiary level. The satisfactory impact of this professional training provides implications, revealing its critical role in progress in technology integration and reflective practice. First, the instructors were provided with the opportunities for developing their teaching practices (Ertmer & Ottenbreit-Leftwich, 2010). They had the opportunity to actively experience new technologies (Driscoll, 2002) not only in the training sessions under tutor and peer supervision but in their own classes as well. The small number of instructors allowed individualized and tailored assistance throughout the professional development. The numbers were kept to the minimum to provide personalized professional training (Hixon & Buckenmeyer, 2009).
Although there were training sessions, the whole professional training extended beyond these, because these alone would not have been sufficient to fully support participants (Koehler et al., 2007). The findings illustrate that, as Rogers (2003) suggested, the five attributes of innovations played a critical role in the instructors' technology acceptance and integration; therefore, such attributes should be made apparent for all participants in professional trainings involving innovations.
The success of this study shows that technology integration is unlikely without time for instructors to test and try the target technologies by themselves (Francom, 2020). Providing instructors with sufficient time might reduce the pressure and stress and contribute to their professional development. Also, the professional training was situated in the instructors' authentic (Driscoll, 2002;McNeil, 2013) and small-sized (Francom, 2020) contexts. In this study, the class sizes were kept between 15 and 20. Based on this, it could be suggested that training contexts should be the same as the implementing contexts, instructors should work a group of 20 students or fewer, and training should involve familiar instructional activities.
As in many previous studies, in this professional training, reflection was an invaluable method for teacher development; and its results highlighted certain implications for such trainings in the future. First, instructors would benefit from having the meaning of reflection clarified and being able to internalize it at the very beginning. In order to be able to reflect better, reflective practice training and well-prepared reflection tasks presented at an individualized pace are of utmost importance.
This study was conducted as a doctoral dissertation at the department of the computer and instructional design at a foundation university in Turkey. The results of this unique study clearly demonstrated that evidence-based multimodal reflection was the prominent factor in the improvement in the instructors' technology integration and reflective practice. Based on a sound methodological foundation, with clear implementation steps, and the necessary tools to collect, analyze and evaluate data, this study provides an example for further in-service ELT professional trainings aiming at technology integration or reflective practice, or both.