AN INTRODUCTION TO REUSABLE LEARNING OBJECTS IN VETERINARY EDUCATION

Reusable Learning Objects (RLO), also known as Shared Content Objects (SCO) are self-contained digital resources such as video, audio, web-pages, documents and graphics which are stored and accessed independently in support of a blended learning approach. This feature of technology enhancing learning is also shared by Open Educational Resources (OER) as defined by Clements and Pawlowski (2012). Kay and Knaack (2007) expand on this definition by saying that RLO are interactive tools which go beyond simply supporting learning, but enhance and amplify the cognitive processes of learners. As yet there is no evidence to suggest that this applies to learners within veterinary education blended learning and therefore to promote effective practice in this subject area, the authors set out to better understand why and how some students in veterinary education would choose to access web-based RLO and others choose not to and secondly, if the performance in summative assessment by students in veterinary education is affected by the introduction of web-based RLO.

REVIEW METHODOLOGY

Literature was selected firstly on the basis of its relevance to RLO, secondly in terms of the course areas from which participating students were drawn and lastly, in terms of its relevance to the enhancement of higher education. The authors searched for literature using Google Scholar and their home institution’s own search engine (FindIt@Harper) which searched EBSOHost, ScienceDirect and Wiley Online databases to obtain 21 research publications and nine sector reports. The search terms included: reusable learning object, veterinary blended leaning, veterinary online learning, science reusable learning objects and sources filtered for full text copies.

In searching for papers to include, only one was found relating to Veterinary Education (Short, 2002) and another concerning Animal and Agricultural Sciences (Hoover & Marshall, 1998). Other subject areas in which studies have been undertaken in relation to RLO and reported in this review include Biology (Kay and Knaack, 2007), Chemistry (Windle et al., 2011; Kay and Knaack, 2007), Physics (Kay and Knaack, 2007; Kurilovas et al., 2011), Human Nursing (Windle et al., 2010; Keefe and Wharrad, 2012; Blake, 2010; Lymm et al., 2008), Human Anatomy (Johnson et al., 2013), Human Medicine (AAMC, 2007; Blake, 2010), Health and Social Sciences (DoH, 2011; Kirkwood, 2008; Evans, 2013; Boyle et al., 2003), Science (Littlejohn et al., 2010; Kirkwood, 2008), Business (Littlejohn et al., 2010), Maths (Kurilovas et al., 2011), Accounting (Concannon et al., 2005), Engineering (Littlejohn et al., 2010), Computer Science (Kay and Knaack, 2007) and lastly International Culture (Sandlin et al., 2014). In the main, these are STEM subjects with the majority offering insights into the medical education sphere. The sample sizes utilised in these studies, where specified, contained a minimum of 80 students, with one exception being the investigation by Kirkwood (2008) where data was collected from just ten individuals. The largest sample was associated with the study by Littlejohn et al. (2010), in which over 2,000 students participated and, while we know that the greater the sample size, the greater the chance of detecting a true treatment effect (Petrie & Watson, 2006), such a large scale study may not allow us to see the nuances of RLO use which are of interest to practitioners.

The consequence of selecting this particular area of focus, with limited availability of subject-related literature, is that the aim of this thematic review was refined to assess what is currently known about RLO and how those concepts can be applied to enhance the delivery of veterinary-related degree courses.

DOES STUDY MODE OR POINT-ON-COURSE MATTER?

The student demographic examined in these studies, where stated, concerned largely mainstream (full-time) undergraduate degree students, with the exception of the studies by Keefe and Wharrad (2012) and Evans (2013) respectively who investigated postgraduate students and Kay and Knaack (2007) who explored behaviours of secondary school pupils. In terms of the year groups (e.g. first, second, third and so on) that were examined, this was unspecified in a number of papers (Windle et al., 2010; Lymm et al., 2008; Sandlin et al., 2014; Bacsich et al., 2011; Short, 2002; Marcus, 2013; Kurilovas et al., 2011; Kurilovas et al., 2014; Hoover and Marshall, 1998). Students from a range of year groups were examined in studies by Keefe and Wharrad (2012), Blake (2010) and Boyle et al. (2003). Interestingly, Johnson et al. (2013) specifically dealt with third year Bachelor of Medicine students and Concannon et al. (2005) and Littlejohn et al. (2010) captured data from first year students only. Data collection for the latter study took place on course induction day which makes this methodology unique in terms of those studies reviewed herein. To go beyond the mainstream student demographic, the review by DoH (2011) and study by Kirkwood (2008) provide an insight into mature learners undertaking further study. Thus the literature base is wide ranging in terms of the point-on-course at which data was captured.

DOES LEARNER GENDER MATTER?

Between 87-95% of students who participated in the studies by Blake (2010) and Lymm et al. (2008) were female, which goes some way to illustrating the typical demographic seen on degree nursing courses. However, this does highlight the need to explore the impact of RLO across a range of settings to fully understand the behaviour of both male and female students.

So, although there have been a number of studies undertaken to examine the role of RLO in blended learning higher education, they originate from medicine and health sciences in the main. Therefore, in the first instance, practice in veterinary education must draw from findings from this allied subject area.

STUDENT’S CONFIDENCE IN USING COMPUTERS

Given that the definition of RLO (explained above) referred to them as being digital in nature, it is no surprise that five studies included in this review mentioned the extent to which the students were confident in using computers or other devices during their studies. Over two thirds of each cohort examined by Windle et al. (2011) self-reported a high level of computer competence with the highest incidence of low confidence found within the most recent cohort. Furthermore, this study found that there was no correlation between computer confidence, study mode (using RLO in a self- or workshop-facilitated setting) and age in terms of how students rated RLO. This finding may challenge our immediate ‘in-practice’ assumptions. Where the use of such technology may have been positively approached by students in a number of studies (Windle et al., 2011; Concannon et al., 2005; Kay and Knaack, 2007), Littlejohn et al. (2010) explained that those students whose previous approach to learning did not involve using technology preferred to continue this strategy. This may go some way to explaining the reported confidence levels within the above mentioned study by Windle et al. (2011). Student opinion about technology enhanced learning (TEL) remained unchanged even with the introduction of RLO to support learning (Johnson et al., 2013). The findings of Concannon et al. (2005) and Littlejohn et al. (2010) agree in that some students did not want the flexibility that RLO offered in terms of learning independently and instead requested tutor-facilitated workshops in which to complete tasks. In practice, the authors would consider that such a workshop could be adapted to take a peer-led format, where students who are further through their chosen degree courses facilitate that session. Subsequently, that could lead to a deeper approach to learning for those facilitators as they would need to demonstrate an elevated level of understanding of the topic in order for either student group to benefit. Using peer assisted learning may also develop confidence in the lower year group in the use of computers, or other devices, as well as independence in blended learning format.

Blake (2010) identified a difference in computer confidence across courses, with nursing students reportedly being three times less confident than medicine students. A difference in attitude to using technology may also be seen by remote and/or adult learners as evidenced in the paper by Kirkwood (2008), where mature students reported that the computer skills they developed at work could then be utilised in their studies. This would be advantageous to learning in light of the findings reported by Littlejohn et al. (2010) who found that students who perceived themselves to have advanced computer skills were most likely to use technology-related study methods. We would anticipate the opposite being the case for typical undergraduates; spending time developing computer-based competencies during their studies which could then be applied in the graduate workplace. Keen use of technology was also noted in students with restricted mobility or who were geographically remote to the institution.

The key message that we can arrive at so far is that there is agreement across the literature that students arrive at university with pre-conceived perceptions of their self-confidence in using computers and technology, and this has been shown to affect how they want to approach studying in HE. Confirmation is needed, however, that this applies to a range of subject areas and settings, including the small specialist universities.

PEDAGOGIC PRINCIPLES UNDERPINNING RLO DEVELOPMENT

The literature identifies that the underpinning rationales for developing RLO vary, but primarily focus around three themes, the first attracting most comment:

Helping Students Achieve Higher Grades

As educators we acknowledge the need to provide more resource (in the same or less time) to teaching basic sciences in a clinical context (Windle et al., 2010), to help students achieve higher marks in summative assessment and/or an improved overall outcome (Trowler, 2010), so we typically supplement face to face teaching (Lymm et al., 2008) to create a blended learning format. The need to do this may in part be explained by the challenging nature of a subject for some students (Windle et al., 2011; Lymm et al., 2008). Lymm et al. (2008) went on to report that 37% of students felt insufficient time was spent on challenging topics during their studies. However, in practice, from across a group of students, individual opinions as to the difficulty of a topic may vary in each annual intake. What was a difficult concept for some members of last year’s cohort may not be the case this year, and so on. The literature goes on to tell us that some students feel they lack time and control to study content heavy modules, so they take a superficial approach to their studies (Windle et al., 2011). This therefore supports the notion that students need to have autonomy (AAMC, 2007) and flexibility in terms of when (Johnson et al., 2013) where, and the pace at which they learn (DoH, 2011) as well as access to well-designed modules, in order to develop a deeper approach to learning. This ethos has been described in the literature as ‘ownership of learning’ (Windle et al., 2010; Johnson et al., 2013) and the nature of RLO in terms of them being web-based facilitates this need.

In line with the principles underpinning andragogy, a more flexible approach to module delivery using technology could well be favoured by students who as young adults are becoming increasingly self-regulated, and on this basis, although literature indicates we should design flexible teaching and learning, we do need to ensure that all students are able to access an equitable learning experience (AAMC, 2007). The review by DoH (2011) stated that, in using RLO, we were of necessity offering a blended learning curriculum, although this was not reflected in the self-study arrangements as set out in the study by Windle et al. (2011).

Students should achieve higher grades where they demonstrate they meet real-world learning needs, which may include the attributes of a successful professional in practice. In the context of veterinary education, the alignment of RLO to defined real-world learning needs (Windle et al., 2010) identifies those resources as being ‘patient centred’, ‘service driven’ and ‘evidence-based’ (DoH, 2011). Adult learners meet real-world learning needs by undertaking Continuous Professional Development (CPD), one route being to engage with RLO, to progress within their employment and develop their professional practice. RLO are well suited to CPD initiatives given their scope to reflect bite-sized chunks of learning, since we know that smaller learning packages are more effective than large resources (Windle et al., 2011) as they avoid cognitive overload (AAMC, 2007). While much of the development of RLO requires a focus on technological functionality, as illustrated by the formative assessment of RLO in the study by Evans (2013), it is the learning qualities which should be prioritised over the technical. However, for RLO to be utilised optimally, a key technical requirement is the system of signposting to the resources which needs to be in place, both for students and staff (Johnson et al., 2013; Evans, 2013). Signposting has been rated a higher priority than the provision of IT training (Concannon et al., 2005).

We know that student engagement per se is the extent to which students take an active role in a range of educational activities and research tells us that this process is likely to lead to high quality deeper learning (Trowler, 2010). To be effective, therefore, RLO need to be integrated into the blended learning course content (Windle et al., 2010; AAMC, 2007) and require students to actively engage with that content (Johnson et al., 2013; AAMC, 2007). This needs to be well thought out; the way that online resources are integrated into learning may be more important than the fact that such items are available (Johnson et al., 2013). For example, we know that collaborative learning is beneficial (Plendl et al., 2009), so we could encourage students to use RLO in pairs or groups where appropriate. Beyond helping students to achieve higher grades, literature indicates that RLO development needs to involve others so that students have the opportunity to experience relationships with peers and operate within a learning community (Windle et al., 2010). This could be facilitated at the RLO design stage as illustrated in the study by Evans (2013), where students were invited to contribute to the brainstorming process, providing a platform for peer assisted learning.

RLO provide an opportunity to implement a constructivist approach to learning and teaching, whereby tutors facilitate the building of knowledge. From the perspective of clinical skill development, Miller’s Model of Clinical Competence illustrates how this may come about (Short, 2002). Sandlin et al. (2014) stated that social learning theory should underpin our development of resources, taking into account that personal, behavioural and environmental factors all affect learning. However, depending on the context and aims of learning, some RLO may be behaviourist by design if the task necessitates. Literature tells us that one purpose of RLO is to provide opportunities for students to learn new skills (Windle et al., 2010) within a controlled environment, at a range of difficulty levels and with arrangements for regular feedback (AAMC, 2007). The feedback functionality is all-important, especially given that a lack of feedback early on in a student’s course has been associated with student dissatisfaction, failure and departure from studies (HEA, 2012). A lack of feedback opportunities within a learning resource could then lead us to categorise it as ineffective. We also have the opportunity to expose students to ‘assessment for learning’ (Bacsich et al., 2011) which, given our aspiration for our students to achieve higher grades, formative assessment and feedback as a function within RLO would enable better preparation for the high stakes summative assessment.

This leads us to draw some interim conclusions in that we can help our students to achieve better outcomes from their blended learning experience by providing flexibility, autonomy, correctly proportioned and well signposted resources, which together have been shown to lead to a deeper approach to learning.

Professional Practice

Like other vocational disciplines, veterinary professions are subject to change in terms of professional practice or policies (Windle et al., 2011; Blake, 2010) and have typically profession-driven curricula (Keefe and Wharrad, 2012). Both these issues could be effectively addressed via the provision of RLO within a blended learning format, which have been reported as effective in developing professional practice for international graduates who are commencing postgraduate courses in the UK but who lack exposure to that country’s professional contexts (Evans, 2013). The DoH (2011) reported that professional bodies recognise that new graduates need ongoing clinical and practical skills training most effectively delivered online, as well as needing to utilise technology in the workplace (Blake, 2010).

Institutional Needs

Since the 1980s, the development of learning technologies has been predicated on three ideas; new infrastructure, creation of learning content and the implementation of teaching practices to accommodate the new technology (Short, 2002). Development of RLO has in some cases been driven by the need for institutions to save money (Johnson et al., 2013; Kurilovas et al., 2014), be more competitive and attract a wider cross section of the potential student market; including part-time, those from under-represented economic backgrounds and/or lifelong learners who, when they graduate, are competent in vocation relevant IT packages (Concannon et al., 2005). Where globalisation is high on the agenda for many institutions, there is motivation to improve the global awareness of students (Sandlin et al., 2014).

In addition, institutions have in some cases needed to reduce staff contact time within a module (Johnson et al., 2013), although teaching and learning content delivered to students can remain unchanged due to the constant availability of online resources in blended learning, which inherently require less manual intervention from an academic member of staff (AMMC, 2007; Kurilovas et al., 2014). The literature identifies with elements of the ‘effectiveness versus efficiency’ argument (Kurilovas et al., 2014). From an administrative perspective, internal systems within an institution utilise technology, so we should consider why the student facing platform for learning cannot also be technological and offer similar advantages in content management (Short, 2002).

To create effective RLO and hence students with a deep approach to learning, the pedagogic style of RLO should be educationally coherent, innovative (DoH, 2011) and consistent within and between resources originating from the same suite (Windle et al., 2010). However, in order to develop such a resource, we first need to understand the learning gaps of our students (Blake, 2010). The stages to RLO development have been documented by Windle et al. (2010), as well as by the RLO-CETL which promotes a 7-step production process (Evans, 2013). Production is typically tutor centred, although there is agreement in the literature that the end goal is to produce a resource which is student centred (Windle et al., 2011; Keefe and Wharrad, 2012; Sandlin et al., 2014) and used in a self-directed manner (Windle et al., 2010). A slightly different approach was taken by Kay and Knaack (2007) where they developed RLO for use solely in tutor facilitated sessions. In practice, this strategy would limit the extent of RLO reuse and hence the scope for deeper learning in students. In practice, where a number of academic staff work to develop RLO to support their own teaching, it is possible that production style will differ to some extent, an issue which was documented by Sandlin et al. (2014).

Where RLO complement a range of different teaching and learning strategies, reuse (or the extent to which RLO ‘travel-well’) increases (Windle et al., 2010). In veterinary education, those different teaching and learning strategies represent clinical variation (AAMC, 2007). In addition to complementing a variety of teaching styles, Kurilovas et al. (2011) stated that RLO need to be developed for use within different platforms and modifiable to suit differing staff and student needs.

Institutions may well need to change the extent of academic staff input to teaching, learning and assessment for a number of business or financial reasons. RLO, developed in line with good practice guidance can provide an efficient and effective alternate means of delivering content.

WHY DO STUDENTS CHOOSE TO USE, OR NOT TO USE RLO?

The question of whether students choose to use RLO as part of a blended learning format is reported in the literature as being affected by a number of factors, one being the student’s prior experiences (Bacsich et al., 2011; Kirkwood, 2008). Use occurs where students have a positive attitude towards computers and they prefer to use technology-based resources (Concannon et al., 2005). Analytical (field-independent) learners prefer to learn in quiet settings, with the sole aim of completing the task set (e.g. formative assessment as evidenced by Bacsich et al., 2011 and Kirkwood, 2008), compared to field-dependant learners who prefer a more informal setting, with background noise, frequent breaks and multi-tasking (Hoover and Marshall, 1998). Blended learning and RLO could meet the needs of both types of learner as they provide students with control (Windle et al., 2011), often where there is a large choice of content which is constantly available (Bacsich et al., 2011). Therefore, a student can maintain their study pattern (Concannon et al., 2005), which is particularly important if students have limited time or need to be strategic in their studies (Kirkwood, 2008; Blake, 2010). The opportunity that mobile devices offer in terms of learning anywhere and anytime were recognised by Short (2002).

Students choose to use when the RLO is aligned to their learning needs (Windle et al., 2010; DoH, 2011; Concannon et al., 2005) which is notably important for those with specific learning difficulties (Blake, 2010). From the perspective of international students, RLO have provided support for the development of language and vocabulary skills (Evans, 2013), an issue also arising in subjects with an extensive technical vocabulary such as veterinary education. Pausing a video in order to look up unknown vocabulary would enable such a student to develop a deeper understanding of the topic addressed by the RLO. The reasons why students chose to use RLO can be closely linked to the pedagogic principles underpinning the production of such resources described above. Commonalities exist between the reasons for students choosing to use RLO and how we should approach the development of such resources. This evidence should inform our practice. Literature indicates that students choose to use RLO when the resources have been produced using consistent pedagogic approaches and complement different learning and teaching styles (Windle et al., 2010; DoH, 2011). Although a consistent pedagogic approach has been found to influence use, Windle et al. (2011) found that design differences across a suite of RLO did not negatively impact on student learning. Students opted to use RLO in cases where they realised the usefulness of the resources as well as the scope for subsequent reuse (Lymm et al., 2008). Students chose to use RLO in cases where they perceived the resources may give them an advantage (Kirkwood, 2008) by being interactive. This, and the opportunity to use a range of multi-media (Bacsich et al., 2011) which they perceive is well supported by tutors (Concannon et al., 2005) are influencing factors in student’s choosing to access RLO. Kay and Knaack (2007) stated that there was considerable variation in student perceptions of the benefits and qualities blended learning and RLO offer. Clements and Pawlowski (2012) argued that uptake of RLO as open educational resources was still low due to a perception of teachers that these resources lack quality.

RLO have been ranked below lecture notes and text books but above journals and other web resources as means of enhancing blended learning (Lymm et al., 2008). Other reasons as to why students have chosen to use RLO include the enhancement of personal knowledge (Bacsich et al., 2011; Kirkwood, 2008) in intrinsically motivated students, to explore interests outside of a professional field, to remain current, simply because they are aware of the resources which are easy to access (Bacsich et al., 2011) and the use of which peers (Concannon et al., 2005) and tutors (Kirkwood, 2008) promote.

Kirkwood (2008) went on to report that students chose to use RLO to answer their questions on a topic, or fill the gaps in order to fully grasp a concept. They then avoided drawing attention to themselves by asking questions in class and the resultant feeling of embarrassment (Blake, 2010). Although this may be considered by students as a positive functionality of RLO, there ought to be sufficient provision in place for students to ask questions and gain prompt feedback without the feeling of peer-pressure. However, in cases of cognitive overload, where material is overwhelming (Bacsich et al., 2011; AAMC, 2007) students have reported a preference to ask questions in person of their tutor rather than via an online forum (Littlejohn et al., 2010).

Ninety-four percent of students have reported that they would recommend RLO to others (Windle et al., 2011) and a separate study found that 94% of students reported they would reuse the RLO themselves (Blake, 2010). However, this has been found to vary across courses, for example Blake (2010) found that nursing students used RLO more than medical students. Reuse has been documented as highest in graduates who had the lowest level Biology qualifications on entry to their course (Lymm et al., 2008). However, Bacsich et al. (2011) took the view that learners were predictable and simply reused resources which had worked for them previously. Actual reuse has been documented as being typically low (Windle et al., 2010), so based on Bacsich’s concept of predictability, that infers that initial usage levels are also low. An explanation for the low usage levels is provided by Kirkwood (2008), who suggested that lecturers lacked awareness of the motivating factors that would encourage RLO use in their students. Usage levels may be lower than we would hope, but the impact of usage shows the value of engaging with RLO via blended learning.

Of the studies which explored this issue, students reported that they chose not to access RLO where they were using self-contained resources that did not require reference to further or external sources (Kirkwood, 2008), where they perceived an additional time commitment in using such resources (Keefe and Wharrad, 2012), where they experienced a lack of support from tutors (Bacsich et al., 2011), computer anxiety or accessibility issues (Lymm et al., 2008; Blake, 2010; Concannon et al., 2005; Kirkwood, 2008). This can be expanded to include a lack of guidance in blended learning (Bacsich et al., 2011; Blake, 2010) or poor instructions for use (Kay and Knaack, 2007). The behaviour of students in terms of whether they chose to access RLO was documented by Concannon et al. (2005) who found that students who ‘crammed’ did not engage with RLO. We can learn from this on a practical level as educators and design learning and teaching strategies in which resources are signposted early on and with which students are required to study at a more continuous pace.

Furthermore, the lack of external motivation, e.g. certification (Bacsich et al., 2011) or the task being made optional for students (Kirkwood, 2008) has negatively affected reported RLO use. Attributing some marks to the student’s engagement with blended learning and RLO within the associated mark scheme may have increased usage levels in the study by Concannon et al. (2005). If the RLO addressed a topic that was not of interest (Bacsich et al., 2011) or had been launched after the topic has been learned (Kay and Knaack, 2007), students were reported to opt out. Concannon et al. (2005) reported that 81% students found lectures more effective than online learning alone. We know that in terms of a student’s confidence levels in using computers, past experience with technology may lead them to decide against engaging with blended learning and hence RLO (Littlejohn et al., 2010). This is corroborated by Windle et al. (2011) who found that students who were provided with a workshop setting in which to use RLO expressed a preference for this setting. However, the concept of ‘digital migrant, digital native’, where for example students with prior non-exposure to web-based resources (Marcus, 2013) is thought to be inaccurate by Johnson et al. (2013) due to its crude assumption.

The majority of the studies which explored why students chose to use RLO, also captured the reasons why students chose not to use these resources, with the exception of the publications by Windle et al. (2010) and DoH (2011). Kirkwood (2008) found that the incentives did not always outweigh the disincentives and vice versa. The reasons why students chose to use or not use RLO are multi-factorial, but our interpretation of the literature is that well planned out delivery of critically aligned RLO, with sufficient support in place for blended learning, provide a positive opportunity for students to take control of their education.

USING INTERVENTION STUDIES TO ASSESS IMPACT OF RLO ON STUDENT LEARNING

According to Mayes et al. (2009), research into the impact of technology on students was on the increase at that time. Their meta-analysis identified shifts in research themes towards a more holistic study design and outcomes of such research were increasingly being used to inform institutional decisions about technology in learner-centred teaching, learning and assessment. Bacsich et al. (2011) stated that at that time, the question of whether RLO had an impact on student attainment was being debated. The use of a pre-test to assess knowledge prior to introducing RLO to a curriculum is evident only to a minimal extent (Johnson et al., 2013; Sandlin, 2014). The impact of RLO has then been assessed by a post-test in various forms; MCQ written paper (Windle et al. 2011), student survey (Keefe and Wharrad, 2012; Lymm et al., 2008; Sandlin et al., 2014; Blake, 2010), mixed methods (Johnson et al., 2013) and semi-structured telephone interviews (Kirkwood, 2008).

In the studies reviewed, RLO were integrated into blended teaching, learning and assessment in a range of approaches, including offering them alongside lectures (Keefe and Wharrad, 2012), replacing lectures with RLO delivered in tutor facilitated workshops (Kay and Knaack, 2007), self-study or launched in conjunction with a workbook (Windle et al., 2011). This latter arrangement mimics the principle of ‘scaffolding’. RLO have been implemented for use in both compulsory and optional formats, as well as being unlinked to assessment (Kirkwood, 2008). However, based on the findings mentioned so far, the lack of alignment of RLO to assessment would be considered an ineffective strategy. In order to assess impact of RLO, studies included in this review utilised differing numbers of RLO. Johnson et al. (2013) implemented three categories of RLO; formative assessments via an electronic voting system, annotated photographs and scenario-based online quizzes where question style was aligned to formative and summative module assessments.

In terms of capturing the student opinion on blended learning and RLO, Kay and Knaack (2007) found this to be generally lacking at the time. Seven years on, the literature indicates mixed response rates of students of between 40% (Johnson et al., 2013) and 92% (Lymm et al., 2008). The lowest levels of engagement were reported in studies which implemented a questionnaire, which could have been perceived by students as a non-essential task and as such ignored the request to provide feedback. Windle et al. (2011) reported that although only 66% of their study participants engaged with the evaluation process following RLO use, of that group, responses were equitable between students who had used RLO independently and those who had used them within a workshop setting. RLO evaluation in some of the studies reviewed was undertaken using the RLO-CETL toolkit (Windle et al., 2011; Windle et al., 2010; Blake, 2010, DoH, 2011), although it was found to yield inconsistent student responses in terms of their ratings of functionality and media across open and closed question formats (Windle et al., 2010; DoH, 2011). Kurilovas et al. (2011) reported this as a fairly subjective method of assessment. To support the data on impact on learning, studies have also referred to virtual learning environment resource user logs, however they have not been relied on heavily in terms of explaining findings (Concannon et al., 2005; Kirkwood, 2008). In more recent years, studies have been published which propose evaluative frameworks and tools which could be utilised by blended learning / RLO developers and students alike (Kurilovias et al., 2011; Kurilovas et al., 2014).

WHAT IMPACT DOES USING RLO HAVE ON LEARNING AND ASSESSMENT PERFORMANCE?

RLO have been found to have a significant effect on examination results (Windle et al., 2011; Keefe and Wharrad, 2012). Students from tutor-facilitated workshops answered a larger proportion of questions correctly following the introduction of RLO. At face value, one could argue that performance improved here due to the pedagogy underpinning the teaching format rather than the RLO since Windle et al. (2010) reported that using RLO within taught sessions could increase the interactivity of students, however the same improved outcome was found in students from a self-study setting. This is further supported by Windle et al. (2010) who reported that assessment performance improved in students who used online blended learning compared to traditional lectures, the decision for which was unaffected by the student’s level of computer experience. AAMC (2007) argued that impact will only result if RLO are used effectively and appropriately, a notion which is supported by the findings of Kay and Knaack (2007) who reported that students benefitted if a RLO had well organised layout, was interactive and used visual aids to explain abstract concepts. Interactivity is evident in tools such as the virtual patient or patient simulator (AAMC, 2007).

The strategic benefits of using RLO in blended learning could include students being able to progress through materials at a quicker pace, whilst performing equitably in assessments to those who attend tutor led sessions scheduled over a longer period of time (Bacsich et al., 2011). The view of students regarding the extent to which lectures or wet-lab specimens helped them prepare for assessment declined following the introduction of RLO in the study by Johnson et al. (2013). However, this opinion was turned on its head once more online anatomy tools were made available, as students requested more text books and face to face tutorials. Beyond examination results a number of other desirable outcomes have been found to be affected by blended learning and RLO use including learning experience (Blake, 2010), critical thinking, practical competence, skills transferability, cognitive and psychological development, self-esteem, formation of identity, moral and ethical development and student satisfaction (Trowler, 2010; Sandlin et al., 2014). The latter is relevant in terms of the annual National Student Survey. Further qualitative studies of student perceptions have found that the majority of students agreed that RLO use had aided their understanding and they felt they could successfully evidence the module learning outcomes. Furthermore, students have reported feeling confident to use the knowledge learnt with RLO, in practice (Lymm et al., 2008).

The key message arising from literature in terms of why performance is positively affected when RLO are introduced lies with the principle of active engagement. Using a RLO to full effect equates to active engagement with learning content. Research tells us that achievement is positively influenced by the amount of active participation students undertake in their own learning process (Trowler, 2010; Roshier et al., 2011). Those findings are supported by Boyle et al. (2003), who found a positive association between a deep approach to learning and academic outcome. Furthermore, students with a deeper approach to learning have been found to exceed those who take a surface approach. Vermunt and Vermetten (2004) stated that “…engaged students share the values and approaches to learning of their lecturers… spend time and energy on educationally meaningful tasks… learn with others inside and outside the classroom… actively explore ideas confidently with others and learn to value perspectives other than their own.” Littlejohn et al. (2010) found that undergraduates acknowledged that their learning at university would be different to previous learning experiences as well as expecting to have increased personal responsibility for their academic successes in order to enter their preferred vocation.

Blake (2010) reported that students were aware that repeated use (reuse) of resources consolidated learning, which could explain why the proactive students in our classes choose to use them. Concannon et al. (2005) reported that students who were required to undertake one compulsory assessment and a further four optional assessments following the introduction of RLO approached learning as they would have done prior to RLO launch, simply using the RLO as supplements, which developed the learning and teaching strategy into one of blended learning. However, engaging with technology in itself resulted in a shift in learning strategies as it provides a platform for continuous revision, not feasibly replicated by a lecturer in the case of a large group. In the same way that RLO can be reused, Johnson et al. (2013) repeated a semester of anatomy teaching as well as introducing RLO and the outcome was that RLO users achieved an improved performance in assessment over non-users. However, the improved performance from those students, who evidenced their proactive approach to learning by being identified as RLO users, could simply be attributed to the fact that they were given classroom time to revisit topics they had already learned, prior to the assessment. The RLO in this case added a third ‘layer’ to learning the same content.

The opinion of Kay and Knaack (2007) was that assessment of impact required multiple RLO to be available to students and that findings across the sector (secondary school STEM subjects) up until that point had been largely descriptive. A number of studies mentioned in this review did not explore to this detail, however, where findings have been documented, the quantitative extent of impact following RLO use has ranged from a 10% improvement for RLO users over non-users (Johnson et al., 2013) to as much as 19.2% (Keefe and Wharrad, 2012). Although Lymm et al. (2008) captured data around the proportion of RLO users who subsequently felt confident in the subject area, or the proportion who then reported that their own practice had changed as a result of using RLO, comparative data for cohorts of non-users was not provided, so it is unclear whether it was solely RLO use that improved the situation in this case.

The effect of not having access to RLO in students in the same cohort does not seem to be an issue according to Keefe and Wharrad (2012), who found that students in their control group who did not have access to RLO performed comparably to other groups who had undertaken the same assessment without those enhanced blended learning resources. However, this could be due to the voluntary-response bias reported in this paper, which led to low levels of engagement by students, therefore wider generalisation of the findings cannot be made.

The development of pedagogic research around the impact of RLO does not come without challenges. To date, these have included staff time to design and develop RLO, time to develop a quantitative post-test as well as the low engagement levels of students with focus groups (Johnson et al., 2013). We do have to consider that some veterinary education cannot be delivered virtually (Short, 2002). There is a sense from the literature that academic staff may be replaced by e-learning technologists in the future, which could then subsequently have an impact on the research generating activities of a university (Short, 2002), where there is a significant shift in the proportion of online learning in a blended learning format. However, without a cross-institutional team of technical staff to support the provision of e-learning, the only academic staff to develop and use RLO in their teaching would be those with prior experience of doing so (Kirkwood, 2008). These challenges are not insurmountable and the barriers to implementing RLO should be addressed, in light of the range of potential benefits to students from actively engaging with RLO (Blake, 2010).

CONCLUSION

The literature reviewed herein provides an insight into an aspect of blended learning which is currently limited in terms of published research. Increasing this documentation of impact is critical in order to advance the development and promotion of RLO in blended learning. The effect of computer confidence on RLO use, students’ choice to use RLO, the impact on student learning, assessment and professional performance are some of the variables which have been measured to date. To improve the student approach to learning, universities should continue to invest in blended learning and RLO, along with sufficient staff and student development in order for those resources to have optimal impact. Learners’ needs for development of digital literacies is also a reflection of the technological expectations of graduates and their role in society. The approach to RLO by students from different courses may vary, but prior experience of technology, alignment of content and learning needs and availability of technical support are some of the key drivers of usage and reuse. A positive effect on student attributes beyond just assessment performance is likely to occur following RLO use because those students have adopted a process of active engagement, which we know can bring about a deeper approach to learning. In turn, these learners perform better than superficial learners. As educators, ways of motivating students to actively engage with materials are illustrated in cases where they have been dynamic in the development of resources.

Overall, whether a student in veterinary education will choose to use blended learning and RLO and whether that usage brings about a positive impact on performance as a whole appears to be multifactorial, as evidenced by the range of subject areas in which empirical studies have been undertaken. Therefore, practitioners within the veterinary education sector must draw from findings from other subject areas, albeit with care, in order to develop the application of RLO within their teaching.

This research was previously published in the International Journal of Mobile and Blended Learning (IJMBL), 9(3); edited by David Parsons, pages 24-36, copyright year 2017 by IGI Publishing (an imprint of IGI Global).

REFERENCES

Association of American Medical Colleges (AAMC). (2007). Effective Use of Educational Technology in Medical Education: Colloquium on Educational Technology, Recommendations and Guidelines for Medical Educators .

Blake, H. (2010). Computer-based learning objects in healthcare: The student experience . International Journal of Nursing Education Scholarship , 7(1), 1. doi:10.2202/1548-923X.1939

Boyle, E. A., Duffy, T., & Dunleavy, K. (2003). Learning styles and academic outcome: The validity and utility of Vermunts Inventory of Learning Styles in a British HE setting . The British Journal of Educational Psychology , 73(2), 267–290. doi:10.1348/00070990360626976

Clements, and Pawlowski. (2012). User-oriented quality for OER: understanding teachers’ views on re-use, quality, and trust. Journal of Computer Assisted Learning, 28(1), 4-14. Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/j.1365-2729.2011.00450.x/full

Concannon, F., Flynn, A., & Campbell, M. (2005). What campus-based students think about the quality and benefits of e-learning. British Journal of Educational Technology, 36(3), 501-512. Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/j.1467-8535.2005.00482.x/abstract

UK Government Department for Health (DoH). (2011). A Framework for Technology Enhanced Learning Department for Health. Retrieved from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/146989/dh_131061.pdf.pdf

Evans, C. (2013). Development and evaluation of ‘reusable learning objects’ (RLOs) to enhance the learning experience of international healthcare students. The Higher Education Academy. Retrieved from http://www.jisctechdis.ac.uk/assets/documents/internationalisation/connections/Nottingham_Evans_Connections_Final_Report.pdf

Higher Education Academy (HEA). (2012). A marked improvement: Transforming assessment in HE .

Hoover, T. S., & Marshall, T. T. (1998). A comparison of learning styles and demographic characteristics of students enrolled in selected animal science courses . Journal of Animal Science , 76(12), 3169–3178. Retrieved from http://www.journalofanimalscience.org/content/76/12/3169 doi:10.2527/1998.76123169x

Jenkinson, J. (2009). Measuring the effectiveness of educational technology: what are we attempting to measure? Electronic Journal of e-Learning, 7(3). Retrieved from www.ejel.org/issue/download.html?idArticle=106

Johnson, I. P., Palmer, E., Burton, J., & Brockhouse, M. (2013). Online Learning Resources In Anatomy: What Do Students Think? Clinical Anatomy (New York, N.Y.) , 26(5), 556–563. Retrieved from http://onlinelibrary.wiley.com/doi/10.1002/ca.22219/full doi:10.1002/ca.22219

Kay, R.H. and Knaack, L. (2007). Evaluating the learning in learning objects Open Learning 22:1 pp5-28

Keefe, G., & Wharrad, H. (2012). Using e-learning to enhance nursing students pain management education . Nurse Education Today , 32(8), e66–e72. Retrieved from http://www.sciencedirect.com/science/article/pii/S0260691712001001 doi:10.1016/j.nedt.2012.03.018

Kirkwood, A. (2008). Getting it from the Web: why and how online resources are used by independent undergraduate learners. Journal of Computer Assisted Learning, 24(5), 372-382. Retrieved from http://oro.open.ac.uk/10071

Kurilovas, E., Bireniene, V., & Serikoviene, S. (2011). Methodology for evaluating quality and reusability of learning objects. The Electronic Journal of e-Learning, 9(1). Retrieved from www.ejel.org/issue/download.html?idArticle=166

Kurilovas, E., Serikoviene, S., & Vuorikari, R. (2014). Expert centred vs learner centred approach for evaluating quality and reusability of learning objects. Computers in Human Behaviour, 30. Retrieved from http://www.sciencedirect.com/science/article/pii/S074756321300277X

Littlejohn, A., Margaryan, A., & Vojt, G. (2010). Exploring Students’ use of ICT and Expectations of Learning Methods. Electronic Journal of E-learning, 8(1), 13-20. Retrieved from www.ejel.org

Lymn, J. S., Bath-Hextall, F., & Wharrad, H. J. (2008). Pharmacology education for nurse prescribing students – a lesson in reusable learning objects . BMC Nursing , 7(1), 2. Retrieved from http://www.biomedcentral.com/content/pdf/1472-6955-7-2.pdf doi:10.1186/1472-6955-7-2

Marcus, J. (2013). Behind the wall: A class that’s disconnected and out of reach Times . Higher Education , 15(August), 16–17.

Mayes, T., Morrison, D., Mellar, H., Bullen, P., & Oliver, M. (Eds.). (2009). Transforming HE Through Technology-Enhanced Learning. The Higher Education Academy. Retrieved from www.heacademy.ac.uk/assets/documents/learningandtech/Transforming.pdf

Petrie, A., & Watson, P. (2006). Statistics for Veterinary and Animal Science (2nd ed.). Oxford: Blackwell Publishing.

Plendl, J., Bahramsoltani, M., Gemeinhardt, O., Hunigen, H., Kasmeyer, S., & Janczyk, P. (2009). Active Participation Instead of Passive Behaviour Opens Up New Vistas in Education of Veterinary Anatomy and Histology . Anatomia, Histologia, Embryologia , 38(5), 355–360. doi:10.1111/j.1439-0264.2009.00954.x

Reusable Learning Objects Centre for Excellence in Teaching and Learning (RLO-CETL). (2005). CETL reusable learning objects evaluation. Retrieved from http://www.rlo-cetl.ac.uk/index.php

Roshier, A. L., Foster, N., & Jones, M. A. (2011). Veterinary students usage and perception of video teaching resources . BMC Medical Education , 11(1), 1. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3025976/pdf/1472-6920-11-1.pdf doi:10.1186/1472-6920-11-1

Sandlin, M.R., Murphey, T.P., Lindner, J.R. and Dooley, K.E. (2014). An examination of the use of reusable learning objects to alter agricultural students’ attitudes and opinions regarding international settings NACTA Journal, 58(1).

Short, N. (2002). The Use of information and communication technology in veterinary education. Research in Veterinary Science , 72(1), 1–6. doi:10.1053/rvsc.2001.0531

Trowler, V. (2010). Student Engagement Literature Review. The Higher Education Academy Retrieved from http://www.heacademy.ac.uk/assets/documents/studentengagement/StudentEngagementLiteratureReview.pdf

Vermunt, J. D., & Vermetten, Y. J. (2004). Patterns in student learning: Relationships between learning strategies, conceptions of learning and learning orientations . Educational Psychology Review , 16(4), 4. doi:10.1007/s10648-004-0005-y

Windle, R. J., McCormick, D., Dandrea, J., & Wharrad, H. (2011). The characteristics of reusable learning objects that enhance learning: A case-study in health science education . British Journal of Educational Technology , 42(5), 811–823. doi:10.1111/j.1467-8535.2010.01108.x

Windle, R. J., Wharrad, H., McCormick, D., Laverty, H., & Taylor, M. (2010). Sharing and reuse in OER: experiences gained from open reusable learning objects in health. Journal of Interactive Media in Education. Retrieved from http://jime.open.ac.uk/2010/04

EBSCOhost – printed on 5/22/2019 10:18 AM via TEXAS STATE UNIV. All use subject to https://www.ebsco.com/terms-of-use

Only gold members can continue reading. Log In or Register to continue

Share this:

• Click to share on Twitter (Opens in new window)
• Click to share on Facebook (Opens in new window)

Related

Related posts:

1. An Up-To-Date Review of Piglet Isosporosis
2. Role of Dairy Farming in Rural Development
3. Understanding Strategies for Implementing Integrated Information Systems for Rabies Surveillance
4. Intelligent Wildlife Tracking Using Ubiquitous Technological Suite

Stay updated, free articles. Join our Telegram channel

Join