Overview

Online learning environments were initially developed as a means to provide regular social interaction for distance learners. However, as technology has become increasingly present in students’ day-to-day lives, such environments have been integrated into mainstream education. Yet there is general controversy surrounding the use of ‘social’ technologies (such as Facebook) in education, and uncertainty about students’ preferences in this area.

Learnium is a social network for learning and teaching, designed to provide a balance between the use of institutionally-mandated virtual learning environments (such as Moodle) and students’ use of social networks (such as Facebook). Coventry University began using Learnium in 2013, and Bernie Davies (Health and Life Sciences faculty) undertook an early evaluation of a number of courses using this technology. In September 2014, the Disruptive Media Learning Lab was asked to support the use of Learnium across a number of modules and evaluate its effectiveness as a tool for learning and teaching. At this time, Learnium was being used in a number of areas. These were:

• Inter-Professional Healthcare Learning
• Forensic Investigations
• Cybersecurity
• Business
• Education

The Disruptive Media Learning Lab evaluation has been separated into two phases. From September 2014 - September 2015, an evaluation was undertaken based upon data from one Inter-Professional Learning module in healthcare and one Postgraduate Certificate in Education module. This evaluation focused upon student engagement of using Learnium and student experiences with Learnium, in order to guide improvements to the website and inform future teaching using Learnium. The findings of this evaluation are discussed briefly below, and full details are contained in the final report.

Whilst Learnium was perceived to have potential as an engaging environment by many participants, technical problems (such as accessibility and volume of email notifications received) played a significant part in shaping participant perceptions of the environment in Phase 1. Based on the suggestions in this report the Learnium team released several improvements before the beginning of Phase 2 (and is continuously enhancing accessibility and the overall user experience):

• Single-Sign-On with Coventry’s Shibboleth system
• Option to aggregate email notification in one daily digest
• The release of iPhone and Android apps
• Ability to delete your own posts & comments
• Ability to upload images
• Ability to see a list of files in chat
• Various layout & navigation improvements
• Other bug fixes

The company also wrote a post detailing how it deals with development based on user feedback available here.

The second phase of the evaluation commenced in January 2016. In this phase, the Disruptive Media Learning Lab will focus upon pedagogical design for and in Learnium, across a number of courses in the University. We are currently seeking interested staff members at Coventry University to participate in this evaluation, so please contact Jacqui Speculand if you are interested.

Research Objectives

• Phase 1: The desired outcomes from phase 1 of this study were:
  1. An empirically informed understanding of students’ engagement with the Learnium platform
  2. An understanding of students’ experiences using Learnium 
• Phase 2: The desired outcomes from phase 2 of this study are:
  1. A continued understanding of students’ experiences using Learnium
  2. An understanding of the types of pedagogical approaches employed whilst using Learnium
  3. Recommendations for pedagogical practice whilst using Learnium

Methodology & Methods

The purpose of phase 1 of this research project was to evaluate students’ engagement with and experiences of Learnium as a learning technology. It utilized Stake’s (1983) responsive evaluation methodology, in which attention is paid to the priorities of stakeholders involved in the project. This mixed-method evaluation was undertaken by Dr Gemma Tombs of the Disruptive Media Learning Lab. Ethical approval was sought from Coventry University Ethics Committee (Ref. P32845), and received in October 2014. It utilised a number of methods, which were:

• The validated User Engagement Scale (UES) (O’Brien & Toms, 2010), delivered 127 students on an Inter-Professional Learning healthcare module
• A custom-developed Student Experience Survey (SES) exploring students’ learning experiences in Learnium, delivered to the same students who received the UES
• Learning analytics data from Learnium, from the 127 students responding to the UES and the SES
• Reflective assessment pieces on the use of Learnium, from 55 students on the Postgraduate Certificate module

Milestones

1. October 2015

   Completion of Phase 1

   Learnium Evaluation Report

   Format: PDF

   Download Report

1. January 2016

   First review point for Phase 2

2. February 2016

   First interim report for Phase 2 due

Blogs

• Learnium and Office Online tested out at Coventry’s Disruptive Media Learning Lab
• Charlie Legge Work Experience Blog - Learnium Evaluation Support
• Charlie Legge Work Experience Blog - Learnium User Guide Development

Research Findings

Engagement in the first phase of this study was measured through the User Engagement Scale (O’Brien & Toms, 2010b), which has been applied in a variety of online settings: shopping websites (O’Brien & Toms, 2013); social media (Banhawi, Ali, & Judi, 2012); and digital gaming (Wiebe, Lamb, Hardy, & Sharek, 2014). The scale measured engagement through a 31-item 5-point Likert scale with six sub-scales (measured through perceived usability, focused attention, aesthetic appeal, novelty, felt involvement, and endurability).

The UES (with 127 students on an Inter-Professional healthcare module) revealed that novelty, endurability, and felt involvement loaded onto one factor. This implies that perceptions of Learnium as worthwhile, fun, and stimulating, were perceived along the same dimension. The first phase also revealed that there was only a weak correlation between the learning analytics data of total posts on Learnium, and students’ engagement with Learnium.

Research Implications

The findings from phase 1 therefore imply that engagement with Learnium did not necessarily play an important role in determining post counts. This finding is particularly interesting; it means that future work should examine possible intrinsic motivating factors for engagement, and identify possible other influences shaping student use of Learnium.

Qualitative findings also indicated that pedagogical design, and the engagement of other students, were also influential in shaping student perceptions of the environment. Consequently, phase 2 will focus particularly upon the design of pedagogy for Learnium and Learnium’s use alongside multiple other online and offline learning environments.

Updates

• 24th March 2016: Learnium Lunch! Jacqui, Gemma and Charlie had an extremely successful ‘picnic lunch’ on the DMLL grass, discussing how people are currently using Learnium and how best we can evaluate its use across campus. We’ve now reached the first review point of phase 2 and things are progressing well. We hope to begin data collection by mid April ready for a final report by the end of May.

• February 2016: Charlie, placement student, is now working on the evaluation with Gemma and is preparing a user guide and identifying any issues with the platform. Gemma and Jacqui are contacting staff using Learnium to discuss progress, identify issues and support use.

Project Timeline:

Associated Faculties, Schools and/or Course(s):

• Faculty of Arts and Humanities
• Faculty of Engineering, Environment and Computing
• Faculty of Health and Life Sciences
• Faculty of Business and Law