Learning Analytics|1062_E-Learning_Group 2

Learning Analytics

Learning Analytics Overview in Higher Education

Overview

The following contents are quoted from "Learning Analytics Methods, Benefits, and Challenges in Higher Education: A Systematic Literature Review" by John T. Avella, Mansureh Kebritchi, Sandra G. Nunn, Therese Kanai of University of Phoenix in Online Learning–Volume 20 Issue 2  on June 2016.

A systemic literature review was conducted to provide an overview of methods, benefits, and challenges of using Learning Analytics (LA) in higher education

The literature review revealed that LA uses various methods including visual data analysis techniques, social network analysis,semantic, and educational data mining including prediction, clustering, relationship mining, discovery with models, and separation of data for human judgment to analyze data.

 The benefits include targeted course offerings, curriculum development, student learning outcomes, behavior and process, personalized learning, improved instructor performance, post-educational employment opportunities, and enhanced research in the field of education.

Challenges include issues related to data tracking, collection,evaluation, analysis; lack of connection to learning sciences; optimizing learning environments, and ethical and privacy issues.

Such a comprehensive overview provides an integrative report for faculty,course developers, and administrators about methods, benefits, and challenges of LA so that they may apply LA more effectively to improve teaching and learning in higher education.

Introduction

The advancement of technology has provided the opportunity to track and store students’ learning activities as big data sets within online environments. Big data refers to the capability of storing large quantities of data over an extended period and down to particular transactions (Picciano, 2012). Users can take big data from different sources to include learning management systems (e.g., Blackboard), open source platforms (e.g., Moodle), open social platforms (e.g., LinkedIn), and different web tools such Meerkat-Ed and Snapp (Reyes, 2015).

Similar to decision making driven by data, analytics refers to the scientific process that examines data to formulate conclusions and to present paths to make decisions (Picciano, 2012). According to Brown (2012), the process of systematically collecting and analyzing large data sets from online sources for the purpose of improving learning processes is called learning analytics (LA). LA is an emerging field in education. 

Experts in online learning in American higher education predict that within the next few years learning analytics will be widely used in online education to identify students’ pattern of behaviors and to improve students’ learning and retention rates.

Learning analytics, educational data mining, and academic analytics are closely related concepts (Bienkowski, Feng, & Means, 2012; Elias, 2011).

Educational data mining

1. focuses on developing and implementing methods with a goal of promoting discoveries from data in educational settings.

2. examines patterns in a large data set related to students’ actions.

3. be utilized to form abetter understanding of the educational settings and learners.

Academic analytics

1. refers to an application of the principles and tools of business intelligence to academia with the goal of improving educational institutions’ decision-making and performance (Campbell, De Blois, & Oblinger, 2007).

2. combines “large data sets,statistical techniques, and predictive modeling” (Campbell et al., 2007, p. 42).

Learning analytics

1. uses predictive models that provide actionable information.

2. be a multi-disciplinary approach based on data processing, technology-learning enhancement, educational datamining, and visualization (Scheffel, Drachsler, Stoyanov, & Specht, 2014).

3. The purpose is to tailor educational opportunities to the individual learner’s need and ability through actions such as intervening with students at risk or providing feedback and instructional content.

Learning Analytics (LA) increases awareness of learners and educators in their current situations that can help them make constructive decisions and more effectively perform their tasks (Scheffel et al., 2014). One of the main applications of learning analytics is tracking and predicting learners’ performance as well as identifying potential problematic issues and students at risk (EDUCAUSE, 2010; Johnson, Smith, Willis, Levine, & Haywood, 2011).

Method

To address the research problem, researchers conducted a literature review using the procedure suggested by Cooper (1988) for synthesizing the literature. This systematic procedure helped to 

 (a) formulate the problem, 

 (b) collect data, 

 (c) evaluate the appropriateness of the data, 

 (d) analyze and interpret relevant data, and 

 (e) organize and present the results. 

Then results were compared with current issues in a large higher education institution.

Formulating the problem

The problem is that embracing LA in evaluating data in higher education diverts educators’ attention from clearly identifying methods, benefits, and challenges of using LA in higher education.

To help solve the problem, the following questions guided this review:

 1. What are the methods for conducting learning analytics in education?

 2. What are the benefits of using learning analytics in education?

 3. What are the challenges of using learning analytics in education?

Data collection

The purpose of data collection was to find empirical studies including quantitative, qualitative, mixed methods, and literature reviews published in peer-reviewed journals since 2000 to identify methods, challenges, and benefits of LA in higher education.

Data evaluation and analysis

Based on the described procedure, 112 articles were found. Of these, 10 focused on issues related to learning analytics methods, 16 on benefits, and 18 focused on challenges.

What are the applications of learning analytics?

  • tracking learners’ performance
  • identifying students at risk
  • predicting learners’ performance
  • identifying potential problematic issues

Language learning and learning analytics

Introduction

Learning analytics is understood as ‘ the measurement, collection, analysis, and reporting of data about learners in their context, for purposes of understanding and optimizing learning and the environment in which it occurs  (Siemens et al., 2011, n.p.). 

Over the last five years learning analytics has seen an increasing amount of educational research investigate its potential for tracking learners’  online and offline activities, and visualizing and analyzing their interactions, either with each other, their instructors, and/or with a variety of digital applications. 

Although research in the past tended to focus on self-reported data from students, typically focused on their perceptions of autonomous learning behavior, the development of large-scale online learning environments, has meant that vast amounts of data can now be collected.

Student retention and success

 One of the potential opportunities presented by learning analytics is that teachers may acquire valuable data on learners who are succeeding and failing and thereby gain insights into the factors which influence course withdrawal or student retention

Faced with such information, teachers and student advisors may be able to intervene and advise students on their further learning paths. Typically, the data are based on the combination of clicks in a virtual learning environment with examination or test results and this does present limitations in that it may inform us about what happened rather than why. 

Nevertheless, it is clear that data of this kind may provide some valuable clues in relation to 

1: retention and success

2: patterns of learning habits

3: uses of specific course components 

4: enable course developers to improve course content and teachers 

5: identify struggling students and design pedagogical interventions.

From the learner perspective access to relevant data in the form of dashboards may enable them to reflect on their achievements and patterns of behavior in relation to others in their cohort. Dashboards may allow students to auto-evaluate their progression and learning behavior and to compare their profile to user patterns of their peers and advised usage of the material by instructors. 

Examples of charts to be included in dashboards could be (individual and group) activity timelines, learner preferences spider charts and educational content heat maps

Extracts from learner dashboards may also be used as indicators of achievement in online student portfolios. 

Educators may thus receive dynamic and real-time overviews of how their students are progressing, which students might be at risk of dropping out or of failing a course and which parts of the courses cause difficulties and/or require more feedback. 

It is no surprise that research on learning analytics should be of value to teachers, course and materials designers, administrators and students themselves, as a variety of stakeholders share an interest in how to optimize the learning process and to understand how learning takes place online.

Effective teaching and learning

 In Affordances and limitations of learning analytics for computer-assisted language learning: A case study of the VITAL project’, Gelan et al focus on how learning analytics can provide valuable insights into task design for instructors and materials designers, as well as help students with effective learning strategies and personalized learning pathways.

The paper is notable for situating analytics within the wider context of research on data-driven learning in computer-assisted language learning, prior to examining findings arising from the EU funded VITAL project, as mentioned above, with data from 285 undergraduate students on a Business French course at a university in Belgium.

Based on a flipped classroom design and using an innovative approach to collecting data in the xAPI format, process-mining tools and data visualization in the form of instructor and learner dashboards, enabled researchers to identify significant differences between successful and non-successful students’  learner patterns.

 

Building on this work on data mining, in ‘ Understanding online interaction in language MOOCs through learning analytics’, Martı n-Monje, Castrillo and Ma~ nana- Rodrıguez explored how the approach can help researchers to understand which learning objects students use, how successful students interact, and to identify examples of successful learner profiles. 

Findings highlighted the importance of video, automated grading activities, and the most prominent types of learner profile. They demonstrate that the data mining approach can help designers with respect to audio-visual content, and to produce content that leads to more well-rounded learners who do not merely view content, but actively engage in problem-solving.

The importance of analytics for learning design

 The importance of analytics for learning design is also evident in Analytics in online and offline language learning environments: The role of learning design to understand student online engagement’  in which Rienties et al sought to combine key principles of learning design, the use of big data and learning analytics to guide the process of course development. The study drew on a student activity based taxonomy adopted by the

Open University in the UK for the learning design and collected data from 2,111 learners to investigate how learning design decisions made by language instructors affected the way students engage with other students and activities in a virtual learning environment.

Based on a quantitative analysis involving fixed effects models, it was evident that the way instructors designed weekly activities had a significant impact on online engagement. 

As a result, the study underlines the importance of big data to teachers and designers when attempting to understand how learners interact online. 

The integration of face-to-face and online instruction is the focus of Rubio, Thomas, and Li’ s study on ‘ The role of teaching presence and student participation in Spanish blended courses, which investigates how both approaches can work alongside one another in a mutually beneficial way. Using virtual learning environment data relating to student participation in the online part of the course, the researchers focus on the importance of active participation, passive participation and continuity. Findings indicate that profiling of online learners can help to identify less successful students and that there appears to be a correlation between lower levels of online participation and engagement and final grades in the course. The analytics were, therefore, able to help course designers and instructors in terms of course design and by identifying potentially at-risk learners.

The final contribution to the special edition, ‘ Statistically-driven visualizations of student interactions with a French online course video’  by Youngs, Prakash and Nugent explores the use of logged data from an online French course. While normally this data is not available to instructors or students, the paper presents a quantitative visualization arising from students’  interactions with a learning video and supplementary questions. The paper explores the potential of this type of approach as a way of delivering an in-depth understanding of learner behavior.

Which is not true? Analytics provide some valuable information for...

  • 1: retention and success
  • 2: patterns of learning habits,
  • 3: uses of specific course components
  • 4: WHY success of failure happened

Learning Analytics Methods in Education

Learning analytics process

With the current advent of both blended and online learning opportunities, big data and learning analytics are predicted to play a significant role in education in future years. When discussing learning analytics methods in education, it is important to provide a background regarding the flow of analytical information.

Campbell and Oblinger (2007) proposed five stages of 

 (1) capturing data, 

 (2) reporting the data pattern and trends, 

 (3) predicting a model based on the data by using statistical regression, 

 (4) acting by using an intervention based on the model to improve learning, and 

 (5) refining the developed model.

Similarly, a learning analytics cycle was suggested by Clow (2012, 2013) in which 

 (1) researchers collect data from the learners, 

 (2) process the data into metrics, and 

 (3) use the results to perform an intervention that affects the students.

Learning analytics analysis

Learning analytics focuses on data related to learners’ interactions with course content, other students, and instructors.

Data visualization tools and techniques

Visual data analysis includes highly advanced computational methods and graphics to expose patterns and trends in large, complex datasets (Johnson,Levine, Smith, & Stone, 2010). One of the standard techniques is visual interactive principal components analysis; it can be used to reduce many variables into a few by finding elements within datasets.

Additionally, learning analytics uses educational data mining methods to analyze large datasets.Within educational data mining, researchers currently use a variety of popular methods. Classified into five categories, these methods consist of 

 (1) prediction

 (2) clustering

 (3) relationship mining

 (4) discovery with models, and 

 (5) separation of data for use in the process of human judgment 

(Baker, 2010; Baker & Yacef,2009; Romero & Ventura, 2010).

Predication

Predication involves developing a model that uses both a predicted variable and predicator variables. A predicted variable represents a particular component of the data, whereas predicator variables consist of a combination of other data elements. Researchers classify predication into three categories known as classification, regression, and density estimation.

Clustering

Clustering entails the discovery of a set of data points that form a logical group together. Therefore, observation reveals the resultant formation of some clusters from the full dataset. By dividing a collection of data into logical clusters,researchers can assess how cluster sets explain the meaning of the data.

Relationship mining

The method of relationship mining focuses on the goal of discovering relationships between variables in a set comprised of a large number of variables. Forms of relationship mining may include learning which variables are related to a single variable or discovering what is the strongest relationship between two variables.

Discovery with models

The goal is to develop a model using one of the following methods: predication, clustering, or knowledge engineering.Knowledge engineering uses human reasoning for model development.

Separation of data for use in the process of human judgment

Researchers classify the separation of data for use in the process of human judgment method as a visualization method, in which educational data have a particular structure and meaning rooted within that structure. This method possesses two distinct goals identification and classification.

What are the categories when learning analytics uses educational data mining methods to analyze large datasets?

  • Clustering
  • Discovery with models
  • Relationship mining
  • Predication
  • Separation of data for use in the process of human judgment

Learning Analytics Benefits in Education

Identifying target courses

An initial benefit that evolves from using big data analysis in education is the ability of educational institutions to identifytargeted courses that more closely align with student needs and preferences for their program of study.  By examining trends in student enrollment and interests in various disciplines, institutions can focus educational and teaching resources in programs that maximize student enrollment in the most needed areas of study.  Schools can better predict graduate numbers for long-term planning of enrollment (Althubaiti & Alkhazim, 2014).

Curriculum improvement

Using big data allows instructors to make changes and adjustments to improve curriculum development in the educational system, such as in the use of curricular mapping of data (Armayor & Leonard, 2010).  Through the analysis of big data, educators can determine weaknesses in student learning and comprehension to determine whether or not improvements to the curriculum may prove necessary.  Instructors can engage in educational strategic planning to ensure that the learning curriculum targets student needs to maximize learning potential.

Student learning outcome, behavior, and process

 Another key benefit of big data and text mining focuses on the ability of schools and instructors to determine student learning outcomes in the Online Learning - Volume 20 Issue 2 - June 2016 19educational process as well as determine how to improve student performance (Bhardwaj & Pal, 2011).  Researchers noted that the use of educational data mining contributed to positive results in the learning process (AlShammari, Aldhafiri, & Al-Shammari, 2013).  Analysis of the data can help educators understand the student learning experience through learner interactions with technology tools such as elearning and mobile learning (Hung & Zhang, 2012).  Use of big data also reveals learning behavior, the impact on adaptive learning, and level of persistence (DiCerbo, 2014) in the learning process.  By understanding the effectson learner outcomes, use of this data also reveals how to make improvements in student learning and performance in academic coursework.  Therefore, LA allows instructors to evaluate forms of knowledge and adjust educational content accordingly.

Personalized learning

Arnold and Pistilli (2012) discussed an early intervention system that demonstrates the benefits and power of learning analytics.  As an example, Course Signal provides students with real-time feedback.  The components of students’ grades, demographic characteristics, academic background, and demonstrated effort are all addressed. The system employs a personalized email and a stoplight, specific color method to indicate progress or lack thereof.  Using learning analytics, the concept of personalized learning reveals student success.  Dietz-Uhler and Hurn (2013) asserted that course designers do not account for students who do not begin specific coursework at the same learning stage and who do not proceed, learn, and master course competencies at the same pace.  Learning analytics allows faculty to use data collected by the learning management system to observe the frequency of student login.  Instructors can also see student interaction within the course, total engagement, pace, and grades.  These components serve as predictors of students’ potential success or failure.  Learning analytics allows for real-time reception of the pertinent data, review as well as the incorporation of data, and real-time feedbackfor every student.

Improved instructor performance

Using this data also helps to assess instructor performance (Mardikyan & Badur, 2011).  The use of data provides an opportunity to improve instructor development so that instructorsare better prepared to work with students in a technological learning environment. Through the acquisition of data generated from instructor usage of technology and research tools in online libraries (Xu & Recker, 2012), analysts can determine online behaviors by educators.  Therefore, use of this information can help identify areas in need of improvement by the instructor to facilitate enhanced instructor-student interactions in the educational environment.

Post-educational employment

Using big data allows educational institutions to identify posteducation employment opportunities for graduates and help target education that more closely aligns with employment market needs.  It can also predict graduate employment, unemployment, or undetermined situations about job opportunities (Jantawan & Tsai, 2013).  Using big data can help stakeholders in the educational system better understand vocational prospects for students and better assess student learning programs for occupational compatibility (Kostoglou, Vassilakopoulos, & Koilias, 2013).  In a global learning environment, this type of information not only can facilitate better educational and posteducation vocational planning, but also may prove useful to organizations as they make hiring and budgeting decisions for college graduates in different disciplines.

Learning analytics practitioners and research community

The research community also benefits from the use of big data in education.  Researchers can more easily share information and collaborate.  They can identify gaps between industry and academia so that research can determine how to overcome problems.  Also, useful data analysis represents an important component of the ability of scholars to generate knowledge as well as continue to progress in research disciplines (Sharda, Adomako, Asamoah, & Ponna, 2013).  However, these benefits are also offset by the need for trained personnel who can use and apply analytics appropriately.  Current researchers note a looming future gap in practitioners possessing requisite analytical skill sets in the area of business intelligence and analytics.  Picciano (2012) noted a lack of sufficiently trained database administrators and designers to address present needs.  This Online Learning - Volume 20 Issue 2 - June 2016 20issue has become an important focus for academic and business researchers seeking to overcome this problem through improved education in this area (Hsinchun, Chiang, & Storey, 2012).

Learning Analytics Challenges in Education

Data tracking

The digital tracking of information is a technique used by analysts to determine how best to present new learning opportunities as the wave of education continues to move forward into the second decade of the 21st Century.  The tracking of big data represents the monitoring system. Current trend tracking indicators regarding the delivery and dissemination of instruction depend on the learning management system used by the institution.  Platforms such as Moodle, Canvas, EPIC, and Blackboard have the capability to track the number of times an individual logs into the course room. These platforms also provide significant documentation to determine how involved the student was upon their login.  Such tracking provides those who plan and implement new educational programs with valuable information.  The monitoring reveals how engaging the curriculum presented is, as well as identifyingareas that cause confusion (Brown, 2012).

Data collection

 The collection of data can be a challenge when looking at LA.  Nonetheless, it represents an important component in planning for continued implementation of educational program growth (Bottles, Begoli, & Worley, 2014).  Educators must consider several elements.  They must consider the availability of resources at a venue.  Next, instructors must establish a viable social platform as it directly relates to interactions between learners to synthesize the educational content.  Finally, instructors must discriminate whether the learner population possesses the requisite suitability for this type of learning environment and knowledge acquisition.  Besides these challenges, gaps exist because of the inability to share proprietary information gathered by the institution.  Further, another problem emerges because the creation of the ideal framework to disseminate educational curriculum takes teamwork, especially among the organizations bidding against one another to capture the learner population who want to engage in this type of learning experience.

Evaluation process

An important consideration of data collection concerns how learning analytics has become a force in the evaluation process.  As greater amounts of educational resources become available online, there is a subsequent increase in the total data available regarding learning interactions.  For learning analytics to help instructor evaluation to function appropriately, data needs to be delivered in a timely and accurate manner (Picciano, 2014). Learning analytics can provide powerful Online Learning - Volume 20 Issue 2 - June 2016 21tools for developing meaning from interactions and actions within a higher education learning environment (Fournier, Kop, & Sitlia, 2011).  With the unprecedented explosion of available data for online interactions, it is critical for the continued development of the evaluation process.  LAcan translate from other fields as interest in the data growth in education becomes more focused.  Lias and Elias (2011) noted that statistical evaluation of rich data sources already exists within otherprofessions and fields.

Data analysis

An important consideration of data collection concerns how learning analytics has become a force in the evaluation process.  As greater amounts of educational resources become available online, there is a subsequent increase in the total data available regarding learning interactions.  For learning analytics to help instructor evaluation to function appropriately, data needs to be delivered in a timely and accurate manner (Picciano, 2014). Learning analytics can provide powerful Online Learning - Volume 20 Issue 2 - June 2016 21tools for developing meaning from interactions and actions within a higher education learning environment (Fournier, Kop, & Sitlia, 2011).  With the unprecedented explosion of available data for online interactions, it is critical for the continued development of the evaluation process.  LAcan translate from other fields as interest in the data growth in education becomes more focused.  Lias and Elias (2011) noted that statistical evaluation of rich data sources already exists within otherprofessions and fields.

Learning sciences connection

According to Pea (2014), personalized learning and learning opportunities demonstrate an inability to leverage learning analyticsoptimally; therefore, “the endgame is personalized Cyberlearning at scale for everyone on the planet for any knowledge domain” (p. 17). Ferguson (2012) asserted that to optimize and fully understand learning requires understanding how knowledge develops and how to support knowledge development.  Further, researchers must understand the components of identity, reputation, and affect.  Researchers must find ways to connect “cognition, metacognition, and pedagogy” (Vahdat et al., 2015, p. 299) to help improve learning processes.  With a stronger connection to learning sciences,learning analytics can promote effective learning design.

Learning environment optimization

Ferguson (2012) noted that as learners expand the boundaries of the learning management system into open or blended learning settings, researchers must discover the problems faced by students and how to determine success from the learners’ perspectives. This process will encumber a shift toward more challenging datasets that may include mobile, biometric, and mood data.  Besides the individual learning aspect of learning analytics, researchers are seeking to address another component known as social learning analytics.  In this context, social learning analytics focuses on the collaboration and interaction of learners in a socialized learning environment, not just on individual learning outcomes (Buckingham Shum & Ferguson, 2012).

Emerging technology

The full potential of learning analytics relating to learning requires continued and emerging technology that presently remains in the younger stages.  Thisrevelationpresents a challenge as the technology continues to develop to stay constant with the growth of learning analytics. Further, to fully understand the method and practice of teaching, more research is needed.  Research focusing on learning analytics and pedagogy is still in the beginning stages (Dyckhoff, Zielke, Bültmann, Chatti, & Schroeder, 2012)

Ethical and privacy issues

 Another issue that emerges about learning analytics concerns the ethical, legal, and risk considerations (Kay, Korn, & Oppenheimer, 2012).  Because of dynamic changes in technology as well as how users store data and applications in cloud-based systems, “the challenges of privacy and control continue to affect adoption and deployment” (Johnsonet al., 2011, p. 3).  Further, the ethical and legal complexities of learning analytics challenge institutions that seek to implement their usage (Sclater, 2014a).  For example, these considerations can include obvious areas of privacy considerations such as consent, data accuracy, how to respect privacy, maintaining anonymity, opting out Online Learning - Volume 20 Issue 2 - June 2016  22of data gathering, and the potential effects to students.  Additional concerns include data interpretation, data ownership, data preservation, sharing data with parties outside of the institution, and proper training of staff members regarding the handling of data (Sclater, 2014b).  Further, the question becomes who owns this aggregate data, because having an infrastructure with the capacity to house large amounts of information becomes a daunting task (West, 2012).  Because of these different issues, institutions must achieve a balanced approach to safeguard data while also assuring benefits to the educational process through the use of four guiding principles.  These principles consist of clear communication, care, proper consent, and complaint (Kay et al., 2012).  Institutions must demonstrate adherence to legal and ethical parameters to safeguard student privacy concerns while also achieving the educational goals for students and educators.

Congratulations! You have finished this course!

Finish

Congratulations! You have finished this course!

See you in the next course!