A rubric for evaluating e-learning tools in higher education — from er.educause.edu by Lauren Anstey and Gavan Watson
The Rubric for E-Learning Tool Evaluation offers educators a framework, with criteria and levels of achievement, to assess the suitability of an e-learning tool for their learners’ needs and for their own learning outcomes and classroom context.

Excerpt:

We organized our rubric’s evaluation criteria into eight categories (functionality; accessibility; technical; mobile design; privacy, data protection, and rights; social presence; teaching presence; cognitive presence). Each category has a specific set of characteristics, or criteria, against which e-learning tools are evaluated, and each criterion is assessed against three standards: works well, minor concerns, or serious concerns. Finally, the rubric offers individual descriptions of the qualities an e-learning tool must have to achieve a standard.

Although our rubric integrates a broad range of functional, technical, and pedagogical criteria, it is not intended to be overly prescriptive. Our goal is for the framework to respond to an instructor’s needs and be adapted as appropriate. For example, when a rubric criterion is not relevant to the assessment of a particular tool, it can be excluded without impacting the overall quality of the assessment.

The rubric reflects our belief that instructors should choose e-learning tools in the context of the learning experience. We therefore encourage an explicit alignment between the instructor’s intended outcomes and the tool, based on principles of constructive alignment. Given the diversity of outcomes across learning experiences, e-learning tools should be chosen on a case-by-case basis and should be tailored to each instructor’s intended learning outcomes and planned instructional activities. We designed the rubric with this intention in mind.

The Rubric for E-Learning Tool Evaluation offers educators a framework, with criteria and levels of achievement, to assess the suitability of an e-learning tool for their learners’ needs and for their own learning outcomes and classroom context. The rubric was designed with utility in mind: it is intended to help decision-makers independently evaluate e-learning tools.

 

 

 

Why demand originality from students in online discussion forums? — from facultyfocus.com by Ronald Jones

Excerpt:

Tell me in your own words
Why demand originality? In relating to a traditional classroom discussion, do students respond to the professor’s question by opening up the textbook or searching for the answer on the Internet and then reading off the answer? Some might try, but by asking questions the professor is looking to see if the students grasp the discussed concept, not if they know how and where to find the answer.

Online students have the advantage of reflection time, along with having the textbook and Internet search engine open when responding to discussion questions. With a few simple clicks, virtually any question can be answered by searching the Internet. Once again, why demand originality? Classroom learning takes place when students are required to think; that’s a few steps beyond clicking copy and paste. As instructors, we should encourage our students to be resourceful and to learn the skills of locating and incorporating scholarly literature into their work. But we also must instill the learning value of synthesizing sources in such a manner that produces evidence of gained knowledge.

 

From DSC:
I like the idea of asking students to put it into their own words. Not just to get by the issue of copying/pasting or trying to stem plagiarism, but because it’s more along the line of journaling about our learning. We need to actually engage with some content in order to put that content into our own words. Not outsourcing our learning to others. Journaling can help us clarify what we’re understanding and where we still have questions and/or concerns.

 

 

Encouraging participation of all in the course: Moving from intact classes to individuals students — from scholarlyteacher.com by Todd Zakrajsek, University of North Carolina, Chapel Hill

Excerpts:

During every class session, read the room by watching individuals. Are students taking notes, nodding along as others speak, or even advancing the discussion by building on the comments of classmates? Are verbal responses merely defining terminology, or do they make connections between the text and real-world examples? Analyze the extent to which certain examples or content areas are received by individual students. Take note when student responses are merely noise to fill the void when you are not talking. Overall, look for individual characteristics that emerge within your course as a community of learning is being established.

Keep in mind that it is often less threatening to one’s ego to claim a lack of preparation for class than it is to admit that one is finding it difficult to understand the material. For those who need a bit of motivation to come prepared, a quiz at the beginning of class will help students to come to class ready to discuss the material for that day.


As all students are pressed for time these days, a quiz might be the added motivation that most students need. These quizzes do not need to be extremely challenging, but they should be challenging enough to ensure the required preparation is done. That is, one should not be able to get responses correct simply by guessing. For students who do not understand the material, quizzes will not prepare them to engage in class discussions or to answer your questions during a discussion lecture. For those students, failed quizzes might add additional pressure and cause less engagement with the material. Struggling students who are not prepared for class need assistance to understand the material. Carefully structured small group projects and discussions might be the best way to get their voices into the class. Ask increasingly difficult questions as part of the discussion, and when you know you have struggling students reserve some of the easier questions for those students.

 

 

Gen Zers look to teachers first, YouTube second for instruction — from campustechnology.com by Dian Schaffhauser

Excerpt:

Students in Generation Z would rather learn from YouTube videos than from nearly any other form of instruction. YouTube was designated as the preferred mode of learning by 59 percent of Gen Zers in a survey on the topic, compared to in-person group activities with classmates (mentioned by 57 percent), learning applications or games (47 percent) and printed books (also 47 percent). A majority (55 percent) believe that YouTube has “contributed to their education.” In fact, nearly half of survey participants (47 percent) reported spending three or more hours every day on YouTube.

The only method of instruction that beat out YouTube? Teachers. Almost four in five Gen Zers (78 percent) reported that their instructors “are very important to learning and development.” That’s nearly 20 percentage points higher than the YouTube option.

While Millennials also value teachers above all else for learning (chosen by 80 percent), that’s followed by printed books (60 percent), YouTube (55 percent), group activities (47 percent) and apps or games (41 percent).

 

 

Also, see the work from Pooja K. Agarwal | @PoojaAgarwal
Assistant Professor, Cognitive Scientist, & Former K-12 Teacher. Follow  and subscribe for teaching strategies at .

An example posting:

Retrieve, Space, Elaborate, and Transfer with Connection Notebooks — from retrievalpractice.org

Excerpts:

How can we encourage students to retrieve, elaborate, and connect with course content? Here’s a strategy called Connection Notebooks by James M. Lang, Professor at Assumption College. Connection Notebooks include retrieval practice, spacing, elaboration, and transfer – all in five minutes or less!

Ask students to dedicate a specific notebook as their Connection Notebook at the beginning of the semester (or provide one for them) and have them to bring it to class every day. Approximately once a week, ask students to take out their Connection Notebook and write a one-paragraph response to a “connection prompt” at the end of class. For example:

  • How does what you learned today connect to something you’ve learned in another class?
  • Have you ever encountered something you learned today in a TV show, movie, song, or book?
  • Have you ever experienced something you learned today in your life outside of school?


Connection Notebooks are effective for a few reasons:

 

 

Also, see the work from Learning Scientists | @AceThatTest | learningscientists.org

An example posting:

 

In this digest, we put together 5 blog posts by teachers that focus on implementing spaced practice in one specific subject at a time. For more of an overview of spaced practice, see this guest post by Jonathan Firth (@JW_Firth).

 

sapced practice in practice
 

 

Despite increase in Instructional Designers, there is no ‘universal profile’ for the role — from campustechnology.com by Dian Schaffhauser

Excerpt:

In higher education, one 2016 estimate pegged the “minimum” number of instructional designers employed by colleges and universities at some 13,000. But, as a new report from the Online Learning Consortium pointed out, “There is still a certain mystery surrounding who instructional designers are.”

Instructional Design in Higher Education: Defining an Evolving Field seeks to clear up the mystery. The report is the second in a series to examine aspects of the digital learning landscape by doing a literature survey of existing research. The first report looked at accessibility. The series is being produced by OLC’s Research Center for Digital Learning & Leadership.

 

 

 

Campus Technology recently announced the recipients of the 2018 Campus Technology Impact Awards.

 

Campus Technology recently announced the recipients of the 2018 Campus Technology Impact Awards.

 

Categories include:

  • Teaching and Learning
  • Education Futurists
  • Student Systems & Services
  • Administration
  • IT Infrastructure & Systems

 

From DSC:
Having served as one of the judges for these competitions during the last several years, I really appreciate the level of innovation that’s been displayed by many of the submissions and the individuals/institutions behind them. 

 

 

How much research has been done on flipped learning? Annual update for 2018 –from rtalbert-org.cdn.ampproject.org by Robert Talbert

Excerpt:

It’s now a tradition here at my blog to do an annual update of my answer to the common question, How much research is out there about flipped learning? I first posted about this two years ago after my book was published, and updated it last June to include info on 2016 and make some predictions about 2017. I’ve gone through and done it again this year, and I’d like to share the results of publication on flipped learning in 2017 and make some more predictions.

 

The amount of research on flipped learning is mainly since 2014

 

Taking the mid-2018 numbers into account, which bring the publication grand total up to 271, this means that over half the research that has ever been published on flipped learning has been published in the last 18 months (i.e. in 2017 and 2018); over two-thirds of it in the last two and a half years; and almost 90% of it in the last three and a half years.

 

 

 

 

You’re already harnessing the science of learning (you just don’t know it) — from edsurge.com by Pooja Agarwal

Excerpt (emphasis DSC):

Now, a decade later, I see the same clicker-like trend: tools like Kahoot, Quizlet, Quizizz and Plickers are wildly popular due to the increased student engagement and motivation they can provide. Meanwhile, these tech tools continue to incorporate powerful strategies for learning, which are discussed less often. Consider, for example, four of the most robust research-based strategies from the science of learning:

  1. Retrieval practice
  2. Spaced practice
  3. Interleaving
  4. Feedback

Sound familiar? It’s because approaches that encourage students to use what they know, revisit it over time, mix it up and learn about their own learning are core elements in many current edtech tools. Kahoot and Quizlet, for example, provide numerous retrieval formats, reminders, shuffle options and instant feedback. A century of scientific researchdemonstrates that these features don’t simply increase engagement—they also improve learning, higher order thinking and transfer of knowledge.

 

 


From DSC:
Pastors should ask this type of question as well: “What did we talk about the last time we met?” — then give the congregation a minute to write down what they can remember.


 

 

Also from Pooja Agarwal and RetrievalPractice.org

For teachers, here’s what we share in a minute or less about retrieval practice:

And when it comes to students, the first thing we share are Retrieval Warm Ups. These quick, fun questions engage students in class discussion and start a conversation about how retrieval is something we do every day. Try one of these with a teacher to start a conversation about retrieval practice, too!

 

 
 

A more strategic approach to arranging students into groups — from facultyfocus.com by Maryellen Weimer

Excerpt:

What’s the best way to put students into groups? It’s the first task that confronts teachers who want students to work together. And the best reply is one of those “it depends” answers. Here are the questions on which it depends.

 

If the group work is a project that requires extended collaboration and will benefit from a variety of opinions and perspectives, letting students form the groups may not be the best approach. On the other hand, for short, ad-hoc group work and for students who may be shy and not used to working with peers, knowing others in the group makes the experience less intimidating.

 

If one of the goals of the group work is getting students acquainted with others in the course or providing the experience of learning to work with peers they don’t know (which frequently occurs in professional contexts), then teachers should consider forming the groups.

 

What criteria should teachers use when forming groups? There’s a range of options. Here’s some of the more common criteria.

  • No criteria
  • Ability
  • Personality traits
  • Skills and experiences

 

 

 

Get started with rubrics — from thecreativeeducator.com by Melinda Kolk
Make assessment a classroom conversation

Excerpt:

A rubric is an assessment tool that clearly outlines expectations for student work. A rubric describes which performances will be assessed and specifies the criteria for assessing them. Rubrics can be used throughout the process of student work, making them useful for project-based learning implementations.

Because rubrics describe what high performance looks like, they are great tools to help students gauge their work and provide an opportunity for conversation between students and teachers about high-quality work.

Creating a rubric for the final product and various components of project work can ensure a more accurate, specific, and useful assessment.

A rubric is an authentic assessment tool that:

  • Provides clear expectations for a project.
  • Considers the product as well as the entire project-building process.
  • Enumerates the performances on which students will be evaluated.
  • Explains what constitutes excellence for each performance.
  • Helps students understand what they need to do to excel.
  • Prevents subjectivity and bias during the evaluation process.

 

 

Also see:

 

this graphic links to a search for rubrics out at FacultyFocus.com

 

 

 

From DSC:
Rubrics need to be very carefully designed/constructed though; otherwise rubrics can spoon-feed students the answers in an age where problem solving capabilities are really needed.

 

 

 
 

What is a learning ecosystem? And how does it support corporate strategy? — from ej4.com by Ryan Eudy

Excerpt:

learning ecosystem is a system of people, content, technology, culture, and strategy, existing both within and outside of an organization, all of which has an impact on both the formal and informal learning that goes on in that organization.

The word “ecosystem” is worth paying attention to here. It’s not just there to make the term sound fancy or scientific. A learning ecosystem is the L&D equivalent of an ecosystem out in the wild. Just as a living ecosystem has many interacting species, environments, and the complex relationships among them, a learning ecosystem has many people and pieces of content, in different roles and learning contexts, and complex relationships.

Just like a living ecosystem, a learning ecosystem can be healthy or sick, nurtured or threatened, self-sustaining or endangered. Achieving your development goals, then, requires an organization to be aware of its own ecosystem, including its parts and the internal and external forces that shape them.

 

From DSC:
Yes, to me, the concept/idea of a learning ecosystem IS important. Very important. So much so, I named this blog after it.

Each of us as individuals have a learning ecosystem, whether we officially recognize it or not. So do the organizations that we work for. And, like an ecosystem out in nature, a learning ecosystem is constantly morphing, constantly changing.

We each have people in our lives that help us learn and grow, and the people that were in our learning ecosystems 10 years ago may or may not still be in our current learning ecosystems. Many of us use technologies and tools to help us learn and grow. Then there are the spaces where we learn — both physical and virtual spaces. Then there are the processes and procedures we follow, formally and/or informally. Any content that helps us learn and grow is a part of that ecosystem. Where we get that content can change, but obtaining up-to-date content is a part of our learning ecosystems. I really appreciate streams of content in this regard — and tapping into blogs/websites, especially via RSS feeds and Feedly (an RSS aggregator that took off when Google Reader left the scene).

The article brings up a good point when it states that a learning ecosystem can be “healthy or sick, nurtured or threatened, self-sustaining or endangered.” That’s why I urge folks to be intentional about maintaining and, better yet, consistently enhancing their learning ecosystems. In this day and age where lifelong learning is now a requirement to remain in the workforce, each of us needs to be intentional in this regard.

 

 

Report: Accessibility in Digital Learning Increasingly Complex — from campustechnology.com by Dian Schaffhauser

Excerpt:

The Online Learning Consortium (OLC)has introduced a series of original reports to keep people in education up-to-date on the latest developments in the field of digital learning. The first report covers accessibility and addresses both K-12 and higher education. The series is being produced by OLC’s Research Center for Digital Learning & Leadership.

The initial report addresses four broad areas tied to accessibility:

  • The national laws governing disability and access and how they apply to online courses;
  • What legal cases exist to guide online course design and delivery in various educational settings;
  • The issues that emerge regarding online course access that might be unique to higher ed or to K-12, and which ones might be shared; and
  • What support online course designers need to generate accessible courses for learners across the education life span (from K-12 to higher education).

 

 

Guiding faculty into immersive environments — from campustechnology.com by David Raths
What’s the best way to get faculty to engage with emerging technologies and incorporate new learning spaces into their teaching? Five institutions share their experiences.

Guiding faculty into immersive environments -- by David Raths

Excerpt:

One of the biggest hurdles for universities has been the high cost of VR-enabled computers and headsets, and some executives say prices must continue to drop before we’ll see more widespread usage. But John Bowditch, director of the Game Research and Immersive Design Lab at Ohio University’s Scripps College of Communication, is already seeing promising developments on that front as he prepares to open a new 20-seat VR classroom. “Probably the best thing about VR in 2018 is that it is a lot more affordable now and that democratizes it,” he said. “We purchased a VR helmet 13 years ago, and it was $12,000 just for the headset. The machine that ran it cost about $20,000. That would be a nonstarter beyond purchasing just one or two. Today, you can get a VR-enabled laptop and headset for under $2,000. That makes it much easier to think about integrating it into classes.”

 

 

Colleges and universities face several hurdles in getting faculty to incorporate virtual reality or immersive experiences in their courses. For one, instructional designers, instructional technologists and directors of teaching and learning centers may not have access to these tools yet, and the budgets aren’t always there to get the labs off the ground, noted Daniel Christian, instructional services director at Western Michigan University‘s Cooley Law School. “Many faculty members’ job plates are already jam-packed — allowing little time to even look at emerging technologies,” he said. “Even if they wanted to experiment with such technologies and potential learning experiences, they don’t have the time to do so. Tight budgets are impacting this situation even further.”

 

 

 

 

How blockbuster MOOCs could shape the future of teaching — from edsurge.com by Jeff Young

Excerpt:

There isn’t a New York Times bestseller list for online courses, but perhaps there should be. After all, so-called MOOCs, or massive open online courses, were meant to open education to as many learners as possible, and in many ways they are more like books (digital ones, packed with videos and interactive quizzes) than courses.

The colleges and companies offering MOOCs can be pretty guarded these days about releasing specific numbers on how many people enroll or pay for a “verified certificate” or microcredential showing they took the course. But both Coursera and EdX, two of the largest providers, do release lists of their most popular courses. And those lists offer a telling snapshot of how MOOCs are evolving and what their impact is on the instructors and institutions offering them.

Here are the top 10 most popular courses for each provider:

 

Coursera Top 10 Most Popular Courses (over past 12 months)

 

edX Top 10 Most Popular Courses (all time)

 

 

So what are these blockbuster MOOCs, then? Experiential textbooks? Gateways to more rigorous college courses? A new kind of entertainment program?

Maybe the answer is: all of the above.

 

 

 

From DSC:
With thanks going out to Nelson Miller, Associate Dean at WMU-Cooley Law School, a Professor of Law, and an author of many of his own books…below are some of my notes and reflections on the following article:


B. Teaching Metacognition in the Classroom


Because preparation and planning are vital to learning, law professors should set clear learning goals at the beginning of the semester and at the beginning of each unit and hand them out to the students, so that the students know what they are expected to learn in the class and to help them set their own goals for the class. (p. 14 of 61)

There are three criteria for creating goals (Insert from DC: i.e., learning objectives): 

  1. “the terminal behavior, or what the learner must be able to do by the end of instruction,”
  2. the conditions of demonstration, or the circumstances under which the learner must be able to perform the terminal behavior,” and
  3. “the standards or criteria, or how well the learner must be able to perform the objective for the instructor to conclude that the learner has met it.”

Goals for a doctrinal class should include substantive, skills (process), and professionalism goals

Teachers need to teach metacognitive skills explicitly. As one author has stated, “[i]f the knowledge is never shared through discussion, modeling, or explicit instruction, it is difficult for students to learn.” Moreover, being explicit about the learning process is a key to creating a learning self-identity. Knowledge of metacognitive strategies, such as those for memorization, studying, and reading, makes students more likely to use them, especially if they are told that such strategies will improve their performance and grades. (p. 15)

Professors should say out loud their thinking process when working sample problemsModeling of strategies”–talking out loud about the steps the teacher uses when solving a particular problem–helps students develop metacognitive process skills (mental apps) by providing models. When students are given a problem to solve they generally focus on task completion, rather than the problem-solving process. Consequently, they will usually adopt a trial and error approach, rather than the process the teacher wants them to employ. Modeling of strategies helps solve this problem. Similarly, telling the students why the professor is employing a particular strategy helps conditional metacognition. After giving the demonstration, the professor should also ask questions that determine whether the students understood the processes for coming up with the answers (the problem-solving strategies). Teachers should also help students create strategies for solving ill-defined problems. In sum, the teacher should demonstrate a sample problem’s thinking process in detail, explain why the teacher used that process, have students do similar problems, then give student’s feedback on their problem-solving skills. (p. 18)

An example of scaffolding is partially filling out a diagram, then letting the students finish it (the partial outline approach). Another one is giving students leading questions before they read a case. (p. 19)

Professors should teach their students how to listen in class. Active listening aids learning, while passive listening often results in little retention. Students should think about what they are hearing in class. How does the material relate to my prior knowledge? What is important and what is less important? How can I use this material later? Where would I put this material in an outline? Students should also critically evaluate what the professor is saying. Do I agree? Is there an alternative argument? What are the implications of this argument? Similarly, professors should teach students effective note taking. (p. 20)

Law professors should also help students with study strategies. Deciding what items to study, how to allocate study time, and what study strategies to use are types of metacognitive control. (p. 22)

Professors should help students use study strategies that reinforce long-term memory and create connections between concepts, processes, declarative knowledge, etc. One way of doing this is through repetition,

Experts advise at least four repetitions of material each at least once within a day for retention.

Professors should help their students develop reading strategies. Engaged teachers help their students extract meaning and comprehension from cases, rather than just producing empty briefs. (p. 23)

 


E. Using Formative Assessment to Develop Metacognition 

Well-designed formative assessments–assessments within the learning (during the semester)–that are related to course goals also aid in learning metacognition. This is because formative assessments force students to think about their thinking. As one scholar has asserted, “[a]ssessment methods and requirements probably have a greater influence on how and what students learn than any other single factor.” As one legal educator has noted, “[f]ormative assessment . . . is designed to provide feedback and guide students to improve and learn further, based on feedback that enhances their capacity to build on what they know and address areas of misunderstanding.” As a group of researchers have pointed out, “[i]f the assessments reflect the contexts in which the knowledge is to be used, this is nothing more than practice.

Under the “testing effect,” “learning and memory for material is improved when time is spent taking a test on the material, versus spending the same amount of time restudying the material” because testing engages students in the subject matter. Also, testing uses retrieval, which as stated above, helps long-term retention. In addition, students retain more if they get feedback on their assessment because without feedback students don’t why they’ve made mistakes. Similarly, students who receive feedback are generally more engagedmore positive about law school, and spend more time studying than those who do not receive feedback. In addition, feedback about process is generally more useful than feedback about product. (p. 32)

There are many different kinds of formative assessments, including writing assignments, problem-solving exercises, multiple choice tests, observations, and [daily/weekly quizzes]. For instance, a professor could give the students a problem-solving exercise at the end of a unit to do at home, then go over that exercise in class. Likewise, the professor could give the students a take-home multiple choice test that could be graded by a teaching assistant. Similarly, the professor could give the students a complaint, a corporate document, or a lease and have the students find errors. Finally, the students could draft a contract clause after the unit concerning that clause. (p. 33)

The general criteria for designing effective formative assessments are:

  1. formulate learning objectives and performance standards; publicize them to the students
  2. design the assessment tool
  3. design instruction and activities to enable the students to learn what they need to fulfill the assessment task
  4. provide feedback/discussion — the teacher and student must discuss and use the results of the assessment measure to further promote learning and teaching

As one scholar has noted, “students learn more effectively when their teachers provide them with the criteria by which they are evaluated.” One way of doing this is through rubrics–“sets of detailed written criteria used to assess student performance.”  (p.34)

[R]ubrics are an attempt to break the grade down into a series of scores that pertain to various aspects of the assignment.” Rubrics identify how a student performed on a particular task, skill, or area. They include both characteristics and levels of quality. They can be a scoring rubric, an instructional rubric, or both. Rubrics should be related to the teacher’s goals for the class. They can employ grades, numbers, or categories.

 


“Successful students take charge of their own learning.”


 

From DSC:
My concern with this great article is that it’s a lot of valuable information to take in all at once — let alone try to act upon that information!  I’ve only touched upon a subset of the items within that article. As such, it’s too much information for faculty members and instructional designers to remember and to act upon – as a whole. 

This is why I’m such a fan of blogging and chunking valuable information up into much smaller pieces — then sending out such information piece by piece, where it’s much easier to digest and act upon.

 


 

 
© 2024 | Daniel Christian