Nine Ways to Reduce Cognitive Load in Multimedia Learning

by Richard E. Mayer and Roxana Moreno

SUMMARY

INTRODUCTION

The articles intended purpose is to provide instructional designers with the needed insight, knowledge, and skills to design and create meaningful instruction that does not result in students experiencing cognitive overload during learning. The article begins by exploring the theory of multimedia learning which explains that humans possess separate systems for processing pictorial and verbal material – called the dual-channel assumption. When either or both of these channels become overloaded with too much data (stimuli) the student experiences cognitive overload which in turn prevents meaningful learning. The authors begin the article by defining multimedia learning as “learning from words and pictures” and multimedia instruction as “words and pictures that are intended to foster learning.” Meaningful learning results in the student achieving a “deep understanding of the material, which includes attending to important aspects of the presented material, mentally organizing it into a coherent cognitive structure, and integrating it with relevant existing knowledge.”

The authors provide a description of the dual channel assumption which explains that the dual channel model includes an auditory/verbal channel for processing auditory input and verbal representations and a visual/pictorial channel for processing visual input and pictorial representations. Key elements of this model are the Dual Channel Assumption (the brain processes data in two separate channels), Limited Capacity Assumption (each channel has a limited capacity for collecting and processing/storing data), and the Active Processing Assumption (the brain must actively select, organize, and integrate the learning). Cognitive overload can and does occur when too much data is taken in, or it is taken in too fast, for the brain to process it. The following illustration shows how new data is processed and shared.

THE CASE OF COGNITIVE OVERLOAD

Cognitive overload occurs when the processing demands demanded by a learning task exceed the processing capacity of the cognitive system. The authors describe three kinds of cognitive demands: essential processing, incidental processing, and representational holding. Essential processing refers to the cognitive processes that are required (essential) to make sense of the material p[resented for learning. Incidental processing refers to the cognitive processes that are not required or essential for making sense of the presented material but are part of the design of the learning – usually more cosmetic than value-added. The last, representational holding, refers to cognitive processes needed for developing and holding a mental representation of what is being learned in working memory. The tezt discusses 9 methods used to prevent cognitive overloading:

Off-loading                                    Segmenting                             Pretraining

Weeding                                        Signaling                                 Aligning words and pictures

Redundancy Elimination                  Synchronizing                         Individualizing

The following define ways in which overload occurs and example of strategies to prent these situations.

Type 1 Overload: Off-Loading – occurs when one channel is overloaded with too many essential processing demands. This often occurs when one channel is forced to deal with input from two different sources at the same time creating a condition referred to as split-attention effect. In the figure above, this is represented by the two arrows from one channel (Multimedia Presentation - Words) providing output to two channels (Sensory Memory - Ears and Eyes) simultaneously. A common cause for this condition is the eyes receiving data from two different sources – the written word and pictures/graphics/drawings, etc. The best solution is to off-load excess data to a different track; e.g., convert the written text to narration.

Type 2 Overload: Segmenting and Pretraining - occurs when both channels are overloaded with essential processing demands. This is typically the result when both channels are overloaded with essential information that is content rich and delivered at a pace too fast for the student to properly receive and digest it. The two best solutions for this condition are to use segmenting and pretraining. To segment, the content is broken down into smaller bite sized portions – a process referred to as chunking - which gives the student more time and a better structure for receive and digest the information. In pretraining, the cognitive load is transferred to an alternate means or location and provided as pretraining. (We use pretraining a lot for Navy training. We do this by providing some of the key concepts, principles and processes online using Navy e-Learning and the Navy Knowledge Online NeL NKO) portal. Putting this information online and making the training a required prerequisite to staring the actual training course lessens the cognitive load in the class.)

Type 3 Overload: Weeding and Signaling – This occurs when the system (one or both channels) is overloaded by a combination of essential processing being combined with incidental processing demands due to extraneous material. Typically, this extraneous information is interesting, but of limited value. The two solutions for this case are weeding and Signaling. For weeding, examine the content and eliminate interesting material that is not value-added. Signaling requires you to organize and present the material in a way so the student can recognize essential content from content that is “nice to know.” I often do this by adding links or pop-ups for “nice to know information, this gets it off the same page as essential ionformation. If on the same page, I might put it in a side bar or in a text box.

Type 4 Overload: Aligning and Eliminating Redundancy. This occurs when the system (one or both channels) is overloaded by both essential and incidental processing demands resulting from the manner in which the essential material is presented – usually in a confusing way. I often see this occurring when the text is on one part of a page and a drawing or diagram is not immediately adjacent, or maybe on a totally separate page. The best solution for this case is to align the text and graphical representations – better yet, integrate the two. The other case for this occurrence is when the material is unnecessarily presented too often and/or in a confusing manner to the point that the material becomes redundant. The solution to this scenario is to eliminate unnecessary redundancy (the authors agree that this is similar to case for weeding above.)

Type 5 Overload: Synchronizing and Individualizing – used when the system is overloaded by the need to hold information in working memory while new or related information is presented in direct, unbroken succession. The example in the text illustrates an example where the material is first presented in one format (narration) and then immediately presented in a second format (animation) before allowing the first presentation to be absorbed. The first solution is to use synchronizing whereby the successive presentations are synchronized or integrated. For this instance, UCSB studies indicate that students understand a multimedia presentation better when animation and narration are presented simultaneously rather than successively. Individualizing can also be used when it is known that the target student population are high spatial learners who have the ability to hold and manipulate mental images with a minimal amount of cognitive energy. (Individualizing will not work for low spatial learners.)

DISCUSSION

This article hits very close to home for me. One of my jobs is to provide an ISD review for all products under development. The purpose for the ISD review is to ensure objectives are met and that solid instructional design principles are/were used in designing and developing the material. Based on experience, I can relate to almost every example given in this article; some of which I am guilty of. It is going to be nice to have an academic writing and the UCSB studies to back up many of the recommendations that I have been making.

For example, this week I reviewed a 130 slide PowerPoint presentation that details how a major system on a ship operates. The presentation is being converted for use as a student handbook (Trainee Guide) to be used as part of on-the-job training presented by subject matter experts during ship visits. One of my first observations to the developer was that he was being redundant to the point that it was confusing. I kept asking myself, why is he presenting this material again? (The developer seemed to think the redundancy was needed to drive the material home. He was overdoing it.) As stated in the reading, effective instructional design depends on the instructional designer (and the content developer) having sensitivity to the existence of cognitive load which, in turn, depends on his/her having an understanding of how the human mind works.

Lastly, this article applies to any multimedia presentation no matter the media used to presnt the material to the student. In particular, I have seen PowerPoint presentations that need lots of help. Especially when the developer thinks that if a little information is good a whole bunch of information must be proportionally better. Try as hard as I can, I am not always successful in convincing developers that sometimes less is more when it comes to providing effective training. And, I am sure most of us are familiar with the term, "Death by PowerPoint."

Effective Web Instruction, Chapters 5 and 6

SUMMARY

Chapter 5, Building a web prototype

In Chapter 5, Frick and Boling have us moving the prototype from paper to the computer for usability testing to see if the design and format make sense and to see if the course will work.  The authors remind us that if we are going to find that a web-based solution is not the right answer, we would like to find that out as soon as possible.  This chapter is mostly devoted to software requirements for getting a course up and going on a web server so that clients can get the course materials.  They start by discussing the fact that the web runs on the internet, and that the net is made up of computers (web servers) talking to other computers (client computers with web browsers).  Web servers and client browsers must be able to deliver and display files that contasin HTML and images in JPG, GIF and PNG formats as well as a wide assortment of other programs such as PDF, RTF, MP3, RAM, etc. 

An important fact is that “A Web server just fetches a page, delivers it to the client (Web browser running on another computer hooked to the Internet), and then forgets about that client - sort of like a computer with amnesia.”  Because of this the designer must design and build (program) transactional capability into the programming.  They offer several options:

• Create your own plug-in or player that does the interaction – or purchase one; eg., NETg (Skillsoft, ColdFusion, and MS .NET.)
• Include scripts in the HTML files – Scripts can be provided Server side or client side.  Client-side scripts are normally included in an HTML file or downloaded with the HTML (similar to image files), and then the user's Web browser interprets and runs the scripts. Server-side scripts are executed by the external computer host and the results are sent via the Web server as standard Web pages to the user.  Java Script, JAVA, Perl, and PYP are the most commn scripting languages used and all require some programming skills

Another major limitation to the web is that at present none of the scripting languages will support natural language dialogue – you can’t talk to the computer and it can’t talk to you.  As a result other methods of communication have to be used, all of which have their own limitations.  Samples include

• Web tools such as e-mail, electronic discussion boards, and instant messaging systems
• Web-based teleconferencing
• Web-based videoconferencing

A good quote from the authors:  “The old-fashioned classroom, where students and a teacher meet face-to-face is actually pretty efficient for natural language dialogue. Instructors don't need to repeat themselves as often, and most of us can talk faster than we can type.”

Other Types of Web Solutions

The following can be used as stand-alone solutions, or intermixed.

  Electronic Documents shared with the students that contain No Internal Hyperlinks

• E-mails with attachments
• Course Delivery tools – created or purchased
• Home-brewed delivery Systems

  Electronic Documents shared with the students that contain Internal Hyperlinks

• Create hypertext so that documents have internal links to other documents
• Requires a Web editor such as Dreamweaver.

   Transaction with Students with Teacher and other Students as Source of Feedback

• Requires discussion boards, chat, instant message systems
• News Groups
• The course delivery systems mentioned above have built-in discussion boards, and you can rely on stability, privacy for discussions, and support
• Chats, Chat Rooms and Instant Messages (IMs)  (Chat is synchronous, meaning that you need to be working together at the same time. This can be a disadvantage, particularly as students are in different time zones in other parts of the world.)

   Transaction with Students with Computer as Source of Feedback

• Multiple-choice responses via hyperlinks
• Web forms
• CourseBuilder
• Web Froms and PHP or Perl
• Games, simulations, adaptive instruction, and virtual worlds

Templates

Commercial Web development tools such as Macromedia Dreamweaver allow you to build templates – a template includes parts that are don’t change from page to page, and place holders for the content that does change.  The main advantage of a template is that it can greatly speed up the development process for prototyping, and once developed allows you to focus more on content.  Templates also make global changes to be accomplished easier and faster. 

Programmers can also develop custom templates which provide even more flexibility.

Creating the Prototype Rapidly

The first time around, the goal is to develop the prototype as quickly as reasonably possible.  The rapid prototype does not have to be final or smooth, but should provide a good representative sample of the final product for review and approval.  The process for the web prototype is the same basic process as used for the paper, except now we are using computers.  If problems are encountered, additional prototyping may be necessary.   

Chapter 6, Assessing and maintaining the site

This chapter deals with accomplishing Bug Testing (Technical Testing) before delivering the final product to the customer/end user.  Bug testing is a 3 phase process that is linear, with each stage being completed before beginning the next phase.  It is also iterative, in that it should be repeated until the product is complete and ready for delivery.  Very important to Bug Testing is that everybody needs to understand that the design should be locked and that now the testers and fixers are looking to find and fix quality and technical issues, and not to change design elements.

Phase I Testing – It is recommended that the Phase 1 Testing be done using checklists that identify the features to be tested and the locations (pages) where the features can be found.  Problems are documented on a Bug Report that is then used for Phase 2, Fixing. 

Phase II Fixing – When doing Phase 2 Fixing, it is very important that all Phase 1 Testing stops or document configuration control problems can occur which normally means unnecessary rework.  (I can attest to this, as can most developers.)  The bug report should classify each problem as to its seriousness and impact on the final product, something like the following:

     Priority 1 – Critical: must be fixed

     Priority 2 – Important: should be fixed

     Priority 3 – Good to fix: would make the overall product look and feel better

     Priority 4 – Nice to fix: only fix if schedule and budget allow. 

Often, priority 4, and maybe even priority 3, items may be rescheduled as a result of schedule or budget concerns.  A priority 1 problem would NEVER be rescheduled or put off until later.

Phase III Regression – During Phase III, the testers repeat Phase I Testing; all of Phase I should be repeated, do not test only the problems or bugs fixed as a result of the original test.  Only when no new Priority 1 or Priority 2 problems are found should the product be considered for delivering.  And, while we would really like to keep repeating the cycle until all problems are fixed and the product is 100%, this is often not practical by time or schedule. 

DISCUSSION

Chapter 5 was a particularly interesting chapter for me because I am not a programmer or media person, and anything I learn in this arena is good info for me for making good decisions.  For instance, what is the difference between a Course Delivery Tool (OnCourse and BlackBoard) and a Learning Management System (LMS) (ThinQ, SABA, Atlas Pro ROCCE, etc)?  Also, when discussing creating templates using commercial software tools (DreamWeaver), is this the same thing as a Learning Content Management System (LCMS)?  I know what an LMS and a LCMS are; I am trying to place them into the discussion here.  I view Dreamweaver as a type of LCMS.  Would you agree?

There was no discussion of Sharable Content Object Reference Model (SCORM) standards here (or in any other class I have had at IU).  Is anybody else working with SCORM and the ADL Co-labs?  I think IU had or has several faculty members active with the ADL Co-labs (e.g., Curtis Bonk.)  Do you think SCORM could become the future standard for e-learning content, or is it pie in the sky wishful thinking?  What about Section 508 of the American with Disabilities Act (ADA)?

I also enjoyed Chapter 6, but because it is what I do and I am familiar with the process.  A couple observations from my experiences:

1. Save time and resources for fixing things that the client finds.  No matter how much Bug Testing you do, your customer is likely to find things they want fixed and you need to have the budget and time left to make the fixes.  Even if you think a bug is preferential, the customer may see it as technical, and the customer is always right.  (NOTE:  In addition to the priority codes listed above we also classify all bugs as either Technical (T), Editorial (E), or Preferential (P).  We strive to fix all technical and editorial comments/bugs.
2. Once bug testing commences, everybody has to understand that the design is locked.  It’s inevitable that somebody is going to find a better way to do something.  Too late; save it for the next change or revision as long as it does not create a technical or editorial problem with the course. 
3. Configuration Control is always a problem.  I would recommend you ensure you have a good way to maintain document/file configuration control and file naming conventions.  I normally assign one person as the cyber librarian,  This person is responsible who is the sole person responsible for configuration control and ensuring file naming conventions are followed.  This person also starts and stops all Testing and Fixing activities. 
4. One last question:  Bug Testing – formative or summative evaluation?  The last formative evaluation or the first summative evaluation?  I personally think it is the last formative evaluation.  

What Makes effective, efficient, engaging instruction?

Summary

In his article, What Makes e³(effective, efficient, engaging) instruction?, Merrill is looking to integrate problem‐based learning, communities of learners, and distributed learning into a single approach that is e³. The paper is about efforts at BYU Hawaii, but should apply to many instructional applications. 

Merrill’s  goal (and BYU Hawaii’s) is for all courses and course materials to be on‐line.  Course assignments, course materials, discussion boards, and other technology facilities should be available to students online.  Students should be able to submit materials online, view the work of other students for collaboration and critique, and be able to interact with each other on line. The instructor should be able to monitor and mentor this student interaction.

Merrill also advocates that a technology enhanced course should be able to be taught on‐campus and at‐a‐distance using the same course materials and interaction facilities. He states that the students should be able to participate in the class on‐campus or on‐line from different locations both synchronously and asynchronously.

He starts his discussion on the two generally agreed upon memory processes:

• Associative memory – the primary model used for topic-based learning that is subject to forgetting if not immediately applied and reinforced. (from Wikipedia:  Associative memory (psychology), the ability of human memory to associate items such as a person with his or her name.)
• Mental models – The primary method for problem-centered learning where a problem‐centered approach facilitates the adaptation of an existing mental model or enables the learner to form a new mental model that integrates the various component skills into a meaningful whole.  Mental models are less prone to forgetting.  (from the article:  A mental model is a set of related ideas – a holistic representation of the parts, relationships, conditions, actions and consequences of a complete problem or task.)

The goal at BYU Hawaii is to make all instruction (classroom and distance): 1. problem‐centered, 2. peer‐interactive, and 3. technology‐enhanced.

1. Problem centered instruction uses mental models which last longer.
2. Peer Interaction (peer sharing, peer collaboration and peer critique) requires learners to test their mental models, to refine their mental models, and to share their ideas.  Activities that facilitate the learners’ ability to apply their knowledge in new situations.
3. Technology-enhancement facilitates a much wider dissemination of the training which enables the student to be engaged in problem solving no matter whether the instruction is in a classroom or at a distance. 

First Principles of instruction:  All instruction should use the following concepts/precepts: problem-centered, activation, demonstration, application, and integration.  Problem‐centered instruction is different from problem‐based instruction.

• Problem-based instruction focuses on solving a problem and is not always effective.  Could also be called problem-solving instruction.  (Constructivist approach?
• Problem-centered instruction focuses on using a successful solution as the basis for instruction.  (I would say this combines a cognitivist approach with constructivist approach.)

1. Problem-Centered Strategy

NOTE – On the PDF document, this graphic was originally titled “Problem-Centered Strategy” not “Task-Centered Instructional Strategy”  (It had a text box pasted over it.)

• For the first iteration (steps 1-3), demo first grouping of task topics (e.g., 1, 3 and 4) from a successful solution (A). 
• For the 2^nd iteration (B), have the student apply these same topics to a new task (steps 4 and 5). 
• For the 3^rd iteration (C), demo remaining topics (e.g., 2 and 5) for the successful task (C) and then apply to new task (D). 
• Drill and practice steps 4 – 7 to mastery. 
• Learners should be able to complete as a whole (topics 1 - 5) without instruction/guidance for (E).     

2. Effective Peer Interaction

Merril lists different types of peer interaction as follows:

• Peer telling – the least effective
• Peer sharing – Uses Activation
• Peer demonstration – Uses Demonstration
• Peer collaboration – Uses Application
• Peer critique – Uses Integration

Peer interaction involves the student in the solution of the problem at three levels:

1. Acquiring the component skills and applying them to an individual solution;
2. Discussing, defending, and elaborating different solutions in an attempt to come to a consensus solution; and
3. Critiquing another solution based on their understanding of the problem and possible solutions.

3.  Technology-Enhanced Interface

On‐line instructional components for a problem‐centered peer‐interactive course.

The instructor has two roles:

(1)    Course development

(2)    Guide and coach during instruction/learning

Conclusion

I really like this model.  I’m not sure I fully understand the difference between associative memory and memory models (even after researching), but I am not sure I really have to at this point.   I also had a little problem understanding the flow chart for the Problem-Centered Strategy, so after study and careful consideration, I reworded the flow narrative somewhat – to what I felt was a simpler, neater explanation.

I wish I would have had the technology-enhanced interface model three years ago when we designed the pre-commissioning unit (PCU) training courses for the CVN 78.  Our requirement is to provide PCU crew training for the initial crew, and to also develop and provide an interim training solution that can be used until the Navy establishes a permanent life cycle training solution for the Navy schoolhouse.  Our solution was to design and develop what we call dual purpose training materials that can be used as (1) computer aided instruction (CAI) in a classroom with an instructor/facilitator, and (2) as computer- or web-based instruction (CBT WBT) at a distance as the interim solution.  The original CAI can also be re-used by subject matter experts and ships force personnel for classroom instruction for follow-on crews.  Reviewing the Technology Enhance Interface model has given me ideas on how we could have increased collaboration exercises for our products (classroom and CBT/WBT). 

Finally, our intention is for the Navy to be able to repurpose our training as part of the life cycle, schoolhouse brick and mortar training.  

Making a Paper Prototype

This blog and summary is on Chapter 4, Making a Paper Prototype, from Carolyn Snyder’s book Paper Prototyping: The fast and easy way to design and refine interfaces (2003). 

SUMMARY

The chapter deals with the mechanics of building a paper prototype.  It explains the what and how; what you will need, and how you will build the paper prototype. 

In the opening paragraph she reminds us to:  “First define the users and tasks and build the paper prototype around them.”  

She divided the Chapter into topic and subtopics which I summarize below.

Paper Prototyping Materials – Most of the needed materials are common office supplies; a few are speciqlized and may need to be ordered online (e.g., removable tape).  See also the author’s website www.paperprototyping.com.  She offers some good ideas – nothing that I found to be particularly earth shattering.  She provides a list of both supplies she uses and those she does not use.

Creating a Background – The use of a background is optional.  The background serves as the “desk top” (my words) and sertves the same purpose as a master slide in a Power Point slide show.  It provides an overall look and feel for the project.  The background can represent the computer and contain permanent or constant elements.

Software Application Background – The developer needs to decide if the background needs to represent the operating system or just your applications desktop.  This is particularly useful if building for a particular system e.g., a Palm Desktop with its controls.

Browser Backgrounds – You normally are not testing or prototyping the browser window or its usability; you are normally testing your own application site.  Backgrounds can be used to “build-in the standards and conventions (look and feel) your site will be using.  Some standard buttons that may be included:  Next; Back; Home; Glossary; Help, etc.  Desktops do not normally show URL address fields and other fields if the student tester does not need to us.  She also warns, “Be careful not to make too many assumptions about what a student can do.”

Small Screen Interface – When designing/developing for small screens – pixels may matter.   If you are going to be designing for small screens, need to be “aware” from early on.  If necessary, grab a screen shot of the screen (e.g., Snagit) and use a graphics program to create your background.

How to prototype Interface Widgets – She provided several pages on ideas and tips on how to incorporate the following widgits into a paper prototype:  buttons and checkboxes, tabbed dialog boxes, text fields, drop down lists, select bar/highlights, expandable dialog boxes, expandable lists, disabled controls, and cursors.

Representing the Users’ Choices – Use removable tape for having the user record his/her actions is important for recording/remembering what a user does when making a choice or selection.  Having the observer monitor and record the computer actions is too cumbersome. 

Hand-Drawing versus Screen Shots – As previously stated by the Frick and Boling handbook, hand drawn is normally fine.  In fact, boxes and words may be all that is needed, instead of trying to draw GUIs.  i.e., don’t draw a logo. Use a box with the word “logo” to represent a logo.  The exception is when the GUI is needed for making a choice and you feel it is needed to convey a concept – selecting the right GUI.

Simulating Interaction – The one exception to “paper prototyping can do anything good enough” is when it comes to simulating interactions.  For interactions, the observer may need to make exceptions and be willing to provide answers and/or oral guidance – brief before you start the test.  Examples that may be exceptions:  Tooltips/Mouse-overs, Rollover/pop-up menus, Beeps, Drag and Drop, Right mopuse menus, Sliders, progress indicators, Animation and video, Web site links, Scrolling

Beyond the Computer Screen – Incorporating Other Elements.  These are the interfaces the user will have to interact with that are not on the screen.

Hardware Props – Items that may be needed as an interface part of the ultimate web based course, and include:  Cameras, Tape backup systems, Portable MP3 players, Touch screen control panel, and inter-connecting cables.  The Keyboard is not normally included as its use is well understood and should not be a problem.

Hardware Devices – Used for building training for real equipment, when the student will have access to the real equipment to practice on?  She is discussing simulating tasks not directly involving a computer screen.  Her example is simulating a can crusher.  I am not sure how this would translate, unless we are building training on how to operate the can crusher and she is recommending we build a 3D mock-up to simulate it.  I would be hesitant to tell the student to go practice on their own without guidance/supervision.  But, what if it is a school house where a “safe” simulator is provided for free play and exploration before going to the real thing?

“Incredibly Intelligent Help” – This is needed or used to help the student when they get stuck.  It can be used to simulate a “help Desk” or similar Expert System.  The “help” may be a single designated person.  Have them start with short answers and build on the answer as needed.  Record all questions and the amount of help required to get the student back on track.  Is this the start of a FAQ?

Human Actors – use when they would be simulating the need to access real people; for example, when calling a 1-800 number or doing a phone interview.  Do not have the tester and observer looking at each other.  This might be a designated person – not the normal observer.

Wizard of Oz testing – Paper prototyping is a type of Wizard of Oz testing where a person (the observer) is playing the role of the technology. Rarely, Wizard of Oz testing can require the observer or some other person to provide more advanced simulations use a computer or other technology.  It is especially useful when the technology is not fully developed yet. 

Documentation, Help, and Training – This section is for people working on documentation, help, or training for an interface.  It included three tips/areas to focus on: Content and Method of Access, Preparing Material for Testing, and Prioritizing the Informational Needs.

CONCLUSION

To me, some of her ideas come across as a cross (blending) of war-boarding (A type of storyboarding used for developing proposals), prototyping, and storyboarding.  She recommend using a storyboard in a format – a poster board – that won’t fit in a std loose-leaf binder from the Frick and Boling book.

It is obvious, that as an ID I do not know or understand the vocabulary of the IT programmer.  I needed to have a glossary of terms as I read her article since she used terms I am not aware of.  She keeps using the term widget and I didn’t really know what a widget is/was.  I used dictionary.com as my glossary. 

• Widget – an application, or a component of an interface, that enables a user to perform a function or access a service.
•  Blinder - Something that prevents someone from gaining a full understanding of a situation.

As I stated in my previous blog (Frick and Boling Chapter 3 and 4 Summary), I still think this process looks rather cumbersome and time consuming – unless it is used for a brand new project with new ideas and concepts the designer/developer is not familiar with.  For example, we normally develop for desk top and laptop screens and not for mobile devices.  As the Navy and DoD starts to explore mobile devices I am staring to see how prototyping may be useful for the first project.  

Effective Web Instruction: Handbook for an Inquiry-Based Process; Chapters 3 and 4

SUMMARY

In Chapters 3 and 4, Frick and Boling continue with their plan/process for preparing (chapter 3) and testing (Chapter 4) the paper prototype.  

Chapter 3 - Preparing for testing a prototype

Chapter 3 starts by identifying the three high level questions the paper prototype should answer:

1.        Is the instruction effective?
2.        Are students satisfied with the instruction?
3.        Is the product usable by the students?

To answer question #1, a 3-step process is used: (1) develop a pretest, (2) Develop usability testing, and (3) develop a posttest (posttest should test the same things the pretest did.)

The process for developing the content (tasks) goes as follows:  Start by teaching it yourself using traditional ILT.  As you teach it, develop the paper prototype rapidly on paper.  Be prepared to explain why the first paper prototype is on paper and not on the web. The prototype should include as much of the content as possible, e.g., critical components (diagrams, demos and presentations) and high-level detail – the breadth.  It should show the depth of the deepest component by showing the entire strand.  It should not include finished graphics or unnecessary detail.  The final prototype should not have a “finished” or “professional” look or feel – you want the test subject to feel free to criticize.

Informational Products, the collections of learning resources used to support the actual instruction, should be assembled in the same manner.  Capture the breadth of the website to the “top Level”; capture the Depth of the website by developing several strands; at least one complete strand, preferably the deepest.

Assembly instructions:  Create a notebook of pages ensuring that you number/code the pages and all hyperlinks.  When building the prototype (and the content) be sure to answer/reply to Merrill’s instructional principles known to facilitate learning:

1. Present in the context of a real world problem.
2. Activate prior knowledge or experience.
3. Demonstrate (show examples) of what is to be learned.
4. Provide opportunity to practice or apply their newly learned knowledge or skill. 
5. Provide techniques that encourage learners to integrate (transfer) the new knowledge or skill into their everyday life.

Develop an authentic assessment for student mastery.  Avoid the word “test” as it has a negative connotation to many.  Assessments should use “authentic” methods that imitate real life and match the instructional goal.  Use rubrics that answer the following:  What is the student performance you expect?  What are the conditions of the performance?  What criteria will you use to judge if the performance is good enough for mastery?  The assessment must be valid and assess task mastery. 

To assess student satisfaction, develop a Level 1 Kirkpatrick Reactionaire.  We will also be accomplishing a Level 2 Assessment of learning using the pre- and posttests. 

Finally, get an expert appraisal by asking a SME or SMTE to review the prototype.  Next, test the prototype yourself.  The self-check should be as much a quality check as a usability check.  Fix technical problems as they are found.  Do not use a prototype that has problems!  Create an “Observation Sheet” for use by the observer during the testing.  The observation sheet should ensure conformity and uniformity and provide a baseline. 

Chapter 4 - Testing a prototype

Chapter 4 provides a sample scenario for testing a prototype.  When conducting the testing, be sure to start with the Big Three: 

1. Use Authentic Subjects – Should match the identified TPOP. 
2. Use Authentic tasks – the learning activities should match the goals/objectives.  For informational content, the tasks should include information finding tasks.
3.  Use Authentic Conditions - Remember, this is for web based content w/o instructor or facilitator.  Consider using instruction sheets to guide tasks.

Next, create a test plan and hold a pilot session.  The Test Plan may or may not be required.  If used, use a Test Plan Checklist.  Identify and recruit subjects (testers) who are the actual TPOP.  Determine how many subjects are needed; 4 – 5 is normal minimum.  The text also discusses sampling methods.

Sampling until saturation – sampling until the results do not change (see Krug, Chapters 9&10). 

Bayesian Reasoning – sequential decision making.  (The number of subjects required to prove a website is not working properly is considerably fewer that the number required proving it works properly.  Frick got similar results when determining lengths of adaptive mastery tests (assessments) using Bayesian reasoning process.)

Pilot Session Observer Guidelines are provided that include: What to do; What to say for noncommittal responses (prompting and probing); What to look for; and how to handle a variety of  situations the observer might encounter (e.g., subjects who get stuck or give up.)  When conducting the Sessions, the observer(s) should provide the following structure.  

1.       Welcome - Provide an in-brief.  The in-brief is used to put the student at ease, tell them what to expect, and tell them to expect the worst – it’s not them it is the design.
2.    Give the pre-assessment.  Assesses what they know before they start.
3.       Present the prototype.  Remind the student to think aloud.  Remind them you expect them to encounter problems and need them to verbalize continuously.  Begin user activities/tasks.  Observe and record data - need to have an “observer sheet.”  Typically one observation sheet per task.
4.       Administer post-assessment (posttest – pretest = Kirkpatrick Level 2 evaluatiuon.
5.      Administer reactionaire (Kirkpatrick Level 1 evalaution.)

.

SUMMARY

I get mixed messages from this guidance about how much content the paper prototype should contain.  The authors say it should be a draft working copy that indicates the breadth and at least one full strand for the depth, but it also says it should allow the student test the entire website which implies it should contain everything.  This whole process seems to be somewhat cumbersome and lengthy, to develop first on apper and then a second time on the web seems less than efficient.  My customers demand efficiencies to keep the cost down and would question how much value-added was gained from this process.

It should be noted that the DoD and Navy both publish developers guides that provide fairly good guidelines for developing for the web.  When coupled with ADA, section 508 requirements, we normally have a pretty thorough set of standards and conventions for all the developers to follow.  And that is not even talking aboput Data Item Descriptions and Contreact Data Requirements Lists (CDRLS) that require detailed design pacjages for web development.

I hate to sound pessimistic, but for normal everyday use I would have trouble getting customer buy-in for paper prototyping.  I have trouble getting customer buy-in for the required 25% IPR proof of concept prototype which is typically done on a representative sampling of the content.  Telling the customer that I need access to a number of TPOP students on which to test they would consider an unneeded inconvenience to the end-user and an unnecessary cost to them.  Especially since there is no Navy instruction or guidance that requires it.  (This is the Program Office’s perception, not the end-users perception.  End-users – the Fleet Sailor – are normally very receptive to testing.)

Changes in Student Motivation During Online Learning

By  Theodore W. Frick and Kyong-Jee Kim

SUMMARY

This article by Frick and Kim focuses on why there is such a high attrition rate among first time online learners, and what we can do as instructional designers (ID) to design and develop online courses that increase a learner’s motivation to learn (and persist with the learning).  The authors first discuss what past studies revealed in Review of the Literature, and  and then conducts its own study in The Present Study.

REVIEW OF THE LITERATURE

Wlodowski (1998) states that learning experiences must be highly motivating to foster the learner’s persistence to pursue lifelong learning.  In a review of current literature, Kim and Frick (2011) identify three factors that influence a learner’s motivation to learn: internal factors, external factors, and personal factors.

Internal Factors

This section begins by discussing Keller’s ARCS model of motivational design whereby attention, relevance, confidence and satisfaction are identified as being key components that must be designed into all training courses. They also discussed confidence and self-efficacy as being important since self-efficacy can play such an important role in a learner’s belief in their own ability to succeed (Bandura, 1997).  In reading this section, I found the authors’ findings and recommendations could be classified as either factors that increase motivation or factors that decrease motivation, as follows.

Factors that Increase Student Motivation	Factors that may Decrease Student Motivation
Computer/internet self-efficacy	Cognitive overload, especially for first time online learners
Academic Learning Time (ALT) – Where learning is scaffolded so that the student is engaged at the proper level for success (learning is not too difficult or too easy – it is in the student’s zone of proximal development)	Going outside the student’s zone of proximal development
When ALT and First Principles occurred student’s are 3.6 times more likely to experience satisfaction with the course	If student’s felt they experienced ALT negatively, they were assessed as 10 times more liley to have a low degree of mastery
The convenience, flexibility, and control found in online learning can contribute to a learner’s motivation	High school students reported that the social interaction available in traditional classroom instruction is important.  Does this lack of social interaction result in decreased motivation in online learning?
The control over pace and timing is also an important factor	Technical difficulties and communications breakdowns
Interface designs for the human to computer interface
Level of interaction

External Factors

External factors are those that are influenced by the environment in which the student takes the online learning.  Examples of external factors include: learner support, technical support, the student’s instructional and organizational climate.

Personal Factors

While studies are not conclusive, some believe if the ID matches the instructional strategies with the student’s learning styles the learner will be positively motivated.  Past studies also attribute the student’s temperament, gender and age to motivation.  Lastly, a prospective student’s past experiences with online learning and the resulting perceptions about online learning can also impact motivation.

PRESENT STUDY

The present study was conducted on self-directed e-learning (SDEL) where adults are taking independent study courses via the web.  SDEL courses typically have the following characteristics:

·         No formal or a less formal enrollment

·         No set schedule or timeline

·         Self-paced with the pace established by the student

·         No instructor to interact with or motivate students

·         Little or no peer interaction

The purpose of the study was to determine how student motivation changes as they progress through the course.  Data was collected from 800 students using questionnaires.  Students were drawn from universities, the business sector, and government and non profits.  The surveys focused on three main areas:

·         Motivation when beginning the course

·         Motivation during the course

·         Change in motivation while taking the course

The types of courses taken and time spent on the courses varied considerably, however 94.2% of the respondents indicated that the reasons for taking the online training was because classroom training did not fit their schedules or a course was unavailable, or because online training was convenient and flexible (Frick and Kim, 2011).   Respondents reported a relatively neutral (flat) change in motivation as follows:

·         40% reported no change in motivation

·         34% reported an increase in motivation

·         26% reported a decrease in motivation

The three most important factors that appeared to determine motivation at the outset of the training were perceived relevance of the training, the student’s technology competence (self-efficacy), and the student’s age.  Two factors seemed to be the best predictors for motivation during the course:  the belief that e-learning is right for me and the level of motivation held at the outset of the training.  The best predictor for a change in motivation is the motivation experienced during the course, and lastly, the best predictor for the student’s satisfaction for the course at completion is the change in motivation experienced during the course. 

DISCUSSION

I have been a fan of Wlodowski since using one of his books as an undergraduate that dealt with strategies for teaching adults - a book that I continue to use regularly to this date. The updated version of the book is: Enhancing Adult Motivation to Learn: A Comprehensive Guide for Teaching All Adults by Raymond J. Wlodkowski (Apr 18, 2008).  I highly recommend the book of you develop training for adults.  

Are internal factors also intrinsic motivators?  External factors extrinsic motivators?  I guess Personal factors could be intrinsic or extrinsic.  I bring this up because I seem to recall reading (probably Wlodowski or Bandura) that intrinsic motivators (like relevance and inclusion) are more deeply ingrained, longer lived, and cause students to persist more so than extrinsic motivators.  Extrinsic motivators (such as rewards) have a tendency to be shorter lived and do not necessarily result in student persistence.  Extrinsic motivators are good to spur slumping motivation. 

To me, the present studies seemed to suggest that motivation begets motivation.  If the student was motivated to begin the class, motivation was likely to continue as long as nothing went wrong – internally or externally. This also suggests to me that a properly motivated student is ours for the losing – if that makes sense.  In my experience, most adult students who come to SDEL of their own volition or choosing do come intrinsically motivated. 

BIBLIOGRAPHY

Bandura, Albert (1997).  Self-efficacy: The exercise of control. Freeman: New York.

Wlodkowski, Raymond J. (2008).  Enhancing Adult Motivation to Learn: A Comprehensive Guide for Teaching All Adults.  3rd Edition.  Jossey-Bass: Hoboken, NJ.