Summary:
Merrill
presents a 5 star rating system for evaluating instruction to see if it
contains the first principles of instruction.
One star is given for each of the first principles used: real-world problem, activation,
demonstration, application, and integration.
If a star is awarded, it is awarded at one of 3 award levels based on the
degree of criteria achievement: gold,
silver, and bronze. The author states
that the rating scale is most appropriate for tutorial or experiential (simulation)
instruction. It is not applicable for
reference material or isolated facts to include what the author calls “Tell and
Ask” training where information is presented and then followed by a paper-based knowledge
test. Merrill states that the 5 star
rating scale may not be appropriate for receptive (e.g., lectures) or
exploratory unstructured problem solving with little or no guidance. He also also states that this rating scale
may not be appropriate for psychomotor tasks.
His rating scale includes 32 questions categorized into the five areas by which to rate the course’s
content to evaluate if it contains the 5 first principles needed to achieve
effective, efficient, and engaging instruction.
Critique/Findings:
It seems like a logical rating scale on which to build a
rubric for evaluating instruction. I also believe it would be able to be used to
evaluate all methods of delivering instruction and not just electronic or
computer/web-based instruction. I assume this could be used for both formative and
summative evaluation depending upon when it is applied. Formative when used to evaluate training as
it is being designed, developed and implemented; summative when applied to evaluate a course
after it has been implemented.
I am not sure why the author states this scale may not be suitable
for use for evaluating a psychomotor task?
If the psychomotor task is based on a real world problem I see no reason
why it could not be used. (If it not a real world problem, what is the justification for training it?) He also does not provide any guidance
on how to apply the rating scale other than to ask the questions. What determines a gold, silver, or bronze
star – or no star? How do you assign an
overall score or rating for the instruction as a whole? I guess I will see tomorrow when I use it to
rate an online boating safety course required by the state of Virginia.
Hello Kevin,
ReplyDeleteI hope you had a good week. I enjoyed reading your post. I especially like how you suggest that the 5 star rating system appears to be equally well-suited for application in both formative and summative evaluation. While the accompanying exercise demonstrated its use in a summative capacity, I am sure we will get some practice applying it to our own instruction in a more formative fashion.
I also concur with your critiques of the system. As you mention, in practice I encountered difficulty determining an appropriate “medal” level to award. I am curious to see if you feel the same way after evaluating the boating safety course you are planning to take.
In terms of the system's application to psychomotor training, I do think there are many similarities between instructional design models for experiential instructional development and psychomotor skill development. In looking very briefly at this topic for an assignment last semester, I found that the majority of research into psychomotor learning takes place within the medical community. I have attached a link here to one of the articles that I found interesting. The major difference I see between the models involves the need for highly repetitive practice in the case of psychomotor learning. In the case of the 5-step model mentioned in the George and Doto (2001) article, the authors also emphasize having the learners repeat back the performance steps to the instructor. Anyway, I think psychomotor training is an interesting topic to look at and as a previous instructor noted, it is not a topic that traditionally receives a great deal of attention within the IST program.
http://www.stfm.org/fmhub/fm2001/sep01/ftobtofm.pdf
Thanks for sharing your insights!
-Kipp
George, J., & Doto, F. (2001). A simple five-step method for teaching clinical skills. Family Medicine, 33(8), 577-578.
Hi Kevin,
ReplyDeleteThanks for the great post!
I agree with you that it was not fully explained how to give a rating to a particular instruction. In my example, I gave 1 star for each section, and gave each star a color depending on how many of the criteria were met for each star. However, I found this very subjective. I could easily see how two or three people could evaluate the same instruction and give very different ratings depending upon their interpretation.
For this reason, I find the rating system lacking in standardization. however, I do feel it does present some useful guidelines to follow when evaluating or creating instruction.
Kassie