My institution has recently adopted Quality Matters standards for Online courses. We will begin by applying the standards/ rubric, but developing our own internal process. Does anyone have anything to share on how they have developed a similar process. Lessons learned, best practices? Any feedback is much appreciated.
Here at Open SUNY's Center for Online Teaching Excellence, we’ve had some experience developing a process, and are in the middle of trying to do it again. We aren’t a campus ourselves, but we work with almost all the campuses across SUNY’s 64-member system.
In the past we tried setting up a cross-campus review process, which met limited success. We did manage to train a couple hundred faculty and instructional designers in how to apply the quality matters rubric, but we completed very few reviews. The issue was faculty time and motivation. We set up a process where faculty members would review each other’s’ courses for free, as a sort of reciprocal agreement. Faculty were all very well intentioned, but many of them failed to complete the reviews in a timely manner. They meant to help, but life got in the way! Some of this is the nature of people’s busy lives, but some of this was due to that fact that we did not incentivize or reward participation properly. We never figured out exactly how to reward and recognize participants.
We’ve recently set out again to create a system of regular review and refresh of courses, with a few major changes:
We haven’t fully rolled out the entire program, so I’m sure there will be unexpected pitfalls and pleasant surprises along the way. All of our work is licensed with a Creative Commons license, so feel free to take a look. If you would like to talk more, feel free to reach out to us at Contactcote@suny.edu or me directly at email@example.com
Open SUNY COTE
This is so helpful!!! Thank you. I'll be sure to tell the team about it and share as we learn more ourselves.
In addition to Dan's comments above, I'd like to offer a few "lessons learned" from our experience with training faculty to apply the QM rubric:
1. Faculty need a clear explanation of the narrow scope of the rubric, in that it looks at instructional design only, not other elements of the teaching/learning process (course management, feedback to students, instructor effectiveness, institutional resources, etc.);
2. It is helpful to clarify the communication path of "the score," as for some, it could be a reason for anxiety;
3. Faculty appreciate recognition for participating in the process. It is a professional development activity and should be recognized by the campus as such. If reviewing another's course, this could be deemed community service as the process requires time and skill;
4. The timing of a review is an important factor in implementation. October and February/March work well;
5. Encouraging informal self-reviews could be helpful in raising awareness and "tuning in" faculty to the rubric;
6. Incentives for reviewers go a long way toward successful participation. Stipends, release time, etc.
Assistant Director of Online Learning
Open SUNY COTE
Thank you Rob!
Regarding point 1. Yes I can see where the rubric review process could leak into some sort of evaluation of the teaching /learning process. We have recently implemented a Teaching Online Self Assessment (outside the QM process). The first round of the self-assessment was encouraged by a dean and voluntary. The results were very favorable. Right now we are creating a personalized framework of workshops, resources, and consultations for the responders if they wish to take advantage of them. Faculty members at a self-determined level of mastery, would be good candidates for internal reviewers, either from there experience or their own development from going from good to great, specifically in the area of design.
I love the idea of the open SUNY rubric not being a pass or fail. Having a low score on a QM review can leave a faculty in bad spirits.
I also love the idea that open SUNY's rubric looks at course delivery. I never understood why QM only looked at course design as design is only part of the story.
I think incentives go a long way to have faculty assist other faculty. And the other option, as Tracey mentioned, is self assessment.
Does anyone have a direct link to the SUNY Rubric, I've been trying to find it under cc?
I have this link Tracy.
I do remember open SUNY saying that the document is still evolving.
Thank You Ian!