OIA Blog

OIA recommends the following reading by the National Learning Alliance:  Committing to Quality -  Guidelines for Assessment and Accountability in Higher Education

This report identifies best practices for assessment in higher education.  OIA is in the process of developing their own handbook that will outline guidelines for assessment at SCAD.  The information in the above report provides a broad understanding of the guidelines OIA is striving to follow  at SCAD. OIA would like SCAD faculty and staff to be aware of the following four guidelines from the report and to understand how OIA aims to meet each of the guidelines.

Guideline # 1.  Policies should be in place that describe when and how learning outcomes are assessed.

- OIA is developing an assessment policy handbook that will outline assessment guidelines at SCAD that apply to all programs. For program specific information, OIA coordinators collaborate with programs to draft assessment plans on an annual basis.

Guideline #2. Assessment processes should be ongoing, sustainable, and integrated into the work of faculty and administrators.

-OIA aims to provide assessment evidence on an annual basis.  To ensure sustainability, OIA will take as much time as is necessary to build and test an assessment plan for reliability and validity.  To ensure the assessment plan is integrated into the work already being conducted by faculty, OIA will only require the collection of capstone course assignments (thesis/project/portfolio) to measure the impact of each degree program.

Guideline #3.  Evidence should include assessments of levels of engagement in academically challenging work and active learning practices.

-OIA will work with each program to ensure that the degree programs are evaluated based on the most academically challenging work assignments (capstone-level coursework).  Through collaboration with programs to update course goals and outcomes and participation in class observations, OIA will be able to engage in open discussions with faculty about the nature of the classwork.  In addition, OIA is happy to conduct additional research, such as alumni surveys and collection of school comparison data, to gather more information for faculty  on perception of  coursework quality.

Guideline #4.  There should be an established process for discussing and analyzing learning outcome recommendations and moving from recommendation to action. Where feasible and appropriate, key recommendations for improvement are implemented.

-OIA will develop a customized process for discussing and analyzing learning outcomes with each program to ensure that the process chosen best meets the needs of each program.  In general, OIA believes its important for results to be provided to a program  in both written and oral formats and for the results to be distributed to all program faculty.  Since OIA works closely with each program and closely with the curriculum management department, OIA feels comfortable facilitating discussions with program faculty about the most appropriate and practical recommendations to implement in future academic years.

And why is it important for us to structure assessment to this degree?  The Learning Alliance says it best:

“Higher education has been entrusted with an important social responsibility. This responsibility calls for a commitment to see that all students reach high standards and fulfill their potential. Doing so requires us to gather and report on evidence of student learning and use it to improve student learning outcomes.”

Please be sure to check our site in the next coming weeks for tutorials on scoring guides. A scoring guide is a simplistic, visually organized way to measure performance on a set of tasks. Scoring guides can be designed to measure performance based on a Likert-scale, checklist, letter-grade, or other point-value system.  Scoring guides can be created for both course and program assessments. Please click on the link below for a mock example of a program scoring guide.

At SCAD, program scoring guides measure capstone-level achievements for each degree program.  Program scoring guides (or capstone assessment tools) are developed by program faculty in collaboration with OIA to ensure they align with updates to program goals and outcomes.

OIA is also  happy to assist individual faculty members develop course scoring guides.   A course scoring guide may simplify and standardize the process of grading more time-consuming projects, such as research papers and design collections.  Course scoring guides benefit students as well.   Having assignment expectations outlined on a scoring guide directs students to the essential features they should focus on when producing an assignment.

Please contact your respective assessment coordinator for additional information on scoring guides.

ScoringGuide_Example

At the Winter 2012 faculty conference, the keynote speaker, Jonah Lehrer, gave an interesting talk about the cognitive psychology behind creativity.  He brought up several points as to why some people  thrive over time in a creative outlet while others fizzle out.  One of his points that stood out to me is  how moments of relaxation or reflection from what you’re immersed in are important for stimulating the part of your brain that will ignite creativity.   Taking long walks, a nice relaxing bath, or listening to music can be just what one needs to inspire creative thought.  Another interesting thought concerns this idea of grit, or persistence.  Some people are more persistent than others when it comes to producing work.  People with a lot of grit are willing to put in the long hours, and willing to see a problem through to the solution, even if it’s a struggle.  Compared to intelligence, Lehrer suggests that grit is more malleable, more easy to improve upon.  Lehrer made the point that people’s IQ scores do not dramatically change over time, but there are every day examples of people who become better at a given task through hard work – assuming they have enough interest in the task at hand to be persistent with it. Thus, grit is something special to any learning process, and it could potentially be just as important to study as the final outputs of the learning process.

As someone working in assessment, I couldn’t help but question what this information tells us about how we should teach students and measure student performance.

are students provided with enough reflection time within their courses to stimulate creativity?  I imagine the 4 day school schedule partially benefits the creative process.  What about the time spent within each class?  Are we providing enough reflection and analysis time or is most of the time devoted to lectures?

furthermore, how, if at all, are we measuring a student’s grit?  I think it’s possible that assessment of student portfolios can be a reflection of student grit.  Rather than a single measure of student work, a portfolio covers multiple works by a student, work conducted across their 4 years as a student.  Plus, portfolio work provides us with information on a student’s process, as opposed to student products.  Thus, portfolio reviews should distinguish students who are persistent and consistent with their work from those who are not.   That being said, the only way a portfolio will measure a student’s sustained practice or grit, is if the assessment rubrics we create to evaluate portfolios weighs the importance of this component.

“It takes a little courage, and a little self — control. And some grim determination, If you want to reach the goal. It takes a deal of striving, and a firm and stern-set chin. No matter what the battle, If you really want to win. There’s no easy path to glory, There’s no road to fame. Life, however we may view it, Is no simple parlor game; But it’s prizes call for fighting, For endurance and for grit; For a rugged disposition and don’t know when to quit.”

Happy New Year and Welcome Back! We have a new tutorial available on our ‘Tutorials’ page. If you would like to learn more about designing an effective multiple choice test, then this tutorial is for you.

If you would like additional assistance with designing multiple choice tests or updating pre/post tests for your department, please contact your assessment coordinator or email us at assessment@scad.edu.