Overview & Deadlines

Every department or program defines key learning outcomes for the majors it offers, gathers information on how well those outcomes are being accomplished, and uses findings to improve curriculum, instruction, and assessment.  Biannual assessment reports are due as indicated below.

Fine Arts, Humanities, Environmental Studies, Gender Studies.

Reports are due June 30 of odd-numbered years (2013, 2015, 2017, etc.).  For June 2013 only: Those who submitted a full report in June 2012 may submit a brief update using the Department Update Template.

Math & Computer Science, Natural Sciences, Social Sciences, East Asian Studies, Linguistics.

Reports are due June 30 of even-numbered years (2014, 2016, 2018, etc.).

Reports should be e-mailed to the chair of the Assessment Committee: Jeff Collett (jeffrey.a.collett@lawrence.edu).

+ Requirements

Every academic department or program does the following:

  • Defines key student learning outcomes for the major and maps those outcomes onto major requirements.
  • Assesses Senior Experiences and other student work to see how well students are achieving the outcomes.
  • Identifies common strengths and areas for improvement, and considers these in light of other evidence and past reports.
  • Plans follow-up study (targeted assessment, surveys, focus groups, interviews, etc.) in areas of concern.
  • Makes adjustments to curriculum, instruction, and/or assessment, and assesses the impact of those changes.
  • Documents these activities in a biannual assessment report.

The Assessment Committee does the following:

  • Provides feedback on assessment reports to help departments and programs improve their assessment and reporting practices. (The committee does not approve outcomes or methods; it offers advice as an outside reviewer.)
  • Compiles and archives the reports. (Reports are accessible to the Assessment Committee, the Office of Reseach Administration, and the Provost, and are used to prepare reports to granting or accrediting agencies.)
  • Prepares a summary report on assessment of student learning at the university.

To make assessment a manageable and valuable part of your department work, read the guiding principles (in the navigation menu) and use the templates and guide below.

+ Templates

  • Curriculum Map Template [XLS]
  • Rubric Template [DOC]
  • Department Report Template [DOC]
  • Committee Feedback Template [PDF]

 

Department & Program Assessment Guide

The sections below will guide you through the process of defining learning outcomes, creating a curriculum map, developing rubrics and assessment methods, using findings, and preparing an assessment report.

+ How to Define Learning Outcomes

Learning Outcomes

Learning outcomes say what students should be able to do upon completion of the major. While students are expected to grow in many ways and to develop their own interests and talents, every department or program should define a few key outcomes that all students in the major are expected to accomplish and that the department will assess to determine the effectiveness of its academic program.

Learning outcomes are written in the form:

Students should be able to <action verb> <object>.

Use just one action verb and one object to make the outcome assessable. State the outcome in language a student is likely to understand.

Examples:

<Department/program> majors should be able to:

Produce sound philosophical discourse.

Analyze a literary text.

Translate prose from Latin into English.

Critique a mathematical argument.

Design an experiment to test a hypothesis.

Describe monuments of art in historical context.

Design a performance environment for the production of a play.

Use tools of economic theory to analyze an economic problem.

Components

High-level outcomes provide a sense of mission and overarching goals but are difficult to assess.  Specific outcomes are easier to assess but fail to communicate an overall purpose or direction.  A useful strategy is to define one or more high-level outcomes for the major and then to define specific components.  Each component can become a category in a rubric or a separate task to be assessed.

Example:

Economics majors should be able to develop an economic analysis. Specifically, students should be able to:

Formulate a problem for economic analysis;

Find relevant literature that bears on the problem;

Interpret quantitative evidence and regression analyses;

(etc.)

Quality Standards

Quality standards for performance are spelled out in rubrics (see "How to Assess" below).


Tips:

  • Define learning outcomes for every major offered by the department or program. There is likely to be overlap, but there should also be differences that distinguish the majors.
  • Focus on a few key outcomes that are expected of all students and that will be assessed. Students should go beyond these outcomes in individual ways.
  • Avoid duplicating GER outcomes. For example, "Students will be able to follow disciplinary conventions when writing or speaking" is a GER outcome that can be assessed along with major outcomes when evaluating a Senior Experience.
  • State outcomes with just one action verb and one object. For high-level outcomes, define the components.
  • State outcomes plainly so that students, faculty, and others can understand what they mean and agree when they have been achieved.

+ How to Prepare a Curriculum Map

Map Outcomes to Requirements

Once learning outcomes have been defined, prepare a curriculum map that shows how the outcomes map onto major requirements.  The simplest way to do this is to make a table or spreadsheet with outcomes as rows and courses as columns.  Indicate in which courses the outcomes are addressed and at what level: introductory / foundational / advanced - or - knowledge / application / synthesis, etc. The curriculum map will help you identify assessment points as well as gaps in the academic program.

Click here for a Curriculum Map Template (XLS).

+ How to Assess

This section describes how to make a rubric, use it to assess student work, and pool findings for analysis.


Make a rubric

For each outcome, create a rubric that describes performance at different levels of development.  Make a separate rating scale (column in the rubric) for each component.  Decide on a number of levels (3, 4, or 5) and a label for each level (example: 1 = initial, 2 = basic, 3 = proficient, 4 = advanced).  Make it clear which level is the target (example: 3 = proficient).  Provide a description of performance at each level so assessors can rate work consistently and so students can use the rubric to check their own progress.

How to write a rubric

  1. Write a description for the "proficient" level first. This should clearly describe achievement of the outcome.
  2. Write the "advanced" level next, describing characteristics of more expert performance. This gives students a higher level to strive for and gives assessors a way to identify outstanding or exemplary performance.
  3. Write the "basic" level, where characteristics are emerging but not yet mastered. This helps students identify how they are progressing.
  4. Write the "initial" level, describing the foundation (prerequisite knowledge or skill) on which the outcome is built. This gives students and faculty a starting point for building the outcome.
  5. Tweak the descriptions to clearly distinguish each level from the one just above and below, forming a progressive scale from novice to advanced.
  6. Edit descriptions to make them as concise as possible, and highlight the text that distinguishes each level. This will make it easier to use the rubric to rate student work.
  7. Visibly indicate which level is the target for students in the major.

Click here for a Rubric Template (DOC).

Sample rubrics:

Lawrence's GER rubrics

AAC&U VALUE rubrics

AALHE sample rubrics


Tips:

  • Give precise characteristics of performance at each level; avoid sweeping statements like "excellent," "average," or "poor."
  • State what is evident as well as what is lacking; avoid defining a level solely in terms of deficits ("lacks X, Y, Z... ").
  • Limit the criteria at each level; focus on the essential.
  • Contrast adjacent levels to make sure they distinguish performance well.
  • Avoid recapitulating the GER writing/speaking rubric. You can use that rubric alongside a rubric focused on major outcomes.
  • Test the rubric with samples of student work and check for consistency among raters. (Also do this for training before rating student work.)

Rate student work

To assess how well students are achieving intended outcomes for the major, use the rubric to rate Senior Experiences (culminating projects or portfolios) or other required work.  Mark the level of achievement on the rating scale and jot comments on strengths and areas for improvement. This can be done for individual work, a collection of work, or samples from a larger body of work. The rubric can be incorporated into regular course grading or used separately to rate samples.

To identify where knowledge and skills emerge or to define a baseline for seeing how much students improve, use the same rubric to assess work earlier in the major, such as in a foundational course or assignment. Highlight the characteristics that result in the work being classified at a particular level, and comment on strengths and areas for improvement. Compare the findings to those for Senior Experiences.


Tips:

  • Focus: Select a subset of outcomes for each cycle of assessment if that helps you make better use of your resources.
  • Sample widely: Rate one item from many students to get a cross-section.
  • Divide the labor: Have one person rate major outcomes while another rates GER outcomes (writing/speaking or integrating knowledge or creative activity).
  • Check reliability: Have two or more assessors rate the same work samples/outcomes and compare. (This is good training and can stimulate useful dialogue.)

Pool your findings

Gather faculty assessments of student work, both quantitative (ratings) and qualitative (comments).  For each outcome, prepare a table or chart of ratings and a summary of comments on strengths and areas for improvement.

Click here for a sample of how to report assessment data [PDF].

Click here for a sample Excel tool for computing results, from the Studio Art department [XLS].


Tips:

  1. Try to have a direct measure (assessment of student work) and indirect measure (survey, focus groups, or structured interviews) for outcomes you are emphasizing in a particular assessment cycle. (Direct measures are essential; indirect are optional.)
  2. Use a chart or table to present the full range of data.  (Avoid giving just an average rating.)
  3. Look for patterns and compare to previous years.
  4. Relate your findings to your learning outcomes and curriculum map.

 

 

+ How to Use Findings

Assessment should stimulate useful conversations among department faculty.  Most important is that you use assessment findings to make improvements, large or small; otherwise, considerable effort will have been expended for little benefit.  Accreditors are mainly interested in how we "close the loop" by using assessment findings for program improvement.

Discuss findings

Gather your ratings and comments and discuss them as a department.  Make this a regular part of an annual department meeting or retreat.

  1. Look for evidence of strengths and weaknesses in the academic program.
  2. Look at your curriculum map to see where particular knowledge and skills are developed.
  3. Compare findings to previous years to identify trends.

Plan actions

  1. Consider possible changes to curriculum, instruction, or assessment.
  2. Plan surveys, focus groups, or student interviews to gather more information on areas of concern.
  3. Decide how you will evaluate the impact of any program changes.
  4. Document findings, discussion, and planned actions for your next assessment report.

Tips:

  • Tweak requirements, assignments, guides, etc., to close gaps in the academic program or strengthen areas of concern.
  • Refine your outcome statements, rubrics, and assessment methods to align them or to make them more useful and reliable (consistent across raters).
  • Identify earlier assessment points to gather developmental data or assess outcomes not addressed by the Senior Experience.
  • Shift focus as needed: Conduct follow-up studies or identify particular outcomes to probe in the next assessment cycle.

+ How to Report

Every department or program submits a biannual report summarizing assessment activities, findings, and planned actions.  The report says what you did, what you found, and what you're doing as a result.  It should be concise and forthright, written with the Provost in mind.  Your assessment report provides a record of progress for the department and an important source of information for the reaccreditation process. 

Prepare Report

Use the Department Report Template [DOC] to prepare your report.  Fill in each section with a brief paragraph or two as directed.  Include a table or chart of data (without student names).  Attach any updated items: learning outcomes, curriculum map, or rubrics.

Once your report is drafted, use the questions in the Committee Feedback Template [PDF] to check the report for completeness.  (The Assessment Committee will use this template to prepare feedback on your report.)

Submit Report

When your report is complete, email it to the chair of the Assessment Committee, Jeff Collett, at jeffrey.a.collett@lawrence.edu.  Reports are due June 30 in odd- or even-numbered years.  Check the deadlines at the top of the page to see when your report is due.

The Assessment Committee will discuss your report and provide written feedback to help you improve your assessment or reporting practices before the next report is due.

Click here for an overview of the review process [PDF].)

+ What to Do After Reporting

Use Feedback

Use the Assessment Committee's feedback to refine your outcome statements, assessment methods and rubrics, and presentation of findings in your next report.  If you are considering follow-up studies for particular concerns, consult the Office of Research Administration or the Assessment Committee for help.

Archive

Be sure to save your assessment materials (learning outcomes, curriculum map, and rubrics), reports, and feedback in your department share space so they are available to the department for follow-up studies and future reports.

Facebook Twitter Instagram YouTube LarryU