A distinctive feature of the Lawrence Fellows in the Liberal Arts and Sciences program is that program evaluation is built in from the beginning, creating a longitudinal study of the program's outcomes and benefits for Fellows, students, faculty, and the University.

+ Read More

Whenever possible the program evaluation:

  1. involves multiple measures to assess goals and outcomes,
  2. uses repeated measurements over time (e.g., to establish progress made by Fellows),
  3. relies on objective indicators as well as subjective self-reports,
  4. when subjective assessments are used, employs the same measuring instrument, independently completed by different people, to control for individual biases (e.g., the same teaching evaluation form will be completed by students, mentors, independent observers, and Fellows - as a self-assessment - to yield a reliable and valid assessment of Fellows' teaching),
  5. uses appropriate comparison groups (e.g., comparing student ratings of Fellows' teaching to student ratings of tenure-track faculty) to assess the program,
  6. is conceived and administered by faculty (with administrative support) rather than solely by administrators.

Because many of the program's goals involve the impact of the program on specific constituencies -- students, Fellows, faculty, departments -- many outcomes are necessarily assessed by surveying individuals in these categories. As noted above, however, measuring independent perspectives on specific outcomes enhances the reliability, validity, and credibility of these assessments. Subjective assessments are also supplemented by objective or documented indicators. For example, the impact of the Fellows program on research opportunities for students is assessed by reports from Fellows, from students, from Department Chairs, but also through such documented outcomes as independent study and tutorial courses, joint publications, and conference presentations involving Fellows and students. Narrative reports are used to inform and deepen the interpretation of these more objective indicators. For example, a student's own assessment of how a research opportunity affected his or her skills and career plans enriches understanding of the quality of outcomes and supplement such evidence as joint papers or presentations.

The program evaluation also assesses long-term outcomes for the Fellows. Because a central aim of the program is to provide the Fellows with a transition between graduate school and an academic career that prepares them to be successful, it is important to assess Fellows' progress and accomplishments not only while they are at Lawrence, but long after they leave.

 

Desired Outcomes and Measures

The desired outcomes of the Fellows program are listed below. Goals are divided by the various constituencies they are most strongly associated with (though, of course, many of the goals address multiple constituencies).

Outcomes for Fellows and Academia

+ Developing Fellows' Excellence as Teachers

The Fellows program includes not only the opportunity for Fellows to gain teaching experience in a liberal arts setting, but rather than simply throwing Fellows into the classroom to sink or swim, we aim to support and mentor them. The support takes the form of peer discussion groups, mentoring by established Lawrence faculty members, opportunities for the Fellows' classes to be observed by others , and co-teaching with Lawrence faculty members.

+ Assessment of Teaching Excellence:

The standard teaching evaluation form is used for ratings of Fellows made by their assigned faculty mentors and by department Chairs (as part of their annual reports). Questions from this form are also incorporated into the Classroom Observation Form, to be completed each time the mentor observes a Fellow's class Fellows will also complete the standard course evaluation at the end of each term.

Thus, although subjective assessments of teaching are a major source for documenting the quality of Fellows' teaching and their progress over their two year appointments, consistent with the program evaluation's general principles, these assessments will be made on the same forms by independent raters representing various constituencies and levels of expertise. Developing Fellows' Excellence in Scholarship/Performance

+ Developing Fellows' Excellence in Scholarship/Performance

The Fellows program provides a position that promotes the development of Fellows as independent scholars and performers by allowing them sufficient time to devote to professional activities and, in many cases, by linking them to established faculty scholars at Lawrence who share similar interests. In contrast, tenure-track hires at small colleges typically face a much higher teaching load that forces them to devote most of their energies to developing classes at a crucially important time in their scholarly development (after having left the resources, support, and specialized focus of graduate training). A lower teaching load, absence of formal academic advising duties, lessened pressure to teach lower-level classes (that intersect less with scholarly interests), and absence of committee assignments are structural features of the Fellows position designed to provide fertile soil for Fellows' development as scholars and performers. Further, the Fellows Program encourages overlap of scholarly interests with current Lawrence faculty (a luxury that cannot typically be afforded for tenure-track hires in the small departments of liberal arts colleges) affording more possibilities of collegial support and mentorship, as well as scholarly collaboration.

+ Assessment of Scholarly Excellence:

A primary means of assessing scholarly excellence is through documentation of what the Fellows have produced (e.g., published articles, books, performances) and the quality of those contributions.. In addition to the quantity of what the Fellows produce, its quality can also be judged, in part, by the prestige of the outlets for Fellows' work. Accepted ratings of a journal's or publisher's quality (e.g., impact and citation ratings of journals, prestige rankings of publishing houses) or of the standards of a recording label provide indices of quality. Additionally, the annual reports document any grants or awards or similar indicators of scholarly recognition and contribution.

+ Fostering Fellows' Long-term Career Success and Contributions to Academia

The ultimate goal of the program with respect to Fellows and the wider academic world is to foster the success of promising young academics so that they become excellent contributors to their fields and the institutions that subsequently hire them. The Fellows position has been designed to promote success by helping Fellows to make a smoother, less frustrating transition into an academic career (especially at liberal arts colleges). Thus, an important measure of the overall success of the program is the long-term success of the Fellows as academics and a lower “attrition rate” (leaving the academy either due to frustration with the demands of academic jobs or inability to secure a tenured position).

+ Assessment of Fellows' Long-term Career Success and Contributions as Academics:

We follow the careers of Fellows after they leave Lawrence, asking them occasionally to update us on their current status, much in the way that Alumni Offices track alumni.  We believe that while it is not realistic to expect former Fellows to complete long questionnaires after they have left Lawrence, they will be willing to respond to an occasional email requesting their current vitas.

Outcomes for Lawrence Students

+ Enrichment of the Curriculum

The Fellows program provides an added pool of teaching faculty on the Lawrence campus whose courses will enrich the curriculum. Not only are the Fellows expected to have a direct effect by offering courses and topics that otherwise might not be taught, but their presence is designed to have indirect effects by freeing up tenure-track faculty to offer courses that they may not otherwise have been able to teach (when a Fellow takes over an existing course). Fellows are also expected to sponsor individualized instruction opportunities (in addition to their prescribed course loads) in Independent Studies (ISs) and tutorial classes. The end result is greater curricular diversity and, as another benefit, an ability to maintain small class sizes across the University (given the net increase in courses taught) than would otherwise be the case. (Because the Fellows program at Lawrence has begun at a time of sharply increased overall enrollment, its contribution may be to maintain lower class sizes when they otherwise would have risen.)

+ Assessment of Impact on the Curriculum:

The course schedule is all that is required to document the courses Fellows offer, which can be considered to be net additions to the curriculum.

Individualized instruction can be documented by reports from the Registrar on ISs and tutorials sponsored by Fellows. Fellows will also be asked in their annual reports to note any individualized instruction they offered that was not offered for credit and therefore not "on the books."

Finally, to assess the overall benefit of Fellows' presence on class sizes at Lawrence, data on enrollment kept by the Registrar can be used to track trends in class sizes within departments and across the University. Departments who have Fellows may service more students, but hopefully with fewer students per class than would otherwise be the case.

Increased Research and Performance Opportunities

Fellows are expected to increase the number of opportunities students have to engage in research or performance collaborations. Some of these collaborations may occur through the ISs and tutorials mentioned above.

Outcomes for Faculty

+ Faculty-Fellow Research and Teaching Collaborations

Because it is likely that there will be greater overlap of professional interests between Fellows and Lawrence faculty than commonly occurs among the faculty, we expect to find frequent collaborating between Fellows and tenure-track faculty in the area of scholarship and creative activity.  Similarly, with overlap between Fellows' and faculty members' interests, co-teaching is a likely outcome (and is also encouraged to help develop Fellows' teaching skills by working with an experienced member of the faculty).

+ Assessing Faculty-Fellow Research and Teaching Collaborations:

Research and teaching collaborations between Fellows and faculty is documented through the annual reports of Fellows, department Chairs, and mentors. Joint publications and performances by faculty and Fellows provide objective indicators of the amount of professional collaboration (which could be compared to the frequency of such collaborations among tenure-track faculty). Given that we encourage a greater degree of overlap of interests between Fellows and faculty than we do among the tenure-track faculty, there ought to be a significantly greater number of Fellow-faculty collaborative efforts than is normally the case among the faculty.

Subjective assessments of the impact and quality of these teaching and research collaborations are also solicited from the Fellows, faculty members, and students involved in them (as well as from faculty more generally -- Brief Faculty Survey.Narrative comments concerning these collaborations from the individuals involved in them l provide insight into the effects of these collaborations.

+ Faculty Members' Opportunities to Teach Advanced Courses and to Provide Individualized Instruction

The presence of the Fellows should create more flexibility in terms of what Lawrence faculty are able to teach (by providing an extra pool of teaching faculty). This, in turn, may enable Lawrence faculty to teach advanced courses they otherwise would not be able to make room for in their schedules or to devote more time to individualized instruction.

+ Assessing Faculty Members' Opportunities to Teach Advanced Courses and to Provide Individualized Instruction:

Documentation of changes in what departmental faculty have been able to teach (including ISs and tutorials) due to the presence of Fellows will come from department Chairs' reports (because Chairs understand what courses faculty members would have had to teach if a Fellow had not been present). This question can also be asked in a brief faculty survey.

 

Summary of Evaluation Plan Goals and Measures

Benefits to Fellows and Wider Academic Community

+ Developing Excellent Teachers

  • Teaching evaluations by students (each class, each term) - standard form
  • Teaching evaluations by co-teachers - standard form plus supplement
  • Self-assessment by Fellows - standard form plus supplement

+ Developing Excellent Scholars/Artists

  •  Annual Reports

+ Fostering Long-Term Career Success and Contributions

  • Tracking of Fellows' subsequent careers (Fellows Follow-up Survey)

Benefits to Students

+ Enrichment of Curriculum

  • Courses, Tutorials, and ISs taught (Fellows Annual Reports, Registrar)
  • Direct effects, such as new areas covered (Department Chair Reports)
  • Indirect effects on what other faculty offered (Department Chair Reports, Faculty Annual Reports)
  • Class size data (Registrar)

+ Increased Research and Performance Opportunities

  • Fellows-student collaborations (Fellows Annual Reports, Department Chair Reports)
  • Documentation of products of collaborations (Fellows Annual Reports)

Benefits to Lawrence University

+ Academic Reputation and Recognition

  • External opinion about Lawrence (Graduate School Advisor Survey)
  • Media coverage of Fellows Program (Communications)
  • Inquiries about Lawrence (other than from applicants) connected to Fellows program, requests for presentations (Administration, Director)

+ Intellectual Excitement on campus

  • Documented presentations, performances, and other campus contributions by Fellows (Fellows Annual Reports, Department Chair Reports)

Benefits to Faculty

+ Research and Teaching Collaborations

  • Documentation of collaborations (Fellows Annual Reports, Department Chair Reports, Mentor Reports)

+ Faculty Opportunity to Teach Advanced and Individualized Courses

  • Documentation of changes in what department faculty teach (Department Chair Reports, Faculty Annual Reports)

 

Facebook Twitter Instagram YouTube LarryU