91PORNA

• Units that have an academic program with a graduate degree component are reviewed at least once within a seven to ten-year period. The term “Unit” may refer to a single program or cluster of degree programs, or may span the offerings of an entire department, school or college. This allows for the range of organizational hierarchy, which may differ across the various colleges at 91PORNA.
• Units containing programs that hold specialized program accreditation may choose to submit the most recent self-study, action letter(s) and final report(s) from the accrediting organization and complete an abbreviated internal review report in lieu of conducting a full academic program review.
• The Office of Accreditation, Assessment and Learning (AAL) determines the schedule of reviews and coordinates the academic program review process. AAL works in close consultation with deans, department chairs, school directors, and program directors/coordinators of that unit to ensure that all deadlines are met and the process flows as smoothly as possible.  
• In general, the review process begins when the unit receives notification of an impending review from the Office of AAL. This notification takes place twelve to fifteen months prior to the anticipated campus visit.
• It is a best practice for units to keep the requirements of academic program reviews in mind throughout each year, given that the review requires ongoing data collection, tracking initiatives, and identifying and pursuing goals.

Orientation Meeting
After notifying units of an upcoming review via email, AAL schedules orientation meetings with representatives from the units that are up for review. These representatives include college deans, department chairs or school directors, program directors or coordinators, and any other relevant personnel. The purpose of this orientation meeting is to verify the appropriate type of review for the unit, discuss the review modality, funding, support, requirements, due dates, and to answer any additional questions. 

Self-Study Report
Each unit is required to develop a self-study report based on a template provided to the unit by AAL. The self-study report is a candid assessment of the program's strengths and areas of growth and involves a deep dive into seven key areas: program and department context; curriculum, assessment of student learning and program improvement; current students; academic support and career development opportunities; alumni; faculty, administration and staff; and facilities and resources. As part of the orientation meeting, AAL advises the unit under review how to obtain key data points needed to complete the self-study. In addition to completing the self-study template, units are required to complete a one to three-page executive summary.

Potential Reviewers List
Units are asked to submit a list of candidates for both external and internal reviewers. The external portion of the review team typically consists of two to three members, depending on the number and/or degree levels of programs grouped in the unit review. The unit is asked to prepare a list of six to ten potential external reviewers from faculty (including some from the industry, in certain circumstances) meeting the requirements listed in the reviewer template. AAL reviews the external reviewer candidates for any potential conflicts of interest using a rubric, asks the unit for prioritization of eligible candidates, and then shares the final list of possible external reviewers in order of invitation with the unit and the college dean. In some instances, AAL may go back to the unit to ask for additional reviewer candidates. To complete the team the unit under review, in collaboration with college leadership, submits three to five names of potential internal reviewers following the requirements listed in the reviewer template. AAL issues invitations to both external and internal reviewers and informs the unit when the review team has been confirmed.

Site Visit Itinerary
Travel and lodging arrangements for the external review team are coordinated by AAL staff. Arrangements for individual meetings during the site visit are coordinated by the unit with assistance from AAL for meetings involving members of the Provost’s office or college deans. Sample itineraries are provided which units can use as a guide to assist with planning meetings.

Review Materials
AAL creates a secure web-based file sharing space for all review materials at least four weeks prior to the site visit, with access provided to internal and external reviewers, the Provost’s Office, and other university stakeholders. Materials include the executive summary, self-study with appendices, review data, site visit itinerary, report guide, CVs of reviewers and other related information.

Site Visit
The site visit typically extends over a three-day period. Day one begins with an opening dinner that evening which includes the College Dean, the Senior Associate Provost and Dean of Graduate College, the Associate Provost of Institutional Effectiveness and the review team. Day two is filled with scheduled meetings and work time. Day three has meetings scheduled for the morning, with a final exit meeting immediately following lunch. If a review is being conducted remotely due to a program being entirely online or extenuating circumstances, the opening dinner is replaced with a virtual social/Q&A hour and meeting times are adjusted to accommodate for differences in time zones among the reviewers. The College that houses the unit under review will be charged for costs associated with the review process including travel, lodging and meals for on-site visits and honoraria for external reviewers. AAL processes payments and other reimbursements and then charges them back to the College. At that point, the College determines how to distribute the costs internally.

Review Team Report
The external review team generates a report within four weeks of the site visit. To facilitate this process, AAL provides an optional report guide to external reviewers with writing prompts.

Fact-Check
The review unit administrator and the faculty within the unit will conduct a fact-check of the Review Team’s report. The Review Unit Administrator collects any comments and compiles them into a report response using the provided template. If corrections to factual errors are relatively minor, the response is added as an appendix to the final report and is accepted. If corrections to factual errors are significant, the unit's comments are shared with the reviewers and they are asked to amend the report prior to final acceptance. The finalized report is then shared with key stakeholders (e.g., Provost, Senior Associate Provost and Dean of Graduate College, AAL staff, College Dean, Review Unit Administrator).

Action Plan
Once the external reviewer report and unit’s response have been received and accepted as final, AAL will email the Review Unit Administrator with instructions for completing the Action Plan Template based on reviewer feedback.

Action Plan Meeting
Once the action plan template has been submitted to AAL, the Associate Provost, Academic Affairs, Associate Provost of Institutional Effectiveness, the College Dean, and Review Unit Administrator will meet to discuss the unit’s action plan. 

Follow-Up Report
Three academic years after the review, the unit is expected to provide an update report regarding progress made toward goals outlined in the action plan. The unit will be sent a reminder notification and a report template by AAL at the start of the academic year when the follow-up report is due.