OUTCOMES ASSESSMENT REPORT

Department of Political Science

Bachelor of Arts Program

1997-98

Program Goals and Objectives: Political science is a highly diverse discipline divided into strongly defined "fields" in comparative politics, international relations, political theory, research methods, and American government. Each of these fields is, in turn, divided into distinct subfields such as political behavior, public policy, developing nations, or international law. Accordingly, students graduating with a BA in Political Science follow a highly diverse set of career paths. There is no dominant line of employment for our majors and no fixed set of organizations that hire most of them. A high percentage of our BA graduates obtain a graduate degree before seeking employment.

We do not seek to equip students with a single set of career-related skills or knowledge. Rather, our program is designed to provide students with foundation knowledge in all of the major subfields of our discipline, to give students an opportunity to gain greater knowledge of their chosen subfield, and to develop stedents' ability to think analytically about political issues and to effectively convey their analyses to others. We consider our undergraduate program successful if our majors (1) can be accepted into and perform well in high­quality graduate programs and professional schools and/or (2) find the knowledge and skills they acquired in our program useful in their chosen line of employment.

Assessment Activities:Our efforts to evaluate the degree to which we have reached these goals fell into the four categories described below.

Student Feedback: We sought the opinions of our students by distributing a questionnaire for graduating political science majors in all of the upperdivision courses we offered in the Spring of 1997. This survey asked students for their assessment of several aspects of our program, and was returned without the student's name. The results of this survey are reported below.

Alumni Feedback: We sought alumni feedback through a survey mailed to all POLITICAL SCIENCE majors who graduated in the past five years. We followed the original mailing with a follow­up mailing to improve the response rate. This process was completed by July 15, 1997. A copy of the undergraduate survey instrument is attached, and the results are presented below.

Employer Feedback: Since the BA in political science is not a terminal degree for most students, many of our majors enter graduate or professional school after graduation. These students often obtain jobs unrelated to political science in order to support themselves during their advanced studies. Those who seek employment immediately after undergraduate school do not go into any one dominant line of employment, making it extremely difficult to contact employers. With these facts in mind, and out of a concern for the privacy of our graduates, we sought employer feedback through a two-step process. First, we mailed all students who had graduated in the past three years a letter asking that they give an enclosed questionnaire and return envelope to their employer and ask that person to fill it out and return it to us. Since neither the graduate nor the employer were identified in the questionnaire, there was no way to attribute a given review to any individual former student. This process should have encouraged frank answers and a better response rate. (To cope with the issues of students going into graduate school or working at temporary jobs, we asked those who were in either of these situations to return the questionnaire to us rather than giving it to an employer so that we did not get an inaccurate picture of our graduates work careers.) This survey was completed by March 15, 1998, and the results are reported below.

Evaluation of Student Performance: To obtain external assessment of our students' performance, we will ask scholars at other universities to assess the knowledge of political science methods and substance reflected in papers written by all senior POLITICAL SCIENCE majors who are taking upper-division POLITICAL SCIENCE courses that require substantial papers in the Fall Semester of 1997. Because of the relative distinctness of the subdivisions of our discipline, we will need to select external reviewers in several subfields to appropriately judge the substance of our studentst work. We will use papers that are turned in near the end of the semester so we have no results yet.

Results: At this point, we have results from only three of our assessment efforts. The findings are presented below.

Student Feedback: The Political Science Department usually graduates between 70 and 90 majors a year. However, the graduating class of Spring, 1997 consisted of only 48 students. (The number will return to more normal levels this year.) This means that the 20 responses we received to our "Senior Surveytt represent 42% of senior Political Science majors at the time.

When asked about their occupational objectives, 30% of these seniors said they intended to enter the legal profession, 30% said they wanted to work for some level of government, 10% wanted to teach at the college level, 10% wanted to go into business, 10% listed a military career, and the rest listed a variety of fields (public relations, international organizations, etc.).

When we asked students how well they thought the political science program had prepared them for "life in general," 70% responded "very well," 20% selected "quite well" and 10% indicated "fairly well." No one choose "poorly" as a response. We also asked the 40% who indicated that they intended to go to graduate school how well they thought our program had prepared them for graduate studies. Of those who responded, 88% chose "very well" as a response and 12% (1 person) chose "fairly well." In all, the senior survey revealed no significant problems with the program.

Alumni Feedback: We mailed 320 questionnaires to students who had graduated in the past five years, 14 of which were returned as undeliverable. We received 92 completed forms for a response rate of 30%.

The responses revealed that our students had followed a variety of career paths. As expected, 63% of them reported having attended graduate school at some time since graduating from Tech. Of this group, 49% sought a legal degree, 18% a degree in public administration, 8% an advanced degree in political science or closely-related field (like public policy), 8% a Masters of Business Administration, and 18% sought a variety of other degrees. At the time of our survey, 24% of the respondents were still in graduate school and classified their occupation as "student." In addition, 27% were in "business," 12% worked for some level of government, 11% were in the legal profession, 6% were in the military, 5% were teachers, 1% worked in nonprofit organizations, and 15% listed other occupations.

The graduates' responses to our questions about their assessment of the program were generally quite positive. Only two areas raised concerns. First, 25% said that they did not feel that the program had prepared them well for their occupation. Since this answer came exclusively from those who had entered occupations not closely related to political science, we conclude that the problem is not with our program but with students' understanding of the nature and objectives of our degree program. The second area of concern is advising. We were surprised to see that 39% of these respondents disagreed with the statement: "The course advising in my department was satisfactory." Upon reading the comments they volunteered, we feel that our former students were not distinguishing between academic advising and career advising (even though the question specified "course advising"). One sign of this is that when we asked at the end of the questionnaire for any suggestions they might make to improve the program, 31% of those who answered said that they wanted more job and/or career counseling and internship opportunities. This comment came primarily from respondents who had also expressed dissatisfaction with advising in the earlier question. We will describe our response to these two concerns in the final section of this report.

Employer Feedback: We sent packets requesting employer feedback (as described above) to 200 individuals. Of these, 15 were returned as undeliverable, 22 were returned indicating that the former student was in graduate or professional school and could not get employer feedback of the type we were seeking, and 42 complete questionnaires were returned from employers. This means that 121 of the questionnaires are unaccounted for. The few responses we received, yield the following highly positive results.

We first asked employers to rate the degree to which our graduates' exhibited a set of skills. The responses were as follows:
Written communication 52%48% ---- --
Oral communication 76%24% ---- --
Systematic reasoning 67%33% ---- --
Critical thinking 65%33% 2%-- --
Understanding quantitative information 62%24% --2% 10%

It is worth noting that 38% of the employers rated our graduates as "excellent" in every area.

We next asked employers to assess (1) the degree to which the graduate's job required a knowledge of public affairs, (2) the quality of the graduate's understanding of public issues, and (3) how well the graduate's college education had prepared him or her for employment. The results are as follows:

Knowledge of public affairs required by the job:
A great deal 52%
Some29%
Very Little 19%
Not at all --

The graduate's understanding of public affairs:
Excellent 48%
Very Good 43%
Fair--
Poor--
Can't Judge 19%

College experience prepared the graduate for employment:
Quite Well 57%
Well33%
Fairly Well 10%
Poorly --
Can't Judge --

Finally, we asked employers to identify any deficiencies in the former student's performance that could have been corrected through college coursework. Only one person indicated any deficiency, writing "Analytical thinking and problem solving."

Conclusions and Responses

The overall results of our assessment have been highly positive and do not suggest the need for adjustments to our program. These results are very similar to those obtained in our last outcomes assessment in 1992-93. The one area that does seem to require attention is the area of career preparation and advising. In response to these findings, we will take the following steps in the Fall of 1998.

First, we will separate the jobs of Internship Coordinator and Career Advisor which have been done by the same individual. With two people doing these tasks, each will receive more attention. Second, we have arranged for a faculty member to have a course release each year to further develop our internship program so that more students can have career-relevant internships. Third, we will redouble our efforts to inform students who declare political science as a major of the nature of our program so that they do not have false impressions about the degree to which it entails job preparation, or the kind of employment they are likely secure with this degree.

Eor additional information, please contact Richard C. Rich at 231-6571 or "urban@vt.edu."