DEPARTMENT OF SOCIOLOGY
ASSESSMENT REPORT
April, 2003
Our Mission: The Department of Sociology at Virginia Tech is
committed to the discovery, integration, dissemination, and application
of the principles that govern human relations and social life. We serve the
university, the citizens of Virginia, the nation, and the global community as a
center for the advanced study of social institutions, social processes, and
societal change. We believe in a
sociology engaged in the real world, employing a diversity of methods and
perspectives in the quest for greater understanding of, and solutions to,
critical issues in social life.
Program Goals and Objectives of the Undergraduate Program:
At the undergraduate level, The Department of Sociology offers a BS degree. Students may choose an emphasis area in crime/deviance, social inequality, or workplace relations. Our program is designed to provide students with a basic knowledge of sociology; to give students an opportunity to obtain more specialized knowledge in an area of concentration; and to develop students' ability to critically analyze social structures and processes and to apply sociological concepts, principles, theories, and methods to practical problems. In addition, students are taught the basic techniques of quantitative and qualitative research methods, and the knowledge to critically evaluate research findings.
Our department has formulated the following goals for undergraduate majors:
1. To gain a basic knowledge and understanding of sociology as it is currently practiced.
2. To be able to critically analyze social structures and processes as they impinge on oneself, on one’s immediate environment, on whole societies, and on the human species globally.
Undergraduate Program Assessment Activities
The
department has implemented an assessment plan designed to provide measures of
the overall functioning of the program as well as specific learning
outcomes. Assessment tools include: a
departmentally administered alumni survey, the post-graduation employment and
salary reports provided by career services, the
Virginia Tech Survey of Degree Candidates, exit interviews of graduating
seniors, the Virginia Tech Senior Survey, a survey of employers, special
multi-faculty assessments of particular student learning objectives and other
departmental data.
Survey of Sociology Alumni. In January
2003, the department mailed a survey (attached) to 180 alumni of our undergraduate program. Thirty-four completed questionnaires were
returned for a response rate of 19 percent.
Given this low response rate, conclusions drawn from these data should
be viewed with caution.
OUTCOMES
Employer Survey. Many of our majors enter graduate or professional school after graduation. These students often obtain jobs unrelated to sociology in order to support themselves during their advanced studies. Those who seek employment immediately after graduation do not go into any one dominant line of employment, making it difficult to contact employers. As a result, and out of concern for the privacy of our graduates, we will seek feedback through a two-step process. First we will include an item on our alumni survey asking for permission to contact the respondent’s employer. We will then contact the employers of those from whom we receive permission. We will ask the employer to assess the ability of the alumnus on the following dimensions: writing, speaking, and teamwork. Our criterion for success is for employers to rank at least 75% of our students satisfactory on these dimensions.
Outcome. One of our objectives is for our students to develop knowledge and skills that enhance employability. Data from Career Services suggests, at least indirectly, that this goal is being achieved. For the year 1999-2000, six months after graduation, 50 percent of sociology graduates were employed. This is compared to 48.9 percent for the college as a whole. Another 13.6 percent were in graduate school and 6.8 percent were in part-time work. However 29.5 percent were looking for work, somewhat higher than the college average. The 2000-2001 survey also shows a 50 percent employment rate, but a much higher rate of graduate school attendance (21.4 percent). In fact no respondents in the 2000-2001 were looking for work six months after graduation, significantly lower than the college 4.8 percent average.
According to our departmentally administered alumni survey, alumni were convinced of the usefulness of their degree in the workplace. In answering the question, “In general, how useful has the sociological perspective gained from your major been in your chosen field of work?” Seventy-six percent of responses were in the 1-2 range on a 5-point scale, with 1 being very useful and 5 not at all useful. One hundred percent fell in the 1-3 range, with no respondents saying that the sociological perspective was “not useful”. Likewise, 88 percent of those who attended graduate or professional school ranked the usefulness of sociology in the 1-2 range, and 94 percent fell into the 1-3 range on the five-point scale.
As noted above, respondents were also asked to give us permission to contact their employers for feedback. A permission form approved by university council was attached to the questionnaire. None of the respondents gave consent. We then contacted a large social services agency that we believed has hired some majors over the years. We asked the Director to assess the major strengths and weaknesses of our students. While it was very difficult to take time out of her schedule to provide such an assessment, the Director did provide the following statement:
VA Tech’s graduates have a comprehensive education. The rigorousness of the educational programs at Tech also guarantees that graduates have the ability to work through problems and stick with a task to completion. Both are assets in employees. The graduates we typically see are those in Psychology or Sociology, Public Policy or similar studies. These students seem to have a good grasp of current events and their impact on human services, which is helpful in working in Social Services. I have seen no weaknesses that were ascribable to being a Tech graduate.
Internship. Alumni were also asked about their internship experiences. Sixty-two percent had completed an internship as part of the degree program. Of these 64 percent found the experience very useful or useful and 27.3 percent found it moderately useful. While good, we expect these numbers to improve based on a revamping of our internship program in the past year. This has included placing internship under a new instructor, and expanding the range of opportunities available.
All of these data, taken together, suggest that we are doing a good job preparing students for employment. Nevertheless, we are in the process of expanding our career mentoring within the department and cooperating more closely with career services.
GENERAL EDUCATIONAL OUTCOMES IN SOCIOLOGY
Respondents were asked to report, on a five-point scale, the degree to which they had achieved a number of educational outcomes as well as the degree to which they believed that those outcomes, if achieved, would be useful after leaving school. Table 1 below presents the results, with the scale collapsed into three categories (high, medium and low achievement and usefulness).
Table 1. Perceived usefulness and achievement of learning outcomes (percents).
This table indicates that students believe that they are achieving a number of learning objectives set forth by the department. They also believe that these objectives are useful or potentially useful after leaving school. The one area where achievement and perceived usefulness are relatively low is with the objective of developing research methods skills. This is not surprising since students often struggle with research methods. However it does point out the need to continue to work to strengthen their experience in this area and to “sell” students on the importance of research methods as potential job skills.
SPECIFIC LEARNING OUTCOMES.
Last year the department decided to assess in greater depth four specific learning outcomes from the list above, using multiple assessment methods. In the paragraphs below we state each learning objective to be assessed, the method for assessing that particular objective, including criteria for success, and the results of the assessment.
Objective 3: Students completing their BS in Sociology will be able to define and provide examples of good sociological theory and will be able to show that they can compare the strengths and weaknesses of various sociological theories.
Means Of Assessment & Criteria for Success: Students in Sociology 3104 (Social Theory) were given an essay question in which they were asked to define what is meant by good sociological theory and to compare the strengths and weaknesses of two theories. A 25% sample of essays was drawn and read by a panel of three sociology faculty members. We will judge that we have achieved our goal if 75% of these essays are judged “satisfactory” (grade of 7.5 or higher on a 10 point scale). In addition, alumni will be asked to assess their satisfaction with their training in theory.
Key criteria: Students should demonstrate the ability to:
1. Articulate why there is a need for social theory.
2. Select and use appropriate criteria to compare theories on
a. conceptual adequacy,
b. logical consistency, and
c. empirical testability
Results. A 25 percent sample of papers from Sociology 3104 was examined by a panel of three faculty members, none of whom were the instructor in the course. The mean rating for reviewer 1 was 8.25. No papers fell below the acceptable criterion. Reviewer 2 gave a mean score of 8.37. All papers were in the acceptable range. Similarly, reviewer 3 found all papers to be in the acceptable range, with a mean score of 8.5. Thus, these data suggest that we are meeting our goal in teaching theory. This conclusion is further supported by data from the alumni survey. Alumni were asked, “In general, how satisfied are you with your training in sociological theory?” Results indicate that 73.5 percent of respondents were very satisfied, and another 17.6 percent were moderately satisfied. Only 8.9 percent were dissatisfied.
Objective 4: Students completing their BS in Sociology will demonstrate a basic understanding of, and elementary skills in, social research methods and data analysis.
Means of Assessment & Criteria for Success: Students in the research methods class were given an assignment in which they are asked to take a research problem and design a study using appropriate methods. A 25% sample of projects was drawn and read by a panel of three sociology faculty members, none of whom were the instructor in the class. We will have achieved our goal if in 75% of these projects students are judged to have an adequate knowledge of quantitative and qualitative research design. .
Key Criteria: Students should demonstrate the ability to
Outcomes. A 25 percent sample of papers from Sociology 3204 was examined by a panel of three faculty members, none of whom were the instructor in the course. The mean rating for reviewer 1 was 8.0. Twenty percent of the papers fell below the acceptable criterion of 7.5. Reviewer 2 gave a mean score of 7.7. As with reviewer 1, twenty percent of the papers fell below the acceptable score of 7.5. Reviewer 3 found all papers to be in the acceptable range, with a mean score of 8.5. These findings suggest that we are meeting our goal in teaching research methods. However, there appears to be room for improvement. For example, one reviewer found that the students met the goals relating only to a quantitative study in (1) clearly defining a research problem; (2) creating appropriate hypotheses; (3) selecting appropriate methods for studying the problem; and (4) operationalizing key variables. However, he also noted that the students did not seem to have a good grasp of the meaning of control variables. This is something we will be working to improve on in the coming assessment period.
As a second method of assessment, we asked a question about research methods on our alumni survey. Alumni were asked, “In general, how satisfied are you with your training in sociological research methods?” Results indicate that 50 percent of respondents were very satisfied, and another 32.4 percent were moderately satisfied. Another 17.7 percent were dissatisfied. This suggests that a small but significant group of alumni did not have a particularly positive experience in research methods. We plan to analyze the data further to see if there is a relationship between satisfaction and the year the student graduated. However the low response rate to our questionnaire will make this difficult. Recently a number of changes have been made to the course in order to enhance learning. Exit interviews with graduating seniors suggest that these changes have improved the learning experience.
Objective 6, students completing their BS in Sociology will develop an appreciation and broader understanding of social science generally, so as to relate the subject matter of sociology to a larger body of knowledge.
The Department of Sociology at Virginia Tech offers Master of Science and Ph.D. graduate degrees. The first Master's degree in sociology was granted in the 1920s; its first Ph.D. was awarded in 1974. A solid grounding in theory and methods constitute the core of our graduate programs. Under our broad umbrella of interests having to do with issues of quality of life, inequality, and work and technology, students have the opportunity to specialize in such areas as health, crime/deviance, gender, race, aging, social organization, and culture. Master's students gain the necessary background for doctoral work, and may choose a thesis or non-thesis (applied) option. The following program goals have been established:
A student completing a Masters degree in Sociology at Virginia Tech should be able to:
1. Describe the fundamental differences between sociological approaches to understanding human behavior and the approaches taken by other social sciences (e.g., psychology, economics).
2. Use sociological theories, concepts, and principles to explain or interpret social events.
3. Critique the strengths and weaknesses of various theoretical traditions -- especially conflict/Marxist theory, structural-functionalist theory, feminist theories, and symbolic-interactionist theory -- concerning their presuppositions, concepts, issues, and empirical findings.
4. Be able to define and give examples of the following concepts: social organization, social class, culture, social inequality, social institutions, socialization, social norms, social roles, self/social identity, patriarchy, dialectics, social power, hierarchy, social deviance, etc.
5. Describe the basic differences in the purposes and procedures of quantitative and qualitative research methods.
6. Critically read, understand, and evaluate quantitative and qualitative sociological research published in mainstream sociology journals and in books.
7. Formulate and articulate answerable research questions and develop appropriate empirical procedures, both quantitative and qualitative, to answer them, so as to successfully conduct and report research for theses.
8. Be able to give examples of how individuals’ social location affects their life chances.
9. Be able to provide a sociologically-informed analysis of current social issues, including most of the following: race relations in the U.S.; gender relations and inequalities; global interdependence; crime in the U.S.; cross-cultural comparison of family systems; the effects of changing technology on the nature of work; the interdependence of the social institutions of the family, education, and work.
In addition to the above, a student completing a doctoral degree in Sociology at Virginia Tech should be able to:
10. Conduct independent sociological research.
11. Summarize and critique the major historical and recent theoretical positions and empirical findings in his or her major substantive area.
12. Use sociological concepts and principles to evaluate social policy.
13. Formulate and articulate answerable research questions and develop appropriate empirical procedures, both quantitative and qualitative, to answer them, so as to successfully conduct and report research suitable for publication in refereed journals in sociology.
14. Teach herself or himself additional methodological techniques -- either existing techniques not covered in their coursework or new techniques not yet created -- needed as a researcher, and to adapt or create research tools demanded by new circumstances of research.
Graduate Program Assessment Activities
The
department has implemented an assessment plan designed to provide measures of
the overall functioning of the program as well as specific learning
outcomes. Assessment tools include: a
departmentally administered alumni survey, exit
interviews of graduating students, and a survey of employers.
Survey of Sociology Alumni. In January 2003, the department mailed a
survey (attached) to 67 alumni of our graduate program. Fourteen completed questionnaires were
returned for a response rate of 21 percent.
Given this low response rate, conclusions drawn from these data should
be viewed with caution.
GENERAL EDUCATIONAL OUTCOMES IN SOCIOLOGY
Respondents were asked to report, on a five-point scale, the degree to which they were satisfied with a number of aspects of our program. Table 2 below presents the results, with the scale collapsed into three categories (high, medium and low satisfaction.
Table 2. Satisfaction with learning outcomes among graduate alumni (percents).
|
Learning Outcome |
Satisfaction Level |
|||
|
|
HIGH |
MEDIUM |
LOW |
TOTAL |
|
Effective
writing skills for work in our field |
71.4 |
28.6 |
0.0 |
100 |
|
Effective
speaking skills for work in our field. |
50.0 |
35.7 |
14.3 |
100 |
|
Critically
analyzing research in sociology. |
78.6 |
21.4 |
0.0 |
100 |
|
Defining
key problems in the field. |
64.3 |
28.6 |
0.0 |
100 |
|
Applying
quantitative methods to analyzing problems in the field. |
85.7 |
14.3 |
0.0 |
100 |
|
Applying
qualitative methods to analyzing problems in the field |
78.6 |
14.3 |
7.1 |
100 |
|
Preparing
for a career |
78.6 |
7.1 |
7.1 |
100 |
|
Understanding
the literature in your substantive area |
85.7 |
14.3 |
0.0 |
100 |
|
training in sociological theory |
85.7 |
7.1 |
0.0 |
100 |
|
Overall training in our program |
85.7 |
14.3 |
0.0 |
100 |
|
Professional development |
71.4 |
14.3 |
14.3 |
100 |
These data indicate a high degree of satisfaction with the program. The areas in which improvement will be sought are “effective speaking skills for work in our field” and “defining key problems in the field.”
SPECIFIC LEARNING OUTCOMES.
Last year the department decided to assess in greater depth two specific learning outcomes from the list above, using multiple assessment methods. In the paragraphs below we state each learning objective to be assessed, the method for assessing that particular objective, including criteria for success, and the results of the assessment.
Objective 3, Students completing their Master’s Degree in Sociology will be able to critique the strengths and weaknesses of various theoretical traditions -- especially conflict/Marxist theory, structural-functionalist theory, feminist theories, and symbolic-interactionist theory -- concerning their presuppositions, concepts, issues, and empirical findings.
Means of Assessment & Criteria for Success. Students enrolled in our graduate theory course will be given an assignment requiring them to critique the strengths and weaknesses of several theoretical traditions. A panel of three faculty members, in consultation with the instructor, will evaluate the assignments. The criterion for success is 75% of the assignments judged by the panel as “satisfactory.” Students should be able to:
1. Outline the major theoretical assumptions of each theory.
2. Be able to apply the conceptual tools of different perspectives to a range of sociological problems.
Exit Interviews with Degree Candidates. An exit interview will be conducted with each Master’s and Doctoral degree candidate. In that interview candidates will be asked to assess their own training and competence in the area of social theory. The criterion for success is that all degree candidates will express satisfaction with their knowledge of social theory.
Alumni Survey. At least 75 percent of our graduate program alumni who respond to a departmentally administered survey will answer “satisfied” or “very satisfied” to the question, “How satisfied are you with your training in social theory while in our program?”
Employer
Survey. We will seek permission
from graduate alumni who work in academic settings to interview their current
employer. Employers will be asked to
compare our former students to other faculty in their knowledge of social
theory.
Outcomes. Exit interviews uniformly suggest that graduating students are satisfied with their training in theory. In fact, students see their training in theory as the strongest and most enjoyable part of the program. Likewise, alumni survey data (see Table 2) indicate that 85.7 percent of our graduate students are very satisfied with their training in theory. Respondents were also asked to give us permission to contact their employers for feedback. A permission form approved by university council was attached to the questionnaire. None of the respondents gave consent. We then contacted an academic department that we knew had hired some of our graduates in the past. While expressing concerns about anonymity, the chair of the department made the following comments:
I could not be happier with the graduates of your program we have hired. All have proven to be excellent colleagues, willing to teach a variety of courses, participate fully in departmental and university governance matters and, of greatest importance to our institution, receive superb student evaluations. They excel in strong teaching.
Regarding areas of improvement, I do not know what the PhD program in Soc requires in methodology. Nor do I know what variation is allowed in a particular student's program of study. Some students may follow more of a qualitative track than others. Looking at the strengths of these faculty, they are stronger in teaching and theory than in quantitative methodology , than in gathering and analyzing data and in publishing in refereed outlets.
…I am very pleased with these folks, and I would hire them again in a heartbeat. In fact, I am more sure of them now than when I first hired them. But if you are looking at your curriculum, and based on a rather limited view…, you may want to look at how your students are mentored in the research and publications process.
These comments suggest that we are meeting our goal in the area of theory, but have more work to do in the area of quantitative methods and professional development.
Objective 11, Students completing their Doctoral Degree in Sociology will be able to summarize and critique the major historical and recent theoretical positions and empirical findings in his or her major substantive area.
Means Of Assessment & Criteria for
Success:
Students enrolled in our doctoral program will be required to successfully complete an oral exam in their substantive areas before an examining committee. The criterion for success is 75% of the exam performances will be judged by the committee as “satisfactory.”
Alumni Survey. At least 75 percent of our graduate program alumni who respond to a departmentally administered survey will answer “satisfied” or “very satisfied” to the question, “How satisfied are you with your training in your substantive area(s) while in our program?”
Employer Survey. We will seek permission from graduate alumni who work in academic settings to interview their current employer. Employers will be asked to compare our former students to other faculty with similar substantive areas in their knowledge of that area.
Outcomes. Two students took and passed oral and written exams in the past year. Faculty found that these students had a good grasp of material in their substantive areas. In addition, a third student recently received the Graduate School’s Outstanding Dissertation Award, suggesting an excellent knowledge of his substantive area. Results from the alumni survey (Table 2) show that 85.7 percent of our alumni respondents are very satisfied that they have a good grasp of their substantive area and another 7.1 percent are moderately satisfied. Taken together, these findings suggest that we are doing a good job of training students in their substantive areas.