Department of Foreign Languages & Literatures
Outcomes Assessment Report
April 1, 2002
Prepared
by Terry L. Papillon,
Assistant
Chair of the Department and Associate Professor of Classics
Department
Chair: Judith L. Shrum
Part I. Introduction
The Department of Foreign Languages &
Literatures works under the following mission statement, and the assessment of
its activities is carried out with a view to its priorities:
The Department of
Foreign Languages and Literatures provides students the opportunity to gain a
knowledge of the culture, literature, structure, and vocabulary of the diverse
languages offered. The department seeks to graduate majors and minors with an
advanced command of the written and spoken language and with the general
analytical, critical and communicative skills and global outlook essential to
an educated member of society. The department fulfills a service mission
through offering foreign languages for students who must meet the foreign
language requirement, for students interested in language study for
intellectual and cultural enhancement, and for students and faculty in other
disciplines for whom a knowledge of foreign languages and cultures is a
valuable professional tool. In all cases, the department strives to broaden
students' awareness of the nature of language and of cultural and national
diversities and similarities. It also strives to break down the barriers of
ethnocentrism and xenophobia, thus increasing communication and understanding
on an international scale. The department strongly promotes the scholarly and
pedagogical activities of its faculty, who advance, preserve and transmit the
collective knowledge of the discipline through research, publication, teaching,
and participation in professional life.
The categories for evaluation rose from the
work of an ad hoc committee during the spring semester 2001 (see Appendix 2 for
the report of that committee) and the actual assessment occurred during the
academic year 2001-2002. The categories of assessment include:
Preparation
for graduate school
Oral
and Written proficiency in the target language
Preparation
for global/interdisciplinary careers
Technological
literacy
Critical
thinking skills
Ability
to use cultural knowledge
Service
to non-major students through the core
Service
to non-major students outside core courses
Quality
of teaching
Instruments used to collect data include:
Senior
oral exit interviews, conducted during May 2001 by the Department Chair
Alumni
surveys, conduced in fall 2001. 26 surveys from graduates from 1998-2001.
Oral
proficiency interviews of ten graduating seniors done by nationally certified
ACTFL examiners in phone interviews, conducted during spring 2002.
Phone
interviews with four employers conducted by the Assistant Chair of the
Department about recent graduates who had given permission for such interviews.
SPOT
form data[1]
taken from a random sample of 12 courses from 1998 and 2001.
Paper
survey of degree candidates during fall 2001,
prepared by the Academic Assessment Program and conducted by Department
faculty.
See Appendix 2 for how each instrument related to
each category. In some cases, more instruments were used than were called for
in the preliminary report (Appendix 2), simply because they were available. The
Senior Exit Interviews and employer phone interviews, for example, offered
information on most of the topics, and so they are included in each section. In
other instances, not all instruments suggested were available and others had to
be substituted with other appropriate measures.
In addition to the above measures, it should also
be pointed out that the Department’s requirement of a minimum grade point in
the major courses ensures a basic level of competency. The Department also requires that every student in the major
demonstrate an oral proficiency level as set by the American Council on the
Teaching of Foreign Languages (ACTFL). This verification traditionally happens
at the point of transition from lower division to upper division coursework.
Meeting the level of “advanced low” ensures that the students have the basic
language skills in speaking and listening to continue with the coursework for
the major. This verification happens in two ways: a student can pass the Oral
Proficiency course (French/German/Spanish 3126) which includes an interview
with a faculty member trained in giving the Oral Proficiency Interview or be
exempted from this course based on an oral interview with a faculty member.
Part II. The Assessment
According to the Alumni survey, 29% thought their
preparation for graduate school was EXCELLENT, 33% said it was SATISFACTORY, 4%
said NOT SATISFACTORY, and 35% said it was NOT APPLICABLE to them. Thus, 94% of
those who answered the question thought that their preparation was satisfactory
or excellent.
Question 8 in the Academic Assessment Survey, which
surveys the topic “Prepare for graduate school,” had the following responses:
8% said "did not achieve," 42% said "achieved to a small
degree," 17% said "achieved to a moderate degree," 33% said
"achieved to a large degree," and 0% said "definitely
achieved." Of the 12 respondents
on this survey, only one said they were planning to attend graduate school in
the fall, though 6 said they would attend within the next five years.
In the Senior Exit Interview Report (Appendix
3), the Department Chair noted the following: “They felt adequately prepared in
writing but not in speaking and wished for more emphasis on speaking at the
4000-level.”
According to the Alumni survey, 40% thought their
proficiency was EXCELLENT, 56% said it was SATISFACTORY, 4% said NOT
SATISFACTORY, and 0% said it was NOT APPLICABLE to them.
The Assistant Chair of
the Department selected at random 10 seniors graduating from December 2001 to
August 2002; of these, 8 were majors in FL and 2 were not. These 10 students
were given an Oral Proficiency Interview
(OPI) via telephone by an external examiner who is certified by ACTFL. On the
ACTFL scale from novice low to superior, students need to be at the 'advanced
low' level (7 on a 10 point scale as reported here) to pass their oral
proficiency level at the beginning of upper division work (see comments on oral
proficiency requirement above). At the time of graduation, the 10 students
surveyed had an average score of 7.5, or half way between 'advanced low' and
'advanced mid' when they graduated. The 8 majors, however, averaged 8.1, or
slightly above 'advanced mid.' In the 1997 Outcomes Report we only reported
outcomes for professor-examined students in the teaching option. This year we examined majors and non-majors
using ACTFL certified external examiners.
While direct comparisons are not possible, we have a more reliable
measure of performance showing that the outcome performance is within our
expected range.
In phone
interviews with employers of
recent graduates, on the question of oral proficiency, not all graduates are
involved in occupations that use their languages, but for those who were the
responses were that employees were 'above average' or 'far above average' in
their abilities.
In
the Senior Exit Interview Report
(Appendix 3), students asked for more upper-division classes in which speaking
could be developed even further, either a 4000-level oral proficiency class, or
reorganization of upper-division courses to allow for less teacher-centered
lecture formats and more interactive student participatory activities. Some
sections and classes have already begun to move in this direction.
Of
the 26 graduates who returned alumni
surveys, 24 said that study abroad should be required. Of the 2 who said it
should not, one had studied abroad and one did not indicate whether she/he had
studied abroad. Students talked about the importance of this for proficiency
and cultural knowledge.
Preparation for global/interdisciplinary careers
According to the Alumni survey, 27% thought their preparation for global or
interdisciplinary careers was EXCELLENT, 62% said it was SATISFACTORY, 12% said
NOT SATISFACTORY, and 0% said it was NOT APPLICABLE to them.
In phone
interviews with employers, on
the question of global or interdisciplinary preparation, the responses were
that employees were usually 'above average' in their abilities. One new
employee is being nominated for the 'best first year teacher' award because of
his work with other faculty and resource people in the school district.
In the Senior
Exit Interview Report (Appendix 3), the Department Chair offered the
following comments: Students felt well prepared in all areas except speaking,
and here they noted a stark difference between what they study and what their
careers will expect of them. Students asked for more courses in technical
writing related to their field, a class on current issues in the target country
in general as well as courses with targeted information related to
agricultural, business, and engineering fields. Students were well aware of the
difference between learning language in instructed settings and using it in
real settings. They felt unprepared for
the real settings and suggested more such opportunities, accompanied by
contextualized courses to reinforce their understanding of language structure,
either linguistic or grammatical.
Technological literacy
According to the Alumni survey, 27% thought their technological literacy was
EXCELLENT, 62% said it was SATISFACTORY, 12% said NOT SATISFACTORY, and 0% said
it was NOT APPLICABLE to them.
In the Senior
Exit Interview Report (Appendix 3), the Department Chair offered the
following comments: Students felt that their some of their classes in this
department were more high-tech than some of their engineering classes. They enjoyed faculty use of web chats,
international videoconferencing, grades on line, on-line courses, research
links on line, use of videodisc programs, writing forums, and electronic
portfolios. They appreciated the distance learning courses but found it
difficult to engage in peer editing using that format. They liked having
insights into the research conducted by faculty, citing the case of one
professor who put a catalogue of language change on line as a result of his
research. They said that the student computer lab available in the department
removed a barrier for them, making it easier to complete technologically
related assignments right here where the faculty members are also available for
help if needed.
According
to the Alumni survey, 46% thought
their critical thinking and problem solving skills were EXCELLENT, 50% said it
was SATISFACTORY, 0% said NOT SATISFACTORY, and 4% said it was NOT APPLICABLE
to them.
In phone
interviews with employers, on
the question of critical thinking skills, the responses were that employees
were consistently 'above average' or 'far above average' in their abilities.
One employer commented about how the employee 'always digs, going to other
people in the county as resources for situations that need to be treated.'
Another employee was meant to be an intern for two years, but her advisor left
for another job, and so she has had to cope with learning her job herself and
was commended for being "a fighter" and finding out what she needed.
A third, new teacher, finds herself preparing for a move from one building to a
split school in two buildings; the principal praised her work dealing with such
a shocking change in just her first year.
In
the Senior Exit Interview Report
(Appendix 3), the Department Chair offered the following summary: Students felt
that 4000-level literature classes were helpful in showing them a step-by-step process
to look deeply into a literary work and analyze it. In addition, writing
intensive classes were helpful in showing them how to construct and edit
quality arguments in a comprehensive essay. German students pointed out that
they were still operating at the level of comprehension, unable to move toward
analysis, and that they relied on the professor more than they wished,
resulting in more lecture format than their preferred discussion format. French
students asked that the selection of literary topics be modernized a bit,
focusing on political systems, culture, popular press, the target country
today, current issues. Spanish students liked the recombination of
Civilization/Culture/Literature courses at the 3000-level. Spanish students
also asked for more discussion format in some classes that are largely
teacher-centered.
In the Academic Assessment Survey, question 2,
which surveys the topic “Improve Intellectual Skills (critical thinking,
logical reasoning, problem-solving ability)” had the following responses: 0%
said "did not achieve," 0% said "achieved to a small
degree," 25% said "achieved to a moderate degree," 33% said
"achieved to a large degree," and 42% said "definitely
achieved."
According
to information acquired from SPOT forms
(Appendix 1) student who evaluate their courses at the end of the semester also
had favorable views of their gains in critical thinking skills. All twelve
courses analyzed had a score of 85% or better where those who responded ranked
their gains in this area as average or more than average. Performance was about
the same from 1998 to 2001.
According
to the Alumni survey, 61.5% thought
their ability to use cultural knowledge was EXCELLENT, 38.5% said it was
SATISFACTORY, 0% said NOT SATISFACTORY, and 0% said it was NOT APPLICABLE to
them.
In phone
interviews with employers, on
the question of cultural knowledge, the responses were that employees were
consistently 'above average' or 'far above average' in their abilities. One
principal was particularly impressed with the teacher's energy and ability to
get her students to do research on modern Hispanic persons for research papers
and reports.
In the Senior
Exit Interview Report (Appendix 3), the Department Chair offered the
following comments: Students said that they are able to use cultural knowledge
especially in the ways they relate to people from the target country and other
international folks. Their self-awareness is also heightened, and they feel
confident that they have the basis to act internationally with high levels of
understanding. French students asked
for more Francophone studies; German students felt especially confident about
their ability to function in German business; Spanish students asked for more
variety in the survey courses, especially more about the Southern Cone
countries and Central America, with a little less focus in these courses on
Mexico and Cuba. Students found the influence of graduate students very
beneficial in the courses where Area Studies Master’s students were involved as
well as in the Intensive Second Language Institute in the summer.
Of the 26 graduates who returned the Alumni survey, 24 said that study
abroad should be required. Of the 2 who said it should not, one had studied
abroad and one did not indicate whether she/he had studied abroad. Students
talked about the importance of this for proficiency and cultural knowledge.
Service through the core
For students who had majors in the Department:
According to the Alumni survey, 52%
thought their experience of the core in our Department was EXCELLENT, 29% said
it was SATISFACTORY, 0% said NOT SATISFACTORY, and 19% said it was NOT
APPLICABLE to them. Apparently, this question was taken by most students
completing the survey as asking them about THEIR experience of core courses in
the Department.
For students who had majors outside the Department,
information was more difficult to gather: According to information acquired
from SPOT forms (Appendix 1)
students who evaluate their courses at the end of the semester had favorable
views of their gains in core skills. All twelve courses analyzed had a score of
85% or better where those who responded ranked their gains in this area as
average or more than average. Performance was about the same from 1998 to 2001.
Service to non-major students outside core
courses
According to the Oral Proficiency Interviews (OPI), the 10 students examined had an
average score of 7.5, or between 'advanced low' and 'advanced mid' when they
graduated. The two students who were not majors had an average score of 5.0 or
'intermediate mid' level. The two non-majors did not advance nearly as far as
those who were majors. One obvious observation is that non-majors have fewer
courses than majors. From the statistics gathered, we are able to draw only
limited information, since so few non-majors agreed to be tested. But those who
were tested show a level of proficiency lower than the majors, and given the
more limited course work, this is logical.
Quality of teaching
According to the Alumni survey, 83% thought the Department’s quality of teaching was
EXCELLENT, 17% said it was SATISFACTORY, 0% said NOT SATISFACTORY, and 0% said
it was NOT APPLICABLE to them.
Six courses – 2 each of French, German, and
Spanish; with statistics from 1998 and 2001, for a total of 12 specimens – were
analyzed for documentation from the SPOT
forms. For question 7, “overall rating of the professor” all 12 sections
rated professors highly. In 10 of the 12 sections all respondents said that the
teaching was either good or excellent. For the other 2 sections, those rating
the course good or excellent were 92% and 81%.
In the Senior Exit Interview Report (Appendix
3), the Department Chair offered the following summary: Students praised many
faculty members and criticized the performance of others. Sometimes praise and criticism were
contained in their comments about a single professor. This section of the report will discuss what faculty behaviors
students found helpful and what they found not to be helpful. Students appreciated faculty members who
were knowledgeable, organized, and consistent.
They praised faculty members who taught courses according to the course
or catalog description sheet, whose syllabus clearly outlined expectations, and
who stayed on schedule. They
appreciated multiple drafts for essays, opportunities to redress errors, and
assignments that were evenly spread across the semester. Students found that
faculty generally stepped out to help students in advising as well as course
work, though some did not know that they had an advisor. The caring and
compassionate attitude of the faculty members was much appreciated. Students
found that it was not helpful if
faculty members asked them to perform busy work, selected materials that were
below the students’ level, challenged them beyond their level without support,
were overly critical without explanation of what the student had done wrong,
and were unavailable for consultation.
Students pointed out a disconnect between the emphasis on communication
of an idea as stated in a syllabus and papers that were handed back to them
with grammatical errors as the only focus. They especially found it annoying to
get a paper back with an 86 grade and no comments on how to improve. In terms
of structuring of the curriculum, students felt that 30 authors in a survey
course was too many and that selection of highlighted authors may be better.
They also suggested that revisiting works in the light of future works would be
helpful as well.
Part
III. Conclusions
The Department is fulfilling its obligations to its
students. The student perceptions of success are high, the external indicators
are strong, and employers are pleased with our 'product.' Specifically,
technology literacy seems to be strong, critical thinking and problem solving
skills are strong, though students are less convinced because they focus on
theory and literary analysis as a major component of this rubric, while alumni
and employers tend to have a more pragmatic emphasis on getting jobs done and
problems solved. Oral proficiency is well within the expected range, and we
have continued our emphasis on ACTFL guidelines, study abroad, and practical
training.
Faculty, students, and alumni all express some
interest in further development of more pragmatic and professional training at
the upper level. These are issues that are already in process or under
consideration for future development (cf. vision statement, Title VI grant for
professional training in French, plans for new MA programs in French and
Spanish) and so we can be optimistic about these desires. There are also
observations about a need for better preparation for those going on to graduate
school. This has been a topic of discussion among faculty and will also be
helped by the presence of our own graduate degrees, which will make the
Department more aware of the needs of undergraduates as they prepare for
further work.
The
Department has done a good job of considering its plans from the 1997 Outcomes
Report and acting on them. There is more emphasis on functional and
professional tracts as well as on contemporary culture. The Department has
tried to get students involved in the research component of the unit (e.g.
undergraduate research, honors theses, federally funded study abroad programs,
honors scholarship finalists, and use of technology to increase student contact
with faculty research). The students also are actively involved in the general
life of the Department and in their own FLL student community with the informal
gatherings in the student lounge and with social gatherings between students
and faculty (e.g. group attendance at international films or lectures on other
campuses, Classics Table, faculty/student dinners).
Appendix 1:
Statistical samples taken from SPOT
forms for French, German, and Spanish
Prepared by the Department Executive
Committee, February 2002
Course # FR3205
|
1998 |
2001 |
|
Less than
average |
7 |
5 |
|
Average |
71 |
55 |
|
More than
average |
7 |
10 |
Course
# FR3305
|
1998 |
2001 |
|
Less than
average |
0 |
10 |
|
Average |
58 |
57 |
|
More than
average |
17 |
14 |
Responses to SPOT question 7: Overall rating of this instructor
Course #FR3205
|
1998 |
2001 |
|
Poor |
0 |
0 |
|
Fair |
0 |
0 |
|
Good |
7 |
5 |
|
Excellent |
93 |
95 |
Course
#FR3305
|
1998 |
2001 |
|
Poor |
0 |
5 |
|
Fair |
8 |
14 |
|
Good |
50 |
52 |
|
Excellent |
42 |
29 |
Course # GR3306
|
F 1998 |
F 2001 |
|
Less than
average |
11 |
0 |
|
Average |
44 |
56 |
|
More than
average |
22 |
44 |
Course
# GR3125
|
F 1998 |
F 2001 |
|
Less than
average |
0 |
0 |
|
Average |
67 |
60 |
|
More than
average |
17 |
20 |
Responses to SPOT question
7: Overall rating of this instructor
Course # GR3306
|
F 1998 |
F 2001 |
|
Poor |
0 |
0 |
|
Fair |
0 |
0 |
|
Good |
17 |
10 |
|
Excellent |
67 |
90 |
Course
# GR3125
|
F 1998 |
F 2001 |
|
Poor |
0 |
0 |
|
Fair |
0 |
0 |
|
Good |
17 |
10 |
|
Excellent |
83 |
90 |
Course # SPAN 3324
|
1998 |
2001 |
|
Less than
average |
|
|
|
Average |
41 |
92 |
|
More than
average |
22 |
0 |
Course
# SPAN 3204
|
1998 |
2001 |
|
Less than
average |
5 |
13 |
|
Average |
73 |
44 |
|
More than
average |
9 |
31 |
Responses to SPOT question
7: Overall rating of this instructor
Course # SPAN 3324
|
1998 |
2001 |
|
Poor |
|
|
|
Fair |
|
|
|
Good |
4 |
23 |
|
Excellent |
96 |
77 |
Course
# SPAN 3204/4984 (pilot # for new 3204)
|
1998 |
2001 |
|
Poor |
|
|
|
Fair |
|
|
|
Good |
41 |
6 |
|
Excellent |
59 |
94 |
|
Course#FR3305 F2001
|
Course#HUM2714 F2000 |
|
a.
Knowledge of principles, theories, techniques, etc. Less than average 10 Average 71 More than average 19 b.
Logical thinking and problem solving ability. Less than average 10 Average 57 More than average 14 c.
Appreciation of the subject matter and discipline field. Less than average 10 Average 52 More than average 38 |
a.
Knowledge of principles, theories, techniques, etc. Less than average 0 Average 57 More than average 36 b.
Logical thinking and problem solving ability. Less than average 14 Average 71 More than average 7 c.
Appreciation of the subject matter and discipline field. Less than average 0 Average 43 More than average 57 |
Language: GERMAN
|
Course # GR3305 S 2000
|
Course # GR3306 F 2001
|
|
a.
Knowledge of principles, theories, techniques, etc. Less than average 0 Average 67 More than average 33 b.
Logical thinking and problem solving ability. Less than average 11 Average 56 More than average 11 c.
Appreciation of the subject matter and discipline field. Less than average 0 Average 78 More than average 22 |
a.
Knowledge of principles, theories, techniques, etc. Less than average 0 Average 44 More than average 56 b.
Logical thinking and problem solving ability. Less than average 0 Average 56 More than average 44 c. Appreciation
of the subject matter and discipline field. Less than average 0 Average 56 More than average 44 |
|
Course # SPAN 4984 (pilot # of new
3204) F 2001 |
Course # SPAN 2744 S 2001 |
|
a.
Knowledge of principles, theories, techniques, etc. Less than average 13 Average 31 More than average 44 b.
Logical thinking and problem solving ability. Less than average 13 Average 44 More than average 31 c.
Appreciation of the subject matter and discipline field. Less than average 6 Average 19 More than average 75 |
a.
Knowledge of principles, theories, techniques, etc. Less than average 0 Average 42 More than average 50 b.
Logical thinking and problem solving ability. Less than average 7 Average 42 More than average 21 c.
Appreciation of the subject matter and discipline field. Less than average 0 Average 21 More than average 79 |
Appendix 2:
Memo to John Muffo outlining plans for
Assessment
MEMORANDUM
Revised on July 5, 2001 (changes in blue)
To: John
Muffo, Director
Academic Assessment
From: Judith L. Shrum, Chair
Department
of Foreign Languages and Literatures
Re: Outcomes Assessment to be conducted in 2001-2002 for the
FLL Dept.
Date: March 27, 2001
We determined this plan
through several meetings of an adhoc committee consisting of eleven faculty
members representing full-time, part-time, tenured and tenure-track faculty
from all languages we teach. We
consulted our departmental plan for 1997-2001, our vision statement approved
in1997, and strategic planning documents from the College of Arts and Sciences
and the University. We believe that
these assessment strategies will provide information needed by your office as
well as information we need to make future decisions.
We are requesting assistance
from your office for some of the assessments, a total request of $2100.
|
OBJECTIVE |
STUDENT GROUP |
MEANS OF ASSESSMENT |
FUNDING NEEDED |
|
Prepare students for grad school |
Majors, Double majors,
minors. |
Alumni survey fall 2001 |
Dept. will fund paper
survey mailing |
|
Develop oral and written proficiency |
Graduating senior majors
randomly selected from among majors and double majors. |
Arrange for ACTFL Oral
Proficiency Interviews in February 2002. Compare results with proficiency
scores when these students completed 3126 oral proficiency course. Augment with question from exit interview
about their perceptions of the OP classes; triangulate with study abroad
data. |
$140 per student x 10
students = $1400 |
|
Prepare students for global/inter-disciplinary
career contexts |
Majors, Double majors,
minors. |
Alumni survey fall 2001,
perhaps edited for specific issues; employer
feedback through phone interviews with 5 employers. |
Dept. will fund paper
survey mailing and phone calls |
|
Citizens who are technologically literate |
Majors, Double majors,
minors. |
Alumni in fall 2001 and
exit survey February 2002 edited to
determine student perceptions of how we use technology to help them learn |
Dept. will fund paper
survey mailing |
|
Critical and analytical thinking |
Majors, Double majors,
minors. |
SPOT form, item 14, fall
2001 |
Dept. will fund copying
costs |
|
Able to use cultural knowledge |
Randomly selected group of
25 students from majors, double majors, and minors |
Sections from a
standardized test, perhaps the Praxis II Subject Area Assessments |
Dept. will fund purchase
of tests |
|
Non-major students in language classes other than Core |
Students in Chinese,
Hebrew, Italian, Japanese classes (many will not be graduating seniors) |
Paper survey done in
class; triangulate with SPOT responses, fall 2001 |
Dept. will fund copying
costs |
|
Non-major, non-minor students in upper division
classes |
5 randomly selected
non-major, non-minor students |
Oral Proficiency
Interviews, February 2002 |
$140 x 5 = $700 |
|
Core Area 2 courses |
All students in Core Area
2 courses |
Paper survey to be
administered to randomly selected students in Core classes fall 2001; paper
survey will be mailed to randomly selected students who completed Core
classes in fall 2000 and spring 2001; triangulate with responses on SPOT form, item 14. |
Dept. will fund paper
survey mailing |
|
Quality of teaching |
All students enrolled in
classes in spring 2001 and fall 2002 |
SPOT forms |
Dept. will fund copying
costs |
Of
the 24 students who graduated from the department of Foreign Languages and
Literatures in May 2001, 10 participated in oral exit interviews conducted on
May 7 and 8, 2001. One additional
student participated via email. The
students were a mix of French, Spanish, and German majors. Our usual format for the exit interview is
to encourage students to tell us in an open-ended discussion what worked and
what didn’t work for them as they pursued their majors/minors/double majors in
our department. This year we added an
additional dimension to the questions, structuring them according to the goals
outlined in our Outcomes Assessment Report, which also form the organizational
structure for this report:
III.
Prepare students for
graduate school
IV.
Develop oral and
written proficiency
V.
Prepare students for
global/interdisciplinary career contexts
VI.
Prepare citizens who
are technologically literate
VII.
Engage students in
critical and analytical thinking
VIII.
Enable students to use
cultural knowledge
IX.
Provide quality
teaching
Among these students, two were planning on graduate
school at VT (Comm. Studies and Education), one was going to law school, one to
medical school, two planned to enter the Peace Corps, one was seeking
employment with the CIA or FBI, one was working at VT admissions, and one was
to become a commissioned military officer. They spoke highly of the
organizational skills they learned in their writing intensive courses; writing
papers for their upper division classes and reading difficult prose enabled
them to think about sophisticated concepts.
They felt adequately prepared in writing but not in speaking and wished
for more emphasis on speaking at the 4000-level.
The
American Council on the Teaching of Foreign Languages (ACTFL) sets guidelines
for performance in listening, speaking, reading, and writing and provides oral
interviews with certified examiners for a fee of $130. The department’s
first-and second-year courses have proficiency objectives, but no examinations
are given at that level. The
department teaches two courses, at the junior level, designed to develop oral
proficiency, 3125 and 3126 in French, German, and Spanish. Only 3126 is
required, and students must demonstrate certain levels of proficiency upon
entry and exit from these courses. The
final exit level is expected to be “advanced” as determined by informal
interviews by our own faculty members who have had training in administering
ACTFL’s Oral Proficiency Interview. Professors Fernández and Witthoeft are
certified examiners and Dr. Fernández is now a trainer of examiners. Results of
the official oral interviews conducted by ACTFL certified examiners are
featured as part of the Outcomes Assessment Report.
In
the exit interviews for graduating seniors, students asked for more
upper-division classes in which speaking could be developed, either a
4000-level oral proficiency class, or reorganization of upper-division courses
to allow for less teacher-centered lecture formats and more interactive student
participatory activities. They suggested debates, opportunities for informal
conversation, web chats that are then brought into class and continued.
Students in French OPI classes liked the idea of using a textbook because it
adds structure. Spanish students said
they liked the informal interaction provided by some OPI instructors, but they
also thought a textbook would add organizational benefits and enable them to
give form to the vocabulary and grammar they are practicing.
Students felt well prepared in all areas except speaking,
and here they noted a stark difference between what they study and what their
careers will expect of them. As one
French/agriculture student put it, “we know all about the bald opera singer but
can’t say ‘milking parlor.’” Students asked for more courses in technical
writing related to their field, a class on current issues in the target country
in general as well as courses with targeted information related to
agricultural, business, and engineering fields. Students supported the study
abroad programs and suggested a requirement of some form of immersion, either
study abroad or work with local target language communities. Students were well
aware of the difference between learning language in instructed settings and
using it in real settings. They felt
unprepared for the real settings and suggested more such opportunities,
accompanied by contextualized courses to reinforce their understanding of
language structure, either linguistic or grammatical.
Students felt that their some of their classes in this
department were more high-tech than some of their engineering classes. They enjoyed faculty use of web chats,
international videoconferencing, grades on line, on-line courses, research
links on line, use of videodisc programs, writing forums, and electronic
portfolios. They appreciated the distance learning courses but found it
difficult to engage in peer editing using that format. They liked having
insights into the research conducted by faculty, citing the case of one
professor who put a catalogue of language change on line as a result of his
research. They said that the student
computer lab available in the department removed a barrier for them, making it
easier to complete technologically related assignments right here where the
faculty members are also available for help if needed.
Students felt that 4000-level literature classes were
helpful in showing them a step-by-step process to look deeply into a literary
work and analyze it. In addition, writing intensive classes were helpful in
showing them how to construct and edit the quality arguments in a comprehensive
essay. German students pointed out that they were still operating at the level
of comprehension, unable to move toward analysis, and that they relied on the
professor more than they wished, resulting in more lecture format than their
preferred discussion format. French students asked that the selection of
literary topics be modernized a bit, focusing on political systems, culture,
popular press, the target country today, current issues. Spanish students liked the recombination of
Civilization/Culture/Literature courses at the 3000-level. Spanish students
also asked for more discussion format in some classes that are largely teacher-centered.
Enable students to use cultural knowledge
Students said that they are able to use cultural
knowledge especially in the ways they relate to people from the target country
and other international folks. Their self-awareness is also heightened, and
they feel confident that they have the basis to act internationally with high
levels of understanding. French
students asked for more Francophone studies; German students felt especially
confident about their ability to function in German business; Spanish students
asked for more variety in the survey courses, especially more about the
Southern Cone countries and Central America, with a little less focus in these
courses on Mexico and Cuba. Students found the influence of graduate students
very beneficial in the courses where Area Studies Master’s students were
involved as well as in the Intensive Second Language Institute in the summer.
The diversity of thought at perspective brought by these students enable
undergraduates to widen their cultural horizons.
Students praised many faculty members and criticized
the performance of others. Sometimes
praise and criticism were contained in their comments about a single
professor. This section of the report
will discuss what faculty behaviors students found helpful and what they found
not to be helpful. Students appreciated
faculty members who were knowledgeable, organized, and consistent. They praised faculty members who taught
courses according to the course or catalog description sheet, whose syllabus
clearly outlined expectations, and who stayed on schedule. They appreciated multiple drafts for essays,
opportunities to redress errors, and assignments that were evenly spread across
the semester. Students found that faculty generally stepped out to help
students in advising as well as course work, though some did not know that they
had an advisor. The caring and compassionate attitude of the faculty members
was much appreciated. Students found that it was not helpful if faculty members asked them to perform busy work,
selected materials that were below the students’ level, challenged them beyond
their level without support, were overly critical without explanation of what
the student had done wrong, and were unavailable for consultation. Students pointed out a disconnect between
the emphasis on communication of an idea as stated in a syllabus and papers
that were handed back to them with grammatical errors as the only focus. They
especially found it annoying to get a paper back with an 86 grade and no
comments on how to improve. In terms of structuring of the curriculum, students
felt that 30 authors in a survey course was too many and that selection of
highlighted authors may be better. They also suggested that revisiting works in
the light of future works would be helpful as well.
Appendix 4:
The Department of Foreign Languages and Literatures’
assessment of its programs is based on the following statement of goals and
objectives for its graduating students:
The
Department of Foreign Languages and Literatures seeks to graduate majors and
minors with an advanced command of the spoken and written language and a
thorough knowledge of the culture of a particular country or area of the
world. In addition, the department
seeks to provide its students with the analytical, critical, and communicative
skills essential to an educated member of society. Likewise, the department has the objective of developing in its
students a global outlook, broadening their awareness of cultural and national
diversities and commonalities. The
department seeks as well to give its students a strong foundation for careers
in foreign language education, in domestic fields requiring foreign language
and culture skills, and in the international sphere, and also for graduate and
professional education.
To assess its success in achieving these goals and
objectives, the department uses the following measures:
ž QCA in the major
ž Oral proficiency interviews
ž course grade averages
ž in-major “quality-control” graduation requirements
ž Student Perception of Teaching (SPOT) scores and
written comments
ž senior exit interviews
ž alumni survey
Subsumed within the discussion in the report are the data
from unsolicited letters of praise and from other student and parent statements
QCA in the major
The department policy whereby all majors must earn a
grade of “C” or better in all required courses in the major insures that all of
the department’s graduates have achieved what the department deems to be at
least a minimally acceptable level of skill or knowledge in the linguistic,
literary, and cultural areas of study comprising the major in a foreign language. The required curriculum for foreign language
majors includes coursework that gives all majors ample exposure to the material
that must be learned to meet the objectives
of the foreign language major.
Oral Proficiency Interview
The department’s Oral Proficiency requirement inaugurated
in 1988, provides a means of accurately evaluating our students’ achievement of
competence in oral communication. To
fulfill the requirement, students must take and/or test out of the second
semester of the Oral Proficiency course sequence 3125-3126.
Admission to each semester is by oral examination, and to
pass this course a student must achieve a proficiency rating of
“Intermediate-High” in 3125, and of “Advanced” in 3126. The course itself and the rating scale used
in evaluating students are based on national standards established by the
American Council on the Teaching of Foreign Languages Oral Proficiency
Interview (ACTFL-OPI) Guidelines. It is
the department’s goal that students who seek to obtain elementary- and secondary-school
foreign language teaching certification through
Outcomes
97, p. 2
the Education Option
program achieve a rating of “Advanced” on the ACTFL Oral Proficiency Interview
scale. In fact, about 20% of the
students accomplish this goal, while the remainder achieve the level of
“Intermediate-Mid” or “Intermediate-High.”
These students also are required to take the Praxis or the National
Teachers Examination, which cover not only language skills, but also literature
and culture. All students passed the
relevant sections of the Praxis or the NTE on their first or second
effort.
Course
grade averages
These data are forthcoming.
In-major “quality-control” graduation requirements
Since its first Outcomes Assessment report in 1992, the
department has taken a number of steps to strengthen its programs based on
previous assessment results. In
particular, efforts to introduce greater variety into the upper division
curricula are reflected in the revision of the 4000-level course offerings in
French to include more cultural and historical topics, in the expansion of the
linguistics component in the Spanish major, and in the revision of the Spanish
4000-level requirement to allow for more non-literary coursework. The French Section also has pioneered the
incorporation of interactive computer technology into its culture and
civilization courses and has created special non-contact technology-based 1C
courses to supplement the junior-level language skills sequence. These changes afford students more exposure
and practice with the language, especially in native-speaking contexts. The Spanish Section has introduced a new
third-year bridge course that gives students a stronger grammar foundation, has
offered advanced courses in film and theatre, and also has begun to make use of
instructional technology in its courses at all levels. Department-sponsored study abroad
opportunities have been expanded through the initiation of a summer program in
France and the establishment of a semester or academic-year student exchange
program with a university in Ecuador.
Further initiatives that were spurred by the last outcomes assessment
exercise, such as the development of professional language tracks and the
development of additional long-term study abroad programs, currently are
underway. The department also is
working on a Writing Across the Major plan that it will submit for approval
during the next academic year. It is
expected that the broad writing strategy together with the addition of advanced
listening comprehension and speaking skills classes will give our programs even
greater coherence and reinforcement of the progressive development of students’
foreign language proficiency.
Student Perception of Teaching (SPOT) scores and
written comments
Further evidence of the department’s success in achieving
its goals and objectives is provided by the average departmental score of 3.67
for the past ten semesters on Student Perception of Teaching course evaluation
forms. Written student comments are
correspondingly laudatory. The
department’s effectiveness in teaching also is reflected in the five
Certificates of Teaching Excellence, two Alumni Awards, two Diggs Teaching
Scholar Awards, and one Wine Award won by Foreign Language faculty members
since 1992. Since1994, an average of
three FLL majors per year have been initiated into Phi Beta Kappa, an average
of seven students per year have graduated with honors, one student won a
Fulbright Grant to France, and four Outcomes
97, p. 3
students have won highly
competitive summer study abroad scholarships offered by the Spanish National
Honor Society, Sigma Delta Pi. Since
1992, an average of ten FLL graduates per year have been accepted into graduate
school.
Senior exit interviews
Student feedback for the present report from exit
interviews with graduating FL majors and double majors is overwhelmingly
positive with regard to the faculty and staff in the department. Students repeatedly praise the dedication,
concern, knowledgeability, enthusiasm, and teaching ability of their
instructors and point to the faculty as the departments’ most salient
strength. So too the small class sizes,
the close, personalized interaction between professors and students, and the
caring responsiveness of the staff are singled out for praise and
appreciation. At the same time, a
number of desiderata that consistently are mentioned include: 1) the need for less emphasis on literature
and more variety of advanced courses, especially courses that focus on
contemporary life and current events, oral skills, grammar, and practical
applications of foreign languages for professional purposes. Students in German
and in Spanish would like to see the civilization and culture courses in those
languages extended beyond a single semester.
Students in Spanish emphasized the desirability of the study abroad
program in Spain. In all three majors,
students generally agreed that greater flexibility in the advanced course
requirements would be advantageous, particularly so as to allow a degree of
concentration on individual student career directions. Interest also was expressed in finding ways
to ensure that FLL graduates have a stronger general knowledge of the world
areas represented by the language(s) studied.
Similarly, the students surveyed stressed a need for more attention to
career counseling. A number of majors
wished for more participation by faculty members in language-related
extracurricular activities, and it was
suggested that the department should seek to inform and involve its majors more
in the research activities of the faculty.
For the most part, the graduating seniors surveyed were
highly complimentary of the department and positive about their experience as
foreign language majors. Many of the
improvements they would like to see already are on the department’s agenda,
while others will be targeted as priority items beginning in the immediate
future.
Alumni survey
An alumni survey sent in spring 1997 to FLL graduates
from 1992-1996 (approximately 100 persons), yielded 46 responses: 16 in French,
9 in German, and 21 in Spanish.
Slightly under one-half of the respondents have enrolled or currently
are enrolled in a graduate program. Of the
26 who indicated that they are or were employed in a language-related vocation
(including teaching as a graduate assistant) since graduation, nearly all felt
that the major had prepared them either “very well” (12) or “well” (13) for
such employment. The one remaining
responded that their preparation had been “average.” Similarly, 43 respondents felt that the major had “broadened
[their] global outlook and/or increased [their] awareness of cultural and
national diversity,” particularly as a result of study abroad in many
cases. Nearly all of the respondents
rated the advising they received while at Tech (their relationship with their
advisor) as either “excellent” (33) or “good” (11). One graduate described the relationship as non-existent, and one
considered the relationship to have been unsatisfactory. The survey also indicated that while only
one-half of the respondents had undertaken study abroad, 87% (40) felt that the
department should require a study abroad experience of all majors. However, the support of such a requirement
by several respondents was with the caveat that scholarship aid would be
indispensable for some students and that there should be the possibility of
exceptions and/or alternatives to the rule in special cases.
With regard to a question (#10) asking about the courses
that best prepared the graduates for their post-graduate job(s), 30% (14) of
the respondents gave no response or indicated that this question was not
applicable. Outcomes
97, p. 4
Otherwise there was no
consensus among the German graduates, while in French and in Spanish the vast
majority indicated that the most useful courses were those that stressed
grammar and composition together with the Oral Proficiency courses, i.e.,
courses that focus on communicative language skills per se.
The alumni responses to questions relating to curricular
deficiencies and areas of our programs that could use improvement (#s11,13,18)
mirror the perceptions of the graduating majors: more functional courses such
as in foreign languages for professional purposes, more emphasis on practical
communicative skills, more grammar, more variety of advanced courses generally,
more concentration on present-day life and culture, and more attention to
career counseling. As was noted above,
the alumni respondents also emphatically endorsed the idea of a study abroad
requirement for foreign language majors.
To date, the department has not sought to measure job
preparedness through employer satisfaction phone surveys. Only about one-half of the approximately 50%
of foreign language graduates from the past five years who responded to a
recent survey are or have been employed in jobs relating to their foreign
language skills and specialized academic preparation. In addition, many of the graduates who responded to the alumni
survey either were not willing to have the department contact their employer or
indicated that this category was not applicable. Of the alumni who did indicate that their employer could be
contacted, most are in public school foreign language teaching where the
department has reliable feedback through its Teacher Certification Program
director, Dr. Judith Shrum, who is in regular contact with supervising teachers
and public school foreign language administrators. Verbal reports consistently indicate that Virginia Tech Foreign
Language Teaching Certification graduates generally are considered to be
extremely well prepared and that they do an outstanding job in the classroom.
Like the exiting seniors whom we interviewed, the alumni
indicated a very high overall level of satisfaction with the department’s major
programs. They too consider the faculty
generally to be outstanding and also cite as distinct strengths of the
department its small class sizes, the individual attention given to students by
their professors, the balanced curriculum that addresses thoroughly the
development of proficiency in all of the principal language skills (reading,
writing, listening comprehension, and speaking), in culture, and in
literature. In many cases alumni also
expressed appreciation for the opportunities for additional language practice
and cultural enhancement provided by the department’s extracurricular
activities.
Summary
All
in all, the picture of the FLL major
programs that emerges from this review is a highly positive one that indicates
that the department is fulfilling its educational mission very
effectively. At the same time, the
assessment process also has been of great usefulness in reinforcing the
department’s sense of some of its relative deficiencies and in bringing to its
attention several less evident additional needs. Already the department has developed an internal resource
reallocation plan for the future that promises to allow for the implementation
of such things as more functional
language courses or even tracks in some or all of the majors. Likewise, curricular revision and changes in
in-major requirements can allow for a greater variety of advanced courses
designed to address the need for more conversational practice and more emphasis
on contemporary life and issues. More
specialized attention and minor resources for space and materials development
can bring about an improvement in career counseling. Greater integration of our majors into the intellectual life of
the department and of the profession itself can be achieved by inviting
students to attend departmental research and other discussion series and by
encouraging students to do research and teaching “internships” under the
supervision of foreign language faculty members. Finally, the inclusion of students in the life of the department
also can be heightened through the creation of a more comfortable and inviting
student lounge space within the department that can serve Outcomes 97, p. 5
as well as a study abroad and career information
center. Basically, nearly all of the
ways to improve the department indicated in the senior exit interviews and in
the alumni survey are useful guidelines for directions the department already
has begun to pursue or will take in the near future. We look forward to reaping the fruits of our efforts and seeing
the benefits of this exercise reflected in the next Outcomes Assessment
report.
June 4, 1997
Appendix 4a (appended to 1997 Outcomes report)
Student Comments during Senior Interview
April 29 and 30, 1997
Compiled
by J. Shrum
Professors Eustis, Head, and Shrum, Chair-Elect, met with
14 students on April 29 and 30, 1997 to conduct senior exit interviews. The format of the interviews was open, with
students given the opportunity to speak their minds as their thoughts occurred
to them. There was no structured set of
interview questions. Students were
simply asked to comment on what they found good and useful about their study in
the Department and what they would like to see changed.
The tabulation of student comments follows in the
Appendix, but their comments can be summarized in these ways. Students were pleased with the academic and
personal quality of the faculty and staff with whom they worked. They liked having professors more than once,
and their enjoyed working with Ph.D. faculty.
Given their high regard for the faculty, students asked for extended
opportunities to work as undergraduate TAs or to share in research projects
with faculty. Clearly, faculty have
been successful role models for students who eagerly seek additional
opportunities to understand and share in the professional life of faculty
members.
Student comments about courses, curriculum, and the major
generally indicate a need for more applied courses, more utilitarian uses of
language in varied content areas, and less literature. Students enjoy literature, however, and
suggested ways to combine it with other aspects of language, e.g. grammar
through literature, or the study of cultural geography through literature. Students feel that their knowledge of
contemporary issues related to the language they study in minimal. Students
also wanted more demanding grammar and oral proficiency courses at the upper
levels of study.
This group of students wants to be more challenged, and
made a number of suggestions for courses, e.g. phonetics and morphology in
French, that fall along traditional lines as well as suggestions for more
unique courses, e.g. medical geography in the target language, industrial use
of language, etc. Students were mixed
in their view of the use of technology for language learning; some suggested
more and others thought it unproductive use of class time.
In terms of pedagogy, students would like to see an oral
component in every class. They would like more participatory and less didactic
courses, and they appreciated opportunities for creative writing and use of
analytical thinking. They suggested
reading shorter and fewer selections, but in greater depth with more
discussion.
The students urged the faculty to re-consider the ordering
of pre-requisites and to be more clear in communicating them. They suggested a careful look at courses to
avoid incongruous sequences, repetition of some material to the exclusion of
other contemporary material.
In
terms of their own physical space in the Department, students would like to see
the lounge more useful, including information on careers, professions and study
abroad. They also urged more
incorporation of the student clubs into the life of the Department, perhaps
through revitalization of the student lounge or inclusion in the research and
professional teaching and research lives of the faculty.
[1] Percentages on SPOT form reporting do not always add up to 100% because of various accounting procedures. All reports given here represent numbers as presented on the SPOT forms.