Susan C. Schneider and Russell J. Niederjohn
Marquette University
Department of Electrical and Computer Engineering
Milwaukee, WI 53201
The primary purpose of the surveys distributed to our alumni and graduating seniors was to begin the process of ``assessment of student learning outcomes.'' The department has developed four primary Educational Objectives. These educational objectives are intended to probe not only our students attainment of knowledge specific to the field, but also to cause them to reflect upon how those aspects of education unique to Marquette University as a Catholic, Jesuit institution of higher learning have fostered their attitudes toward their jobs and their fellow humans. From the objectives, several Assessment Criteria were proposed which then led to the development of the survey instruments that were used. A secondary purpose of these surveys was to determine alumni attitudes to various features of the Electrical Engineering Curriculum and to see if these attitudes change as our alumni progress through their careers. A third purpose of the survey was to develop some demographic information about our alumni.
The specific details of the Graduate Exit Surveys and Alumni Surveys used by the Department of Electrical and Computer Engineering are discussed including the educational objectives, assessment criteria, and resulting survey instruments. In addition, partial results from these surveys are described, particularly referencing the success of the department in attaining its educational objectives and as a planning document for curriculum modification.
Marquette University began to address the issue of Assessment of Student Learning Outcomes in 1991 in preparation for the 1994 accreditation visit to the University by the North Central Accreditation group. As part of the preparation, each department and/or unit within the University was asked to form an assessment committee to assume responsibility for developing plans and implementing assessment programs. No particular approach to assessment was mandated, rather the departments were charged to develop a program that would be useful in evaluating broadly defined student outcomes.
In the Department of Electrical and Computer Engineering, assessment responsibility was assigned to the department's Goals Committee, consisting of five faculty from the department including the department chairperson and associate chairperson. A series of initial discussions related to assessment issues, most particularly what to assess and how to assess was begun in November 1991. These discussions were guided in part by information in The Departmental Guide to Implementation of Student Outcomes Assessment and Institutional Effectiveness by James O. Nichols The development of the department's assessment plan consisted of the following major steps:
Initial discussions during faculty meetings regarding assessment issues which included investigating the need and purposes of assessment, were held during the 1991-1992 academic year and through the fall semester of the 1992-93 academic year. Following these discussions, the Goals Committee worked on developing the preliminary materials for the department assessment which included a initial draft of the department objectives, as well as preliminary statements of the assessment criteria. In addition, a draft version of an alumni survey was prepared. These materials were presented to the faculty during the Spring 1993 semester for more discussion and approval. In Fall 1993 final versions of the educational objectives, assessment criteria, and alumni and graduate exit surveys were prepared. These surveys were distributed to the alumni and graduating seniors during the Spring 1994 semester.
Our department seeks not only to prepare graduates to enter the engineering field, but also to instill in them the attitudes and provide them with the skills they will need to remain current in the field for their entire lives (``life-long learning''). As a result, probing these areas was considered essential for our assessment. In addition, we wished to determine our students' perceptions of the unique ``intangible'' benefits accruing from a college education at Marquette University, a Catholic, Jesuit institution of higher education.
The process of developing the objectives started by first examining the existing (in 1992) Mission Statement for the University. This mission statement reads, in part:
``Marquette is committed to academic excellence grounded in a vigorous core curriculum and expressed in a rich diversity of liberal arts, professional and graduate programs of instruction and research. Marquette promotes the free and unrestricted pursuit of the truth and its application in the development of personal integrity, human values and social justice in all members of the academic community. Marquette's distinctive culture of learning depends upon these qualities of curricular diversity, freedom of inquiry, the integration of education with values and the creative balance between research and instruction.Marquette is devoted to the Jesuit tradition of cura personalis and seeks to aid all of its students in their intellectual, spiritual, emotional, social and physical development. Marquette encourages its students to seek positions and attitudes of leadership which reflect both the full exercise of their own talents and a commitment to serve others.''
From the University's Mission Statement, several department goals were structured. The primary department goal for our undergraduate educational program is:
The Department of Electrical and Computer Engineering of Marquette University seeks to offer high quality, up-to-date, nationally recognized undergraduate programs in electrical engineering consistent with the missions, goals and purposes of Marquette University.
From this goal statement, four specific objective statements were developed. These objective statements are as follows:
Two assessment instruments were chosen to evaluate attainment of the educational objectives listed above: an Electrical Engineering (EE) Alumni Survey and an EE Graduate Exit Survey. Consistent with the objectives listed above, assessment criteria were developed as follows.
For objective 1, the following assessment criteria are used.
To determine if BSEE graduates are readily employable (objective 2), the following assessment criteria are used:
To determine our students' commitment to life-long learning (objective 3) a partial listing of the assessment criteria includes,
And finally, for objective 4, some of the assessment criteria developed by our department are:
The primary purpose of both the Electrical Engineering Alumni Survey and the Electrical Engineering Graduate Exit Survey was to assess alumni/student responses to the four primary objectives that had been developed. The second purpose of the surveys was to determine attitudes related to various features of the Electrical Engineering Curriculum (past and present). A third purpose of the surveys was to develop some demographic information about our alumni/students as they progress through their careers.
The Electrical Engineering Alumni Survey consisted of 41 questions grouped in several categories including information related to employment, professional activities, continuing education, service activities, ``education impact,'' and curricular issues. The Electrical Engineering Graduate Exit Survey consisted of 60 questions in the same areas with additional questions related specifically to the current facilities and other academic issues. Both surveys were structured for use with computer coded response forms. Both surveys also included an optional narrative section for the alumni and students to expand on any of the issues raised in the body of the survey.
In Spring 1994, the Alumni Survey was distributed to 436 BSEE graduates from the classes of 1992, 1988, 1984, and 1980. The 146 response forms returned to the department were subsequently computer scanned and a frequency count and cross-tab report by graduation year was prepared using the statistical package, SAS. The EE Graduate Exit Survey was administered to 57 students enrolled in EECE 147 ``Senior Design;'' the response forms were processed similar to the alumni surveys. In Spring 1995, the EE Alumni Survey was distributed to 516 graduates from 1993, 1989, 1985, and 1981; 198 responses were returned. The EE Graduate Exit Survey was given to the 49 students in EECE 147. At the time of this writing, only the 1994 survey data are available.
Full reports of the results for the 1994 Alumni and Graduate Exit Surveys were prepared in Summer 1994 and were subsequently brought to the Goals Committee in Fall 1994 for review. Several representative results from these surveys are presented below.
When responding to the statement ``I feel confident of my abilities when asked to take on new assignments or responsibilities,'' 92.4%of all alumni responded that they strongly agreed or agreed with the statement while 83.3%of our graduating seniors strongly agreed or agreed. This assessment criterion for objective #1 had set a goal of 90%.
To assess the attainment of goal #2, that BSEE graduates will find ready employment in their field, the department had set a target of 95%for employment within 1 year of graduation. For the 1992 graduating class (first year of the alumni survey) 90.25%of the alumni who responded were employed full time or enrolled in graduate school. Only 2.4%indicated that they were currently actively seeking full-time employment while 7.3%responded ``other.''
To assess our students' commitment to life-long learning, the department had set a target of 75%of all full-time electrical engineering students becoming (student) members of a professional engineering society. From the results of our graduate exit survey, we found that 66.7%of our seniors (class of 1994) had joined an engineering society. We had also established a target that 75%of our alumni would remain or become members of at least one engineering society. For the class of 1992, 71%had been student members while only 61%were still members of an engineering society. For those members of the class of 1988, while 62%had been student members of a professional society, only 32.6%were still members 6 years after graduation.
Again, to determine a commitment to life-long learning, one of the assessment criteria dealt with continuing education. We found that within 10 years of graduation, 83%of our alumni were involved in some form of continuing education, and within 14 years of graduation that percentage had climbed to 90%.
For objective 4, appreciation for human values and their relationship to home, work, and everyday life, it was anticipated that 90%of BSEE graduates would agree or strongly agree with the statement that their Marquette education had prepared them to make intelligent decisions on issues which involve moral and ethical judgements. Of our alumni, 80%responded that they agreed or strongly agreed with the statement and 90.7%of our graduating seniors indicated agreement.
There were also questions posed to our students and alumni regarding various curricular issues. From our alumni, we found that overall 69.7%recommend that emphasis on the design process be increased, 69.2%recommend an increased emphasis on business related factors, and 69.2%recommend that electrical engineering laboratory work be increased or substantially increased. On the other hand, 67.1%of our alumni recommend that the current emphasis on humanities/social science topics remain the same (15%recommend an increase, 14.4%recommend a decrease), and 65.1%of our alumni recommend that the current emphasis on electrical engineering course work (non-laboratory) remain the same while 30.1%recommend or strongly recommend an increase in this course work.
Since the purpose of performing assessment is to provide feedback to the department to use to improve the quality of the education provided to the students, these survey results have been reviewed by the department's Goals Committee. Since the 1994 surveys were the department's first experience with both writing and administering surveys as well as reviewing the results of these surveys in light of the assessment criteria previously established, we expected that the first few administrations of the survey would help us ``fine tune'' both the surveys and the target figures and would lead to modifications of both items. For example, the department had established a target of 95%employment within one year of graduation. We did not meet that criterion. However, it is most likely that the initial target of 95%was overly optimistic, and the better way to phrase the assessment criterion would be to say that within 1 year of graduation less than 5%of BSEE graduates will still be seeking full-time employment. Employment categories such as ``other,'' which would include graduates who for one reason or another are voluntarilynot in the Science, Engineering and Mathematics (SEM) workforce would automatically reduce our result. In fact, 1986 national employment statistics indicate that only 95%of the available SEM population is in the labor force.
The role of professional society membership as being an indication of commitment to life-long learning might need to be examined. While we have not met our target of 75%student membership, specific actions have been taken within the department to increase the percentage of our students participating in professional engineering societies. It is, however, disheartening to learn that within 6 years of graduation less than 1/3 of our alumni are still members of an engineering society. This statistic is to be contrasted with the ``enrollment in continuing education'' target which was also established at 75%. With 83%of our alumni participating in continuing education within 10 years of graduation, this target is well exceeded. To probe the discrepancy between professional society membership and continuing education, several questions were added to the 1995 Alumni Survey to attempt to determine the difference in employer support and perception of the relative merits of these two forms of life-long learning.
The information obtained from the surveys related to curricular issues has led to the most immediate action on the part of the department. The perceived need by our graduating students and our alumni to increase the emphasis on laboratory work in EE has been directed to the department's undergraduate committee. That committee is currently investigating means to increase this emphasis by increasing the total number of hours devoted to formal laboratory work in our curriculum, including some type of freshman EE laboratory.
The use of this survey as a longitudinal study of the perceptions of our students/alumni has only begun. For example, a popularly held belief is that students appreciate the value of the humanities/social science component of the curriculum more as they age (in both years and experience). The data from the survey show 85%of our alumni from 1980 recommend that the emphasis on these courses remain the same with only 10%of these alumni recommending an increase and 5%recommending a decrease. For 1992 gra duates, however, 14%recommend an increase, 61%recommend that this content remain the same, and 24%recommend a decrease. It would appear that our students do appreciate this component of their education at Marquette more as they age, but it is shown notthrough a demand for more of this material, but a conversion from demanding a decrease to a decision to leave the amount of this material alone. This interpretation must be tempered with caution, since information on how the 1980 alumni felt about these courses just after graduation is not available.
Such longitudinal information will be available, since the alumni survey is structured on a four year cycle. Thus the classes surveyed in the 1994 and 1995 alumni surveys will be recontacted in the future. In addition, with graduating exit surveys from the classes of 1994 and 1995 available, it will shortly be possible to track how our students perceive their education as they progress through their careers. Another benefit from conducting these surveys is the enhanced contact with our alumni. It is clear from the optional comments provided by many of our alumni that they are truly interested in helping their alma mater improve and grow.
The process of student learning outcomes assessment is important. Recently circulated draft revisions of ABET criteria indicate that maintaining ABET accreditation may require demonstration of specific program outcomes. The Alumni Survey and Graduate Exit Survey we developed will allow us to track our graduates as they proceed through their careers to see ``how they fare'' in the changing marketplace. These surveys, however, form only part of the total process of studentlearning outcomes assessment. The Goals Committee is now directing its attention to matters of ``internal'' assessment, to develop the specific list of skills and abilities BSEE graduates should possess as well as how to assess the student attainment of those skills and abilities.