Edward W. Thomas
Jack Marr
and Neff Walker
Schools of Physics
and Psychology
Georgia Institute of Technology, Atlanta, Ga 30332
Engineering students undertake an introductory course sequence in Physics before embarking on the courses of their major. Performance on this sequence is often poor leading to high attrition. We tested various interventions designed to improve student success and to reduce losses from the Engineering programs. Our trials were performed in the E&Mcomponent of the course which the students find to be the most difficult part of the Physics program. The difficulties arise because the subject is abstract, highly mathematical, and limited in tangible experience.
Performance is measured in terms of ability to solve quantitative problems in tests. We shall not debate whether this is desirable or effective but take it as the normal procedure. ``Training'' of the student involves having them review worked text examples and performing further examples as homework. Thus problem solving is the key to successful performance.
Experts rapidly analyze complex problems into components or subgoals, plan a solution strategy, and then evaluate the components based on experience. In this analysis a high degree of intuition is utilized (Larkin et al [1].). By contrast, the novice prefers to refer to memorized steps from previous examples used in the training stage; this strategy is not successful for solution of novel problems (Catrambone [2]). It has been shown that the performance of the novice may be improved by training which clearly labels the subgoals of a problem and emphasizes the use of intuition to plan the solution (Catrambone[3]).
The first intervention is designed to enhance basic skills of the student through the development of rapid and correct responding to relatively simple problems where solution strategy is not an issue. This intervention gives almost one letter grade improvement in performance by those students who are at most risk of failure. The second intervention is designed to enhance intuitive reasoning so that strategy for solution of complex problems becomes more apparent to the student.
The problem sets of two popular texts [4,5] have been analyzed according to solution technique. Typically a subject area, or text chapter, includes one or two basic definitions or laws which give rise to algebraic expressions which must be learned. We divide the problems from the chapter into two groups which we call ``direct'' and ``indirect.''
A direct problem is a word description where all but one of the factors in the basic definition equation are given and the student is required to find the unknown factor. The relevant equation must be identified, properly recalled, and the data correctly entered with attention to units. Proper solution of direct problems draw on memory and basic skills in manipulation of quantitative factors. An indirect problem is one where the basic equation is still employed but one (or more) of the relevant terms must be found from a subsidiary calculation or subgoal. This type of problem builds on the basics, but requires evaluation of a solution strategy and drawing on subsidiary information which has often been learned in a previous course or chapter. This subsidiary information was often a basic law in a previous chapter and training in its use was based on solution of appropriate direct problems.
As an example of the classification scheme, consider problems involving forces resulting from magnetic fields. The basic law is the force on a moving charge and a second closely related basic result is the force on a current carrying wire. A direct example of the first of these would be calculation of force on a moving charge with a specified velocity in a magnetic field. An indirect example would be a charged particle accelerated by a potential and then entering a defined magnetic field where the force has to be calculated. In this latter case the student must plan a strategy. Two of the factors in the basic equation (velocity and force) are unknown and one of these (velocity) must be determined from a subsidiary routine introduced earlier under electric fields and potential. An expert will plan the strategy, determine where the unknown information will come from, identify a basic law or definition which will give this information, evaluate velocity and then use the direct routine to arrive at the required answer. In doing this the expert uses intuition to develop an early analysis of the steps which must be followed. The novice will attempt to find ``an equation'' which represents the whole problem and then substitute numbers.
For each subject area we arrive at:
Together these represent a small set of information to be learned and a carefully selected problem set to provide training.
A first and most essential step is to train the student in the solution of direct problems representing substitution of information into basic equations. These represent part of the problem set provided with a subject, and they also represent the subgoals which are required in the solution of the more complex indirect problems. We require the student to perform a group of relatively simple direct problems at a high rate of speed with the criterion of success being a high rate of correct responding. This approach, where the time criterion is critical, is sometimes known as ``Precision Teaching.''
Five problem sets are written for each week of the course corresponding to roughly one Chapter of a standard text. Within a set half the work is simple single concept computation problems. In addition, there are short series of problems related to units, to underlying mathematics, and to intuitive deductions from a diagram illustrating a physical situation. The student is required to perform one such problem set on each of five days during the week with performance time limited to a maximum of 40 minutes. The ``score'' on the exercise is the number of correct answers divided by the time taken to execute the work.
A standard lecture class of 150 students was divided into six ``recitation'' sections. Two sections were given the Precision Teaching program on a computer platform format. Two sections were given the Precision Teaching on pencil and paper. The final two sections were used as controls and performed homework problems, drawn from the text, in a class supervised by a graduate TA. Allocation of the students to groups was random and the GPA (grade point average) of students at entry to the class were obtained so that performance could be related to past class performance in the Institute.
In the computer-delivered Precision Teaching format the problems were presented on the screen, the students entered responses by the keyboard. The computer graded answers, measured time, monitored student compliance with the rules, and provided an index of the rate of correct responding at the end of the session. Students were permitted to perform the work at any time of their choosing on one of a number of computer clusters on the campus. To permit the research team to review student progress and to answer questions the sections met once a week for an hour and performed the last of the five exercises in a class room situation; we do not regard the weekly meeting as a necessary component of the program.
The second Precision Teaching group received their materials on paper, graded their own results, were responsible for their own time keeping. Thus there was no objective measure of student compliance with rules related to time on task or frequency of performance. This group also met once a week to perform the last set of the exercises under supervision.
The control group was given a set of problems each week from the book to execute in their own time. They were further required to meet with a TA once a week for a classroom session where any problems could be answered and the TA could review the proper format for answering questions. Thus the control group was forced to spend the same time on task as are the two test groups. The control group received a conventional training exercise related to performance of complex word problems. It is interesting to note that it is only this control group which was trained to answer the sort of questions which will be asked on tests.
The measure of performance in the class was the success on periodic quizzes and a final examination. The problems and these tests were similar to those in the text and relatively complex in word formulation. It should be noted that testing was done with multiple choice answers, with the incorrect answers being attainable by incorrect manipulation of given information. Marks are obtained only for a correct choice of answer, a purely objective criterion. There was no attempt to assign a partial credit for wrong answers.
Figure 1 shows the performance of students on the bi-weekly periodic tests, plotted as a function of incoming GPA. There was no significant difference between the three groups. Students undertaking the precision teaching program to develop basic skills perform at the same level as students who work problems similar to those expected on the test.
Figure 2 shows the performance of the three student groups on the class final. For students with the highest GPA the control group carrying out a regular schedule of word problems performed best. But at all other levels the two precision teaching groups out-performed the regular homework section, with the computer-based group having a generally superior performance.
We would conclude that skills development through precision teaching is of significant value in enhancing long term performance with most value accruing to students at the lower end of the GPA scale. The computer-presented format was the most successful perhaps because it required the student to be honest about time keeping. The general form of the observations is consistent with the experiments reported by Thomas [6] in a test where the program was presented only to students at risk of failure.
The questions in the Precision Teaching program represent the direct questions of a typical text and also the subgoals used in the solution of more complex problems which we classify as indirect. Apparently training the student to efficiently and accurately perform such direct questions has a significant impact on their long term ability to solve complex word problems. The task now is to further enhance performance through training in an effective strategy for solution of more complex indirect problems.
An analysis of student performance on the various component of the Precision Teaching exercises shows that there is a strong correlation between ultimate success and the facility with which the student answers intuitive problems. Development of intuition may be an important part of overall training.
We have developed a computer simulation routine which allows the student to draw and simulate most problems from the text. The idea is to provide a visual representation of a situation of which the student has no direct experience. For example the student may draw a group of charges and the program will show for them all the components of force on a test charge, the fields, and the potential. The student is thereby provided with a set of clues as to what needs to be evaluated and how it will fit together. The student can rotate the model in space, redefine axes, and alter magnitudes. By ``playing'' with the problem the student develops experience and can choose representations which provide the most appealing format for solution. In a sense the program provides the intuitive ``feel'' that comes naturally to an expert.
Following the discussion of Section B above we have designed a homework problem set which covers all the direct applications of basic material and also the most frequent types of indirect problems. Students will be required to perform this problem set as homework for review by a GTA in a recitation session. The TAs are trained to systematically analyze the problems into goals and subgoals before ever attempting solution.
The effect of these two interventions is now being evaluated. One group of students, the test group, is being provided with the computer simulation and with the systematic approach to problem solving. A control group is receiving a standard homework assignment chosen by the lecture for its ``interest'' and with no attempt the provide a systematic approach. The lecturer in this test is not associated with the design of the program and his choice of material is that which he has employed for many years in teaching the course.
We expect to demonstrate that systematic design of problem sets and the availability of simulations to enhance intuition will result in a significant improvement of student performance.
This material is based upon work supported by the National Science Foundation under Grant No. DUE-9455470.
