In Optimizing The University - Why We Need a New Educational Model For A New Century I laid out some of the issues facing post-secondary science education and how changing student demographics and modern faculty responsibilities have exacerbated the challenges of adapting science education to fill the educational demands of modern society. I also discussed how the new knowledge gained from advanced assessment techniques have shown us the extent of the shortcomings.  

In Optimizing Science Education: What We Will Need For The University Of 2020 I looked at what this new optimized university - the university of 2020 - will look like. Here I will discuss two major cultural myths we need to dispel before we can make progress, and then I will list the ways “educational productivity” (amount learned for given amount of student and faculty time/effort) will be improved.

Myth I. Optimum teaching and learning styles are specific to each individual teacher and student

It is often claimed that teaching effectiveness is dominated by the personality or some other innate qualities of the instructor; some “have it” and others do not (with the implicit assumption that those who don’t have it can’t be good teachers, no matter how hard they try). A frequent corollary is that what is or is not an effective teaching method depends largely on the personality of the instructor and how well it suits them.  

These claims are clearly contradicted by the data, again from Physics Education research although similar results are also now emerging from other disciplines. This shows that when careful assessment of student learning is carried out, even the most dynamic, interesting, and entertaining lecturers fail to achieve good learning outcomes with the traditional lecture format, whereas “ordinary” teachers can achieve much better learning outcomes if they implement research-proven effective practices.  

A related common myth is that each student has an individual style by which they learn best. Since this issue of individual teaching styles for faculty and individual learning styles for students is such an important and often misunderstood point, I will explore this in some detail. I would argue that research clearly shows that the same basic factors are important for learning for everyone:  

(1) All people learn by building on their past knowledge and thinking.

(2) Learning any serious subject requires extended focused effort to construct one’s own understanding -- understanding cannot simply be passed along effortlessly from hearing the explanation of a marvelous teacher. It is well established that true expertise in anything requires thousands of hours of such intense effort and, perhaps surprisingly, natural ability has at best a minor influence on the amount of time required. 

(3) Everyone learns a subject far better if they are motivated to learn. Otherwise they will not put in the requisite extended strenuous effort. Everyone is far more motivated to learn a subject if they know why it is of value to them to learn it.  

(4) All people suffer from the limitations of working/short term memory and the related limitations on how much new material can be processed and learned in the time period of a typical class.  

These similarities dwarf the differences between people. There are a number of specific educational practices that benefit nearly everyone on which they have been tested, such as having timely well-targeted feedback that directly addresses one’s reasoning and says what is right and wrong about it (“formative assessment”).  

While there are frequent claims about students having very different learning styles, this is not supported by research, and in fact substantial research suggesting the opposite.   Individual differences in motivation due to different backgrounds and experiences are clearly evident, but these are much like differences in preparation, and are learned and changeable rather than being innate.  Similarly, differences are evident when one gets down to the level of student thinking on specific topics because of differences in past experience and education.  

However, even these individual differences in thinking about specific topics tend to be fairly limited in extent. Detailed research into both students’ general beliefs about science and learning science and the learning of specific topics reveals that there are a number of particular characteristics that are shared by substantial fractions of the students, so one can find a few categories that encompass nearly all of the different student thinking.  

As an example of such categories we can use basic electricity, where nearly all students who do not understand it will have one of a few incorrect ways of thinking about electricity. The misconception that electric current gets “used up” in flowing through a light bulb is an example of a widely shared misconception. Knowing these categories of thinking, being able to recognize when a student’s thinking falls into one of them, and knowing the most effective feedback to help them correct their thinking is an example of the “pedagogical content knowledge” that a good teacher will have. If a teacher has this knowledge, they are much more likely to be effective than if they do not, quite independent of their personality or that of the student.

Myth 2. Educational technology is a crutch for poor teachers but unnecessary for good ones

It is sometimes mistakenly thought that technology, particularly IT, reduces the interaction and hence educational effectiveness of a good teacher.  I would argue the value of the instructor is enhanced rather than diminished by the use of technology. In my own educational R&D efforts, as well as those of others, using technology such as good educational simulations such as our PhET simulations (phet.colorado.edu) and giving students suitably challenging tasks (not too hard and not too easy) which they must provide an answer for which they are accountable results in nearly all students becoming far more engaged in the subject. All of these are enhanced by use of technology. Students who are engaged in the subject will explore it in more depth and examine how the ideas apply in much wider range of contexts. That will result in far more and more challenging questions from the students and greatly enhanced student-instructor interaction.  

Technology that can allow the instructor to quickly probe the level of understanding for many students in the class makes it possible to provide much more timely and effective feedback to more effectively guide student thinking. This is a vital element of effective learning. Such technology can also allow the teacher to know how to better keep the students engaged and learning by providing them with suitable types and levels of challenging tasks.  

Relatively simple software can quickly determine the students' thinking and misunderstandings and provide the instructor with that knowledge. The instructor could then adjust instruction to provide immediate and helpful feedback to the students. Similar benefits are provided by well chosen problems that the teacher poses to students in a classroom, where they answer using personal response systems (“clickers”). The computer records those responses and tells the instructor in real time how well students understand the material.  
   
The result of such technology that keeps the student more engaged and enhances student-instructor communication is far more numerous and deeper questions, so an instructor who is an expert in the subject is essential -- considerably more so than in the case of traditional science teaching. Rather than merely a filterer and transmitter of information, the instructor is now routinely called upon to help students examine and understand the ideas at a much higher cognitive level. Also, via the technology, this instructor will be far better informed about the students’ strengths and weaknesses and thus can have much more educationally effective interactions with them. 

This guidance/feedback step could in principle also be done with a sophisticated computer program rather than an instructor, and such “intelligent tutor” software (1,2) has been developed for teaching basic algebra. While it is likely practical with today’s technology and understanding of learning to develop such programs for many more subjects and educational levels, it is vastly more complex and difficult to develop a computer that provides effective feedback than it is a program that merely identifies likely student thinking. So for the foreseeable future, for most university classes a good instructor enhanced by technology will remain the optimum mode for learning. 

Gains in both effectiveness and productivity

The optimized university will have enormous improvements in effectiveness and productivity. These are obviously connected, because we are defining “educational productivity ” to be the amount of student and instructor time (= money) required to achieve a given amount of learning. The effectiveness comes from:

* The value of having research-tested teaching methods in widespread use by faculty who understand and know how to use them; 

* By the use of extensive technology-based formative assessment so that each student is being challenged at the level where they can successfully build their understanding and expertise at the optimum rate; 

* By the timely and targeted feedback provided by technology-aided instruction to guide the student’s thinking. Also, students will be receiving instruction that reflects an ever improving state-of-the-art knowledge as to what topics and skills are important in the subject in question and how to help students best learn these.  

This ongoing improvement will take place because of the existence of good outcome assessments and the widespread dissemination of results and dissemination and duplication of successful practices.

The potential gains in efficiency can be seen by comparing the operation of the optimized university with the typical current university, where there is a largely unplanned and ill-structured course of instruction based on tradition, textbooks, or habit.  

Here is a list of the major gains in efficiency (by reducing misspent time) which thus enhances productivity:

1. Disseminating and copying what works

Each time a faculty member now goes to teach a new course, they typically reinvent it from the beginning. Thus they must spend time creating new learning goals, lectures, exams, etc. After some iterations teaching the class, the good teacher will gain a better understanding of what does and does not work, and there will be improvements, but these will be never exceed the knowledge, experience, and capabilities of that particular faculty member. Sadly, as soon as a new faculty member takes over the course, the situation reverts to the beginning.  

The teaching of science is, in this regard, exactly opposite of the practice of science.

Quite aside from the questionable educational effectiveness of this approach, it uses up a great deal of faculty time by forcing each to redo the work of their predecessors, as well as often repeating their mistakes. 

 It is often said that teaching is an individual activity that each person must do in their own style and therefore such reinvention and abandonment of what has been done in the past, with its corresponding inefficiencies and deficiencies, is inherent to the teaching process.  

However, this argument could be made with just as much validity (or lack thereof) to the scientific research that these same faculty members are engaged in. In science research, it is obvious how it is possible for researchers to continue to build upon and extend the advances of their predecessors with their own quite individual efforts and styles.  

Through this process they achieve results far beyond the capabilities of any single person. There is no reason why the teaching of science cannot be as successful as the practice of science in this regard.  

2. Eliminating the problem of vast discrepancies in student backgrounds

The greatest source of inefficiency in the current system of higher education is the enormous variations in student backgrounds (knowledge, skills, beliefs about how to learn and why to learn) encountered in nearly every undergraduate science course.  

This variation in students, combined with the lack of good ways to measure and respond to those differences, causes great difficulties and wasted time for faculty and students alike. The typical college science class, when it is going quite well, has perhaps 30% of the class bored because they already mastered the material (often in a previous course), 30% of the class so lost they are not learning anything (often because of a small but crucial deficiency such as knowledge of a particular terminology or mathematical technique), and 40% are getting some educational value.  

This 40% that are learning something is probably the best case scenario; often, because of lack of knowledge about the students or pedagogical miscalculation by the teacher, that fraction is much less. This means that a large fraction of both student and faculty time is being wasted because there is not a good way to routinely assess student learning. If there were, all students could achieve a clearly established mastery and the faculty could align the level and material covered accordingly.  

The resulting smaller spreads in student preparation, and better knowledge of that preparation by the instructor, would improve efficiency by making it possible to design courses that are optimized for the learning of the great majority of students, rather than the current inefficient compromise that is not well suited for anyone.

The extreme case of variation in backgrounds are those students who are enrolled in a course for which their preparation is clearly inadequate. This is a frequent source of anguish and large amounts of wasted time for both students and faculty. The wasted time and hardship that students encounter in trying to master material, when their inadequate preparation makes it impossible, is obvious.  

What is not so obvious is the large hidden cost in instructor’s time spent dealing with students who are not adequately prepared. These students often take up a disproportionate amount of instructional time, both in the need to provide them with extra assistance, and in dealing with the repercussions of failing students-- complaints from students and parents, pleas and arguments for regrading, special exemptions, etc.  

In teaching a class of 100 students or more, a conscientious faculty member will spend a significant fraction of their teaching time in such activities addressing these serious problems for a small fraction of the students. This is time that comes out of what is available for the education of the far larger number of properly prepared students.  

This inefficiency could be easily avoided if one has good diagnostic exams, as discussed below, to assess student preparation and ensure students are only enrolled in classes for which they are adequately prepared. This would benefit both the students who are not prepared for the course, and the students who are properly prepared. The same sorts of exams would rigorously assess the learning achieved in completing a course and hence, how well prepared a student was for the subsequent class.  

What are these new assessment tools?

I am frequently asked what the difference is between the kind of assessments I am calling for in the discussions above and the usual examinations that are used in college and university classes. The flippant answer is, “at least six months of hard work”. When the typical science exam is examined carefully, in spite of the best but usually untrained efforts of instructors, most students can and do complete them successfully using strategies based on simple memorization of facts and problem solving recipes. They test very little of the deeper more meaningful learning.

Valid assessments of the desired deeper understanding require a detailed examination of student thinking in the context of the specific subject material and the specific understanding and problem solving skills that are the goals of the course. Only then is it possible to create an assessment instrument that provides the requisite accurate probing of student thinking.  

The experience of assessment experts show that even exams constructed on such a foundation must still be carefully tested for validity and reliability with students, before it is possible to be confident of their value.(3) Some examples of such assessment instruments that have been created for use in introductory physics include the Force Concepts Inventory (FCI), the Force and Motion Concept Exam (FMCE), the Basic Electricity and Magnetism Assessment (BEMA), and the Colorado Learning Attitudes about Science Survey (CLASS).(4,5)  

A similar design concept but a more extensive development effort is required when the assessment device is part of a software system that provides real time feedback to guide learning. Examples of such systems are the Diagnoser program of Minstrell for diagnosing student understanding and misunderstanding in areas of high school physics, and the Cognitive Tutoring systems of Koedinger and coworkers for teaching algebra. (2)  

3. Avoiding unnecessary repetition

Another striking example of inefficiency of the current system is the way in which the same science topics are covered repeatedly in the curriculum for a science major, but each time covered so rapidly and taught so ineffectively that students do not achieve mastery. This sort of repetition has been noted as one of the distinctions between the K-12 education system in the US and several Asian countries that score far higher on mastery of science and math in international comparisons. (6)  

As an example from higher education, an undergraduate physics major will cover nearly every specific topic two to three times over their course of study. Other sciences have similar repetition in their curricula. I am aware of no evidence indicating value to the current system of rapidly covering the same material multiple times. On the contrary, research shows that when students develop misconceptions from their initial instruction, these tend to be maintained throughout subsequent instruction. On the other hand, when they have true mastery and understanding of the topic, it is robust and sustained.  

Thus, it is likely that such repetition of coverage is not only unnecessary but is even detrimental. Careful measurement of student learning to ensure they master the topic when it is first encountered will make it possible to design curriculum that avoids repeating coverage of the same material. This will eliminate an enormous inefficiency in the current system. 

4. Eliminating expensive faculty time being spent on low-level tasks

Another easily remedied inefficiency in the current system is the large amount of faculty “teaching” time spent on rather low-level tasks that could be performed by far less expert and lower cost staff. This involves routine class maintenance, recording of grades, dealing with students who are dropping or adding classes, dealing with special student circumstances such as missing assignments or exams due to medical or family emergencies, etc. The fraction of the “teaching” time required for dealing with these issues scales with the number of students.  

When typical class sizes were small this burden was insignificant, and the common organizational structure of having faculty handling such tasks developed during an era of such small class sizes. As economies of scale have driven up class sizes, there have seldom been appropriate organizational changes in response. As a result, for large classes these low level tasks can take up a large amount of faculty time, because there is seldom if ever support for staff to carry out these tasks.  

Some, although not all, of this time required for general class management could be handled by technology. There has been substantial progress in course management software, but, surprisingly, commercial products are still far from optimum. There are many individual efforts to develop better software, but the level of effort required is such that these “homebuilt” systems can never be as well maintained and easy to use as a large-scale commercial product.  

There are market reasons why commercial development has lagged in this area, but this is clearly an area where suitable investment would provide substantial returns. The optimized university will have suitable software and staff to avoid using any expensive faculty time on such low level tasks.
 
5. Optimizing the cost and effectiveness of support and feedback to students

In the optimized university, all the faculty instructional time will be spent on high-level educational tasks befitting both their expertise and cost, such as: delineation of desired expert skills in the discipline, pedagogical design and testing including enhancement of previous work, high intellectual level student interaction, and guidance of TAs.  

There will be a fairly clear hierarchy of support for student learning that will provide the optimum benefit for a given amount of financial resources. This is not a hypothetical model; I have implemented it in several science courses.

In this model the student is first mentally engaged by being given some suitable intellectual challenge, most commonly a homework problem carefully designed by a faculty member. In the current system, a student will typically work on homework in isolation (encouraged in this relatively unproductive activity by general policies and curve-based competitive grading systems). If they receive any feedback to guide them in their learning, it will likely come in the form of submitting a solution, and one or two weeks later finding out if their answer was correct.  

Research shows that such feedback serves very little if any pedagogical function. If they have a small class and a dedicated teacher, they may get more useful feedback by talking to the instructor about the problem. However, this is seldom practical on a widespread scale for large classes. Also, it is often a poor use of resources, since frequently the feedback, although quite necessary for the student to make progress, is very low level (“The reason your answer did not make sense was not because you misunderstood the concept, but because when you put this number into your calculator, you accidentally put in 200 instead of 2000.”) 

In contrast, in the optimized university, the student will have many levels of support and feedback. At the lowest level they will have intelligent tutoring systems and/or collaborative fellow students (in person or online) providing them with feedback. So, rather than struggling in isolation with the problem and making little progress for hours, they will have the fellow student quickly point out to them their calculator error, or nearly as often, they will discover their own error in the process of explaining to their fellow student how they are trying to do the problem.  

Structures and grading policies of the course will encourage such student-student collaboration and their associated well-established pedagogical value. When the problem encountered by the student become so intellectually challenging that collaboration and feedback from fellow students is not sufficient to allow the student to make further progress, then there will be trained undergraduate and, when that is not sufficient, then trained graduate teaching assistants to provide the necessary guidance. There is a large range of student difficulties and situations where such guidance is as adequate, and even sometimes superior to that provided by a faculty member, because the teaching assistants can have a better perspective on the student’s thinking and thus provide more effective feedback.  

Finally, for the most challenging issues that demand expertise beyond that of the TAs, the faculty will provide the necessary feedback. Such a hierarchical support system can work by taking advantage of the capabilities of modern communication systems. The far more expensive faculty time is then utilized when, and only when, it is required. This model allows one to provide a highly supportive and effective educational environment for large numbers of students, at a reasonable cost. 

6. Training Teaching Assistants to become important contributors to undergraduate education

Graduate student teaching assistants (TAs) have taken on an increasingly large part of the teaching at research universities. While this clearly has economic benefits because TAs are far less expensive than regular faculty, it is a source of frequent and usually well justified complaints. As in so much of the higher education system, the use of teaching assistants developed in a haphazard way and hence could be dramatically improved by some strategic planning and optimization.  

Originally, TAs were used for routine grading of exams and homework. Then economic pressures moved them into lower level teaching jobs such as overseeing students in labs, where relatively little supervision or planning was required. TAs can now be found carrying out a large fraction of the teaching in many situations.  

Just as there is little or no attention to training faculty for teaching – because there has long been the implicit, though now thoroughly discredited, assumption that if one masters the content, one can teach it effectively – a similar assumption has been made about teaching assistants. This has resulted in many classes being staffed with poorly trained and poorly supervised TAs whose teaching results in many calls to replace TAs in the classroom with faculty.  

However, this does not make sense either economically or educationally. There are now clearly proven examples of how well designed and tested training programs can routinely produce extremely well-qualified TAs who provide excellent educational experiences for undergraduates by every measure, and for some (though not all) aspects, better than a faculty member.(7) Such TA training programs do require small investments (several days of time for each TA, plus faculty oversight), but the return on this investment has been clearly demonstrated to be extremely high in terms of educational value and student satisfaction.  

From the perspective of optimizing resources to provide the best possible undergraduate education at a reasonable cost, well-trained TAs with suitable faculty supervision clearly makes sense.  
 
7. Optimizing the effectiveness and reducing the costs of teaching laboratories

A unique aspect of science instruction where there is a great deal of educational inefficiency is the teaching laboratory. Undergraduate teaching laboratories are particularly expensive in terms of facilities and student and faculty time, and, as they typically function, are doing a poor job of achieving the desired educational goals. This is a subject that attracts particular passion.  

Most faculty members feel strongly that because experimental research is such an essential part of doing science, laboratory classes must be an equally essential part of science instruction. The “argument” is usually presented in the form of an impassioned cry “This is the only way for students to learn how science is actually done!”  

However, the educational research reveals no indication that the typical laboratory class actually achieves this pedagogical function, and considerable evidence that it does not.(8) This educational failure arises generally from rather poorly thought out and conflicting educational goals for the lab classes13 and a dramatic mismatch between faculty intentions and pervasive student perceptions and cognitive practices in lab courses.  

Considerable improvement in effectiveness and efficiency could be achieved by a judicious examination of the educational goals of science lab classes, the assessment of how well they are achieving those goals, and the best and most cost effective ways to reach those goals.  

Class size

In considering how to optimize higher education, it is impossible to avoid the question of optimum class sizes. While everyone involved would prefer individualized instruction with class sizes of one or two, this is clearly impractical. From a purely economic point of view, the larger the class sizes the better. So the real question is, what is the tradeoff between class size and learning that is the optimum use of resources?  

An extension of this is, do we even need classes anymore? Can’t we just teach everything online with the proper software?  

I would argue that while online classes could easily replace classes involving students sitting in a cavernous auditorium listening passively to a lecture, it is much harder to see how they can replace classes designed around the interactive engagement of students in ways that have been found to be much more educationally effective (the norm in my optimized university).  

In these sorts of classes, there are social interactions (student-student and student-instructor discussions) that clearly play a large educational role.  

These same teaching style issues are relevant to the question of optimum class size. A class that relies on the traditional passive lecture format is equally ineffective with 20 or 200 students. Also, a large fraction of the learning in most good science courses happens outside of the classroom, and this outside-of-classroom learning is only indirectly affected by class size.  

However, without the use of technology it is clearly more difficult to achieve pedagogically effective social interactions and targeted individual feedback in a class of 200 students than in a class of 20.  

So the uses of research and technology as discussed above to make classes more intellectually engaging and educationally effective often have the most obvious gains for large lecture courses. There are demonstrations of classes of 200 or more achieving very good learning gains by utilizing technology and research based practices such as: clickers and peer instruction,(4) good computer graded homework systems, encouragement of pedagogically effective student-student collaboration, extensive course webpages, and email and online communications and survey systems. Learning gains in such classes have been measured to be as good as the best achieved in much smaller classes.  

Therefore, I do not think it is possible yet to say what class size would result in the optimization of learning within a fixed amount of resources, and the standard mantra of “smaller is better” is almost certainly not the optimum.  I have searched for data on this subject and have found very little.

The very limited data I know about (much of which comes from work of my group and collaborators) suggests that the optimum depends on room layout, and probably other factors, and is less than 400 but is perhaps more than 50. It is clearly in the interests of higher education to carry out studies on the tradeoffs between learning and class size.  

We will wrap up this series and discuss how we can reach these goals, plus maintain a balance between teaching and research, in the conclusion.

NOTES:
(1) J. Minstrell, http://www.diagnoser.com.
(2) Koedinger, K., Anderson, J. R., Hadley, W. H.,&Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30-43.
(3) J. Pellegrino, et al, Knowing what students know; the science and design of educational assessment, NAS Press, Wash. D.C., 2001
(4) E. Redish, Teaching Physics with the Physics Suite, Wiley (2003)
(5) W. K. Adams, K. K. Perkins, N. Podolefsky, M. Dubson, N. D. Finkelstein and C. E. Wieman, A new instrument for measuring student beliefs about physics and learning physics: the Colorado Learning Attitudes about Science Survey, Physical Review Special Topics: Phys. Educ. Res. 2, 010101, 2006, and K. K. Perkins, W. K. Adams, N. D. Finkelstein, S. J. Pollock, and C. E. Wieman, Correlating Student Beliefs With Student Learning Using The Colorado Learning Attitudes about Science Survey, PERC Proceedings 2004.
(6) http://nces.ed.gov/timss/
(7) K. Heller, University of Minn. Physics Dept., and G. Gladding, Univ. of Illinois Champaign-Urbana Physics Dept. Their TA training programs and materials are available on website and/or by request.
(8) Singer et al. (including C. Wieman), Americas Lab Report, NAS Press, Wash. D. C. 2006