Previous: Background -- education  Main page  Next: Background -- concept maps

Assessment We have seen that assessment is critical to ensuring that the university meets its obligation to its stakeholders. Furthermore, computer science assessment faces extra challenges in that it is a relatively new discipline. We will now examine assessment in greater detail. Taxonomy of assessment Assessment is classified as either formative or summative, depending on its purpose and the uses to which its results are put. Formative assessment is student-centred, in that its main purpose is to give feedback to students. In other words, it helps students to form a picture of the course content and where they stand in relation to it. Students can then plan their study more efficiently, and will be able to make better use of the resources provided to help them. The verbal feedback a demonstrator gives to a student can be considered formative assessment. Formative assessment is therefore one of the ways in which a university discharges its obligation to students. Summative assessment is designed to summarise a student's knowledge of the course material. It is one of the means by which universities fulfil their obligatons to industry and to professional bodies, ensuring that a student who does not meet minimal standards do not pass. When "assessment" is mentioned, the examples most people call to mind from their school years are largely summative: exams, tests, essays and quizzes. As a rule of thumb, any assessment which counts toward the final subject mark is summative, while any assessment for which the student receives significant feedback is formative. Obviously, very few assessment activities are purely summative: that would imply that the students are never even told their mark. Introductory CS assessment methods A wide range of assessment methods have been used in computer science courses. Exams are used almost everywhere for summative assessment, and pracs are also popular (Knox and Woltz, 1996; Chamillard and Joiner, 2001). Multiple choice testing is controversial. Although it is widely and justly criticised, it is also widely used. It is one of the very few assessment strategies that can be assessed accurately by a computer, so it is ideal for courseware delivered over the World Wide Web. Formative assessment must not be neglected. Hagan and Sheard (1998) have demonstrated that group discussion classes have a positive impact on student results. Related to group discussion is peer learning (Wills et. al., 1999), one of the so-called "alternative assessment methods", in which students work in groups and may comment on each other's work. Challenges of assessment We have seen that students need formative assessment if they are to maximise their chances of success in the course. Yet it has been shown many times that students tend to deprioritise assessment that is not summative. This tendency, called "selective negligence", was first described in Snyder (1973). This influential book documented the disparity between academic staff's stated ideal of engaged, imaginative learners and the cram-and-forget strategies that students found actually worked in their weekly quizzes. Snyder saw this "hidden curriculum" as a property intrinsic to the institution, but Sambell and McDowell (1998) point out that students come to university with a great deal of experience with assessment, and that this prior experience informs their attitudes to their tertiary study. Selective negligence may cause students to ignore activities that do not count toward the final mark, but it has been shown that making all assessment summative also has negative effects on students' study habits. Thomson and Falchikov (1998) found that, if students feel time pressure, they are more likely to adopt surface learning strategies such as rote memorisation. This is true whether the time pressure was imposed by poor planning on the part of the teaching staff or by poor time management on the part of the students. Good assessment must therefore be sufficiently attractive to induce the students to pay attention to it, while not being so cumbersome as to impose an undue burden on their time. This is particularly true of formative assessment. Good assessment must also be equitable: students should not be disadvantaged on the basis of gender, cultural background, disability, or any other grounds not directly related to merit in the subject. Much work has been done on addressing the perceived disadvantages women face in computer science (Cohoon, 1994; Clarke and Teague, 1994; Carter and Jenkins, 1999; Brown et.al., 1997), while Paxton (2000) discusses some cultural issues raised in South African universities. Evaluating assessment also poses a problem. It is easy to show that a given task produces consistent results, but it is difficult to prove that the task is actually testing the concept that it is supposed to test. Many assessment activities do not give much feedback to the course designer, especially if the course designer is not the marker. Despite all the hyperbole one hears about academia versus the "real world", universities also have real financial and practical constraints on their operations. Examiners, lecturers, invigilators, demonstrators and markers all have to get paid, and therefore cost must be considered when designing an assessment task. Universities also have a limited supply of equipment and teaching space. An assessment strategy that is otherwise perfect will not be workable if it is too expensive, or if it requires resources that the university does not have and cannot readily obtain. What is needed is a method that can determine what students know: not only its content, but its structure. This method needs to be able to be implemented without undue effort on the part of the students, and without entailing a lot of expense for the university. This thesis presents such a method.

Concept mapping for introductory programming Thesis main page Introduction and background     Background: education     Assessment     Background: concept maps   Aims     Competency mapping     Benefits   Method and results     Data sets     Method     Results     Random data   Analysis and conclusions     Factor analysis     Cluster analysis     Methodological problems     A better test     Conclusion   Appendix I: Datasets Appendix II: Activities Appendix III: MDS coordinates Appendix IV: Data generation scripts   Bibliography

Previous: Background -- education  Main page  Next: Background -- concept maps