DEVELOPMENT AND TESTING OF ASSESSMENT INSTRUMENTS FOR MULTIDISCIPLINARY ENGINEERING CAPSTONE DESIGN COURSES

Robert Edward Gerlick

The research presented in this manuscript was focused on the development of assessments for engineering design outcomes. The primary goal was to support efforts by the Transferrable Integrated Design Engineering Education (TIDEE) consortium in developing assessment instruments for multidisciplinary engineering capstone courses. Research conducted in this study included (1) the development of an assessment structure and methodology for assessing engineering design processes and products, (2) exploring the reflective practices of students participating in a capstone design project, and (3) testing multiple TIDEE assessment instruments for inter-rater agreement. The assessment structure developed includes a comprehensive plan for assessing students' design process activities and products within the context of three design phases: the problem scoping, concept generation, and solution realization phases. For each phase, one formative assessment is implemented mid-phase and one summative assessment is implemented end-of-phase. The rationale for this structure is to provide students and instructors an optimally timed and sequenced plan for facilitating student reflections and for documenting achievement of student learning outcomes. Student reflective practices were studied and compared with existing conceptualizations of reflection. A modified analytic induction approach was used to review and reformulate existing reflection concepts based on student data. Six general categories were found to represent student reflective practice: realization of a problem or contrasting idea, clarification of the problem or idea, analysis of one's thinking leading to the problem or challenging belief, synthesis of existing knowledge and theories to make interpretation and meaning of the problem or belief, validation of one's existing or revised belief through corroboration, and gaining of a new perspective through critical thinking on foundational premises of the belief. Finally, seven TIDEE assessments were tested for inter-rater agreement, including Team Member Citizenship, Team Processes, Team Contract, Growth Planning, Growth Progress, Growth Achieved, and Professional Practices. Multiple faculty and teaching assistant raters scored subsets of student work after being given a brief training session on the assessments and scoring procedures. Results indicated acceptable inter-rater agreement for each of the assessments intended for formative use, and suggested one summative assessment, Growth Achieved, may need revisions to the assessment, performance criteria, or rater-training protocol.

DEVELOPMENT AND TESTING OF ASSESSMENT INSTRUMENTS FOR MULTIDISCIPLINARY ENGINEERING CAPSTONE DESIGN COURSES

Files and links (1)

Abstract

Metrics

Details