Sub-Badge 6: Design learning assessment
Challenge 1: Identify the learning processes and outcomes to be measured
Criteria for successful completion of this challenge: Evidence of understanding the learning processes (instructional strategies) by identifying clear, measurable, specific, attainable learning outcomes. Reflection must address: How learning activities (processes) directly relate to the learning outcomes of an instructional plan (how they work together to support the learning process).
Examples: EDCI 588 Final Project, EDCI 572 Design Document #2 or Final Project, Individual Game Design Document (EDCI 556), Learning Design Activities (EDCI 575) artifacts (design, performance, workplace, educational, other) meeting the above criteria demonstrating understanding of the learning process.
Reflection
I present a design document I created while completing my coursework in EDCI 572, “Learning Systems Design,” in the summer of 2022. This design document followed the Dick and Carey model of systematic design. The design document’s initial sections focus on identifying the training goal, determining the learner’s current level of competency to achieve the goal, and the performance objectives that a learner must meet to achieve competency. My design document was used during the analysis and design phases of the ADDIE model. During my development phase, I frequently revisited the design document to ensure the learning activities directly correlated to the specified learning objectives.
I was the instructional designer and subject matter expert as I developed my training module that instructs learners on how to use a Fluke 437-II energy analyzer. As I contemplated what specific and measurable skills were required to perform this task, I began by completing the task myself and documenting each discrete step. Next, I asked a knowledgeable coworker to perform the same task as I observed. We discussed the specific steps a novice would need to know to perform this task safely and according to standard procedures. Next, I reviewed the manufacturer’s instruction manuals to ensure I included all the pertinent information in the proper sequence. This process led to five distinct performance objectives: installation, configuration, verification, initiation of a test, and interpretation of the test data. I continued developing these learning outcomes by listing subordinate objectives necessary to achieve the performance objective. With my learning outcome clearly stated, I began developing learning activities strongly correlated to the learning outcomes. The learning activities were primarily hands-on physical tasks. To guide the learner through his process, I developed a performance rubric (Appendix C) that clearly states the specific, measurable steps required to master this task. The learning module I created supports the learning process by systematically exposing learners to each learning outcome. The learner receives feedback via knowledge checks to assist them in identifying and remediating specific learning outcomes that are not yet mastered.
Before completing EDCI 572, I had yet to have formal experience documenting learning outcomes and mapping them to the learning process. We have a saying in my industry that technical experts with no formal background in education often “shoot from the hip” and hope for the best, meaning they may be subject matter experts, but they are not particularly effective educators. Reflecting on my previous instructional successes and failures, I am beginning to see more clearly that a seasoned instructional designer should be intentional in closely tying the learning process to the desired learning outcomes.
I plan to continue my evolution toward intentional, thoughtful design practices. I plan to continually improve my ability to identify specific, measurable, and attainable learning outcomes early in the design process. One particular area for growth I plan to implement is to take the time to develop a comprehensive design document and then storyboard the design before beginning the development phase.
Challenge 2: Ensure that assessment is aligned with instructional goals, anticipated learning outcomes, and instructional strategies
Criteria for successful completion of this challenge: Evidence of an assessment measuring and testing knowledge or skills aligned with the course goals, objectives, outcomes, and instructional strategies. Evidence must show an assessment plan (quiz, test, discussions, experiment, worksheet, graphic organizer, assessment framework, etc.). Reflection must address: How the assessment plan aligns with the course goals, objectives and instructional strategies. Your reflection must include why you selected and developed the assessment method of your choosing.
Examples: eLearning Project Proposal (EDCI 569 if taken prior to Spring 2021), EDCI 572 Design Document #3 or Final Project, EDCI 588 Final Project, LDAs (EDCI 575), artifacts (design, performance, workplace, educational, other) meeting the above criteria.
Reflection
I present a design document I created while completing my coursework in EDCI 572, “Learning Systems Design,” in the summer of 2022. The design document is a guiding framework to ensure that the course goals, instructional strategies, and assessment tools are tightly linked. I periodically reviewed the design document as I chose the types of assessment I planned to implement. Additionally, I created a table to show the correlation between my instructional goals and assessment items.
The design document provided a framework to follow as I moved through the development stage of the ADDIE instructional design model. At this stage in the process, I knew the purpose of the training and the instructional materials I planned to use. My next step was to identify a plan of action to measure whether this training was successful. To accomplish this task, I created a design evaluation chart. Each main step and the corresponding sub-step of my performance goal are listed alongside the test item I made. It is paramount when designing assessments to match the testing item with a learning objective to create a reliable and valid test. Using various question types (short answer, multiple choice, and hands-on performance), I made an effective test instrument that measures both the knowledge and physical, demonstrable motor skills. The short answer and multiple choice style questions are hosted on our LMS. A trained proctor administers the performance exam using Appendix C’s testing rubric. This rubric promotes fair and consistent evaluation by a proctor in the laboratory environment.
As a vocational trade school educator, I have written countless test questions and administered hundreds of performance exams. I am a certified proctor for national certification exams that cover industrial instrumentation and electric vehicle charging infrastructure equipment. These exams provide me, the proctor, with a detailed assessment plan and evaluation rubric clearly tied to performance objectives. I have adapted some of these techniques in my curriculum development. Now that I have learned more about the theory and practical implications of aligning assessments to instructional goals and desired learning outcomes, I am confident in producing high-quality assessment tools.
I plan to continue to improve in this area as I learn more about test instrument development. One area that I am excited to explore is the use of statistical measures to validate existing assessment items. My LMS provides a suite of statistical measures to evaluate learning outcomes and identify opportunities to improve test questions based on real-world data. This information, coupled with my experience aligning assessment items to instructional goals, will undoubtedly increase my ability to accurately and meaningfully assess learners.