Thursday, April 3, 2008

ANGEL Assessment/Survey Design Flaws

This past month of March several issues regarding Submission History and Review of Assessments and Surveys within ANGEL have surfaced. Two separate tickets have been submitted to ANGEL by Dale and I; The first ticket dealt with scores being rounded to whole numbers within the Submission History table and was dubbed a design flaw, while the second ticket concerned Survey issues and is pending has followed suit.

Assessment Issue: Once an Assessment is completed and feedback is viewed you are taken to a table showing your Submission History and points awarded at which point you may review your submissions or take the Assessment again.  The problem is that the Assessment Submission History table displays whole number points awarded, while in comparison Quiz Submission History displays to 2 decimal places. The Assessment history therefore will round to the closest integer. Feedback from assessment will show missed questions and proper points and score, but when one continues to the Submission History Table it often displays full points awarded even when questions were missed.

ANGEL Response: Thank you for the very clear example.  I have confirmed this behavior.  This looks like a design miss.  The grade is recorded accurately to the gradebook but the submission history list in the student perspective should display the score to two decimal places (as it did in quizzes) rather than rounding to a whole number.

Survey Issue #1: Similar to the Submission History problem for Assessments, this involves the table with the rounded numbers. The survey is not tied to the Grade Book so no points show in the table, however, ANGEL labels the Point Column with a value of 100 Points by default as seen in Test Survey (Full Review). This is a problem because the survey is not worth any points and students could percieve otherwise. This is remedied when review is disabled as seen in Test Survey (Review Disabled)

(edit) ANGEL Response: By default when a content item such as a survey is not associated with a gradebook assignment, the submission review charts indicate that the item is worth 100 points.  I agree that this not a very good design and leads to confusion for both students and instructors.  I have logged an issue on this design for future review.

Survey Issue #2: This issue concerns Section Headings within Surveys which are used to label a set of questions regarding the same topic. This occurs when students or faculty review submissions either through the Submission History Table or Utilities>View, Grade, Delete Submissions.  As a student, once the quiz is completed, it will display feedback similar to Assessments with Section Headings and Question Responses displayed properly. However, when the student continues to the Submission History Table and views their past submission, the Section Headings are not displayed. This poses a subsequent problem for administrators who, when viewing submissions through the Utilities menu, are not able to see what Section Heading topic the questions and responses pertain to.

(edit) ANGEL Response: I have reproduced the second scenario where section headings do not appear in when reviewing a submission as a student or viewing a submission as a course editor.  I can see how the absence of these headings leads to a loss of important context if the heading provided instructions.

Judging from these responses it does seem that there were more than a few oversights in design for the current release of 7.2.  Although none of these issues have been confirmed for older releases the Assessment score rounding issue does not occur with Quizzes, the predecessors to Assessments, which is evidence that it was overlooked during upgrade.

Posted at 10:34AM Apr 03, 2008 by aolson in ANGEL