Quiz Basics 3: Attempts and Scores

Intro

In the first two articles of this basic course about Quizzing, I explained the Terminology, and the Submit Process.

Those posts applied to all types of questions: scored, random, Knowledge Check, pretest slides. This post will be talking exclusively about scored (graded) quiz slides and random quiz slides which are graded. For those slides the results are stored in the quizzing system variables, and you can have the score slide in your course. You will get some tips about the default score slide at the end. Scoring doesn’t exist for Knowledge check slides, you can only choose attempts on question level, not on quiz level.  Pretest questions have scoring as well, but the values are stored in different system variables, and are only meant to navigate the learner to another slide,  based on the pretest result.

This article will explore the scoring and attempts on two available levels:

  • on question level
  • on quiz level

Attempts/Scores on Question level

Setup for both is done in the Quiz Properties panel. That panel appears automatically in the newbie UI when you insert a quiz slide (or a random slide) in the right docking station. For quizzes however I strongly recommend to switch to the Expert UI (check the option ‘Enable custom Workspaces….’ under Preferences, General Settings). Set up a workspace where both the Quiz Properties and the Properties panel are visible. Reason is that you need the Properties panel for partial scored MCQ slides with multiple correct answers.

Attempts

In the default setup the number of attempts for graded questions is set to 1. The feedback messages for Correct and Incomplete are checked and 1 Failure message. The actions for Success and Last Attempt are both set to Continue. You could check the option Infinite Attempts, but I strongly doubt you’ll want to have the learner in such an infinite loop.  A limited number of attempts however can be a good choice. If you allow more than one attempt, do not forget to check the ‘Retry message’ because that is not done automatically. Have a look at this screenshot for True/False Question, to the arts marked with a blue rectangle. I set the number of attempts to , and checked Retry. The Failure message was left at its default 1 message.

I kept the default Continue actions as well, but moved the pausing point closer to the end of the question slide (see previous article) to minimize the waiting time after the second step of the Submit process.

Score/penalty for 1 correct answer

Default score for all questions is set to 10 points. Since some LMS’s don’t like to see total scores greater than 100 points, and all questions don’t merit the same score you will certainly want to change those scores. BTW: later on I will offer an exploring post about the wonderful Advanced Interaction panel (F9), unknown to many because it never appears automatically in the newbie UI. It is not only about ‘advanced’ actions at all.

Changing the score happens in the Quiz Properties panel for all question types with a black/white score. B/W score means that you only get the score if everything is correct, score will be zero in all other cases. All question types, except the MCQ with multiple answers are validated with this rule, even Matching, Hotspot and Sequence. In the screenshot above (T/F), you see the score reduced to 4 points.

I also added a penalty: this score will be subtracted from the total score if the answer was wrong after the last attempt. You don’t have to enter a negative number here. Beware: in SCORM 1.2 reporting a negative result at the end will be reset to zero.

MCQ with multiple correct answers and partial scoring

Have a look at this screenshot, set up for such a question slide, with 5 answers. Two out of them are correct, each has a positive score of 5 points. The 3 wrong answers get a penalty of 2 points each. Neither the score nor the penalty can be set up in the Quiz Properties panel, they are dimmed. You have to select each individual answer, and setup score in the Properties panel of that answer (visible in the screenshot as floating panel next to the Quiz Properties):

The 4th answer, wrong answer, is selected in this screenshot. There is no Penalty field in the Properties panel, tab ‘Options’, only ‘points’. Since this needs to be a penalty I entered -2points. Same for the other wrong answers. The correct answers got positive points (5 for each). The dimmed numbers for Score and Penalty on the Quiz Properties panel are calculated by Captivate to 10 points and 6 points. I had to check the ‘Partial correct’ message, was not done automatically although Multiple answers was chosen.

In this example I changed the actions (Success/Last Attempt) to ‘Go to Next Slide’, didn’t move the pausing point of the quiz slide. This is the second possibility to narrow the waiting time after Submitting the result.

You see that 3 attempts are possible for this question (green markings).  But the Retry message is unavailable, dimmed. Reason is that I have chosen to show 3 Failure messages, different one after each attempt. You need to include the warning about clicking the slide (or pressing Y) for the last Failure message which appears after the last attempt.

Quizzing System Variables

These quizzing system variables (see also: Using Quizzing System Variables) are linked to individual question slides:

  • cpQuizInfoAnswerChoice: after submitting the answer (see this post for a typical use case)
  • cpQuizInfoLastSlidePointScored: after submitting the answer
  • cpQuizInfoMaxAttemptsOnCurrentQuestion: while on that slide. Beware: there is no exposed system variable telling which attempt the learner is taking at this moment on question level, only the maximum allowed attempts can be retrieved.
  • cpQuizInfoNegativePointsOnCurrentQuestionSlide: maximum penalty for this question slide
  • cpQuizInfoPointsPerQuestionSlide: maximum score for this question slide.
  • cpQuizInfoQuestionPartialScoringOn: Boolean
  • cpQuizInfoQuestionSlideTiming: if you use a time limit on the quiz slide
  • cpQuizInfoQuestionSlideType

Attempts/Score on Quiz Level

The number of allowed attempts on (total) quiz level is set up in the Quiz Preferences, Pass or Fail.  Default setup is one attempt. If you allow multiple attempts, don’t forget to check the option to ‘Show Retake Button’. When clicking that button, all questions will be reset and the quiz system variables will be cleared.

If you also allow Review (Quiz Preferences, Settings), you have to be aware of the fact that all attempts on Quiz level will be considered to be exhausted if the learner clicks the Review button (also on the score slide). To prevent confusion use this easy trick: drag the Retake button on top of the Review button on the results master slide (or the score slide). Once the attempts are exhausted the Retake button will disappear and Review button becomes visible. Problem is that this is perfectly possible for a non-responsive (blank) project or a responsive project with Breakpoints. But a normal fluid box doesn’t allow stacking of buttons, unless you define the fluid box as being static. I will post a workaround in my “tweaking posts”, later in the Quiz sequence of posts.

The Total Score on quiz level will be calculated from all the scored objects in the course (see Advanced Interaction panel) and stored in a quizzing system variable cpQuizInfoTotalQuizPoints. Except for the feature ‘Branch aware’ that will be a fixed number when starting the course.  Other quizzing variables  (see post) linked to the quiz level are:

  • cpInQuizScope
  • cpInReviewMode
  • cpInfoPercentage:  appears on the results (score) slide as ‘Accuracy’ ‘percent’
  • cpInfoAttempts: appears on the results slide as ‘Attempts’ ‘total-attempts’
  • cpQuizInfoPassFail: Boolean
  • cpQuizInfoPointsscored: appears on the results slide as ‘You scored’  ‘score’
  • cpQuizInfoQuizPassPercent
  • cpQuizInfoQuizPassPoints
  • cpQuizInfoTotalCorrectAnswers:  appears on the results slide as ‘Correct Questions’ ‘correct-questions’; beware: partially correct questions are seen as correct
  • cpQuizInfoTotalProjectPoints: appears on the results slide as ‘Maximum Score’ ‘max-score’
  • cpQuizInfoTotalQuestionsPerProject: appears on the results slide as ‘Total questions’ ‘total-questions’
  • cpQuizInfoTotalUnansweredQuestions

The Continue button on the Score slide has about the same functionality and importance as the Submit button on quiz slides. That actions specified under Quiz Preferences,  Pass or Fail (after Last attempt on Quiz level) will be done after clicking that button. The pausing point on the score slide is linked with that button. You can move that pausing point the same way as for the quiz slides, closer to the end of the score slide. It is recommended not to have the score slide as last slide in a course, but have at least one more slide. That way you’ll be sure that the results will be transferred to the LMS.

Create Review State for scored Drag&Drop slide

Intro

Drag&Drop slides are popular both as Knowledge Check slide and as scored Quiz slide. No problem to integrate the score in the quizzing system variables. However there are still features where the D&D slide is not behaving like the other quiz slides. Some examples:
  • Pausing point is at 1.5 seconds, same as for Quiz slides, but that point is not visible in the Timeline.
  • Last Attempt action is wrongly indicated as ‘Failure’ action. Same as for normal Quiz slides, D&D slides have NO Failure actions.
  • You cannot have a Retry message when you provide more than one attempt.
  • There is only one Failure message, you don’t have the possibility of up to 3 different messages as is the case for other quiz slides
  • There is no Review status, which indicates if the question was answered correctly, and which answers were wrong or correct.
  • There are no Review navigation buttons. That is a problem, because during Review when the learner gets to a D&D slide, he will be stuck without those buttons.

For the problems concerning Try again, or multiple Failure messages I already offered a solution with a dynamic feedback message in a previous blog post. In this post I will try to explain a workflow for the Review issues.

The result of this workflow was visible in the sample project which I posted last week. There is a scored D&D slide as last question in  first part (Timeline).

Setup

Checkmarks

I used the checkmarks used for the other quiz slides, imported them into the Library. You can find them in the Gallery folder Quiz\QuizReviewAssets. Their name can be seen in this screenshot of the Library:

The small versions are used for the individual answer feedback, the normal versions for the global feedback Look at this example

Those checkmarks were used for multistate objects, with an empty Normal state. They are already present in the correct location during the quiz but invisible due to that Normal state. Two more states show either the correct or the incorrect checkmark. This is an example of the Object State panel, for the answer about the Shape button.

Review Navigation buttons

The quiz buttons in this theme are of the type 'Transparent button' That is the case for all themes packaged with Captivate. Too bad: it is impossible to apply any Quiz style to an object  a non-quiz slide, even on the Drag&Drop slide. The new feature Copy/Paste appearance will not work neither between items from the Standard and the Quizzing category. You have to create a style for a Transparent button that looks identical to the style of the Quiz buttons. As you probably know, the button label is simply text ‘<<‘ and ‘>>’. They trigger the commands ‘Go to Previous Slide’ and ‘Go to Next Slide’. Similar to the quiz slides, those buttons will only appear during Review (see On Enter action).

Variables

Four Boolean variables were created, with a start value of 0:

v_DD: will be toggled to 1 if the D&D slide is correctly done.

v_EffectDD; will be toggled to 1 if the Effect color is correct

v_SButtonDD: will be toggled to 1 if the Shape button color was correct

v_ShapeDD: will be toggled to 1 if the Shape color is correct

The system variable cpInReviewMode was used as well.

Events – Actions

Three events were used, all to trigger advanced actions: Success and Last Attempt event of the Drag&Drop slide (in that panel) and the On Enter event of the slide.

CorrectActDD

Why was it not possible to use the same Correct shared action in this case?  A D&D slide has no two-step Submit process, where the first step will keep the slide paused to allow the learner to read the feedback message.  It is possible to imitate that behavior, but I wanted to keep it simple, by keeping the slide paused for a couple of seconds before proceeding to the next slide. Second reason: for a correct answer need to toggle the variable  v_DD to 1. To achieve this, I used the shared action ‘CorrectAct’ as template to create this advanced action, which has two extra commands:

WrongActDD

Similar to the previous action, I used the shared action WrongAct as template, to create this advanced action. I needed only to add the Delay command, since the user variable v_DD had already the value 0.

EnterDD

Now the most complicated action, with many decisions to check all the possible situations. The decisions are mutually exclusive, which is the safest way to get a correctly functioning multidecision action. Hope you can figure it out with this Preview:

If you have problems or suggestions, post a comment please!

Quiz: Replace Score by 100% or 0%

Intro

Recent question on the Adobe forums is the reason for this blog. User didn't want the score (points or percentage) to be transferred to a LMS. Since the questions were random questions, using pools, it was impossible to use Knowledge Check slides, nor Pretest slides. The requirement to succeed the assessment was to have a minimum amount of correctly answered questions (7 out of 8). If that was the case, the transferred result to be transferred to the LMS should be 100%, and on failure 0%. I designed a workflow based on previous blogs about Random quiz slides and Reporting custom quiz slides. You will be able to check out a published example project, and follow the setup, Step-by-Step. To save some time I used a couple of ready-to-go slides from the project 'Alliance', but had to do edit of course.

Example

You can watch this file either from this link, (scalable)  or in its embedded version (fixed resolution):

Project has 11 slides, one of them (default results slide) is hidden. A pool was used to insert 5 random quiz slides. You need to have at least 4 correctly answered questions to obtain a score of 100%.


Setup

Here is the filmstrip of the example file:

Slides 1-3 are based on ready-to-go slides from the project Alliance.

Slides 4-8 are random quiz slides taken from a pool with 10 questions.

Slide 9 is again taken from Alliance (but with lot of multistate objects, see later).

Slide 10 is the hidden default score slide. It will automatically be moved after slide 9 because of the scored button on this slide (see step 3).

Slide 11,  final slide allowing to be sure that the final result (100% or 0% ) is correctly transferred to the LMS. I inserted the system variable cpInfoPercentage in a shape on this slide (top left).

Step 1: Score random question slides

Supposed you have the question pools ready, insert the required amount of random quiz slides in the project. All slides in a pool have by default the same score (10 points), it is not possible to have partial scores for MCQs with multiple correct answers.  There seems to be a bug for a FIB slide, where the correct answer appears as tooltip in case you have that question. Will log the bug, seems new.

Once the random quiz slides are inserted, it IS possible to edit the score. I took all score out, set them to 0. Just a tip, check the Advanced Interaction panel (F9, or under the Project menu) to see that the total quiz score is now indeed set to zero.  Penalty has no sense in this situation because success doesn't depend on the acquired score of the quiz.

To avoid long waiting time after the submit process for a question, I moved the pausing point near the end of the slides in all the pool questions.

Step 2: Tracking correct questions

To track the number of correctly answered question, you need to create a user variable v_counter, with a start value = 0. 

That variable needs to be incremented with each correct answer.  Number of attempts on question level is set to 1. It is then sufficient to create a simple advanced action to be triggered by the Success event of all random quiz slides:

Step 3: Results slide

I used a ready-to-go slide to show the result, but have converted several objects to multistate objects. The Normal state is the one shown on Success, the Failure (new) custom state will be shown on failure. This is the timeline of that slide:

Moreover I added two interactive objects to the slide, both invisible in output to start with (eye button in Properties):

  • Bt_Failure: is a transparent button similar to the Next button in the Alliance project. It has its default command 'Go to Next Slide'. 
  • SB_Success: is a start shape marked as button. Special is the fact that this button is set to Report (Actions tab ), and to include in Quiz. When you check in the Advanced Interaction panel (F9), this shows:

 Step 4: Conditional action EnterResult 

Use the On Enter action of the Results slide (slide 9) to show the correct state of the 4 multistate objects shown in the Timeline screenshot, and to show either the button SB_Success or Bt_Failure, based on the requirement of having at least 4 answers correct. Of course, the number 4 can be changed (OP wanted 7 correct answers). The action is self-explanatory:

This trick with the two buttons was published in my blog, over 8 years ago. The example file in that article is of course SWF output.  I didn't want to use two identical buttons - same location - in the example of today because that is impossible in Fluid Boxes projects.  This was not such a project, but it is perfectly possible to use the same workflow in a responsive project. It was also the main reason for the multistate objects used on this Results slide.

Extra: Editing Alliances

This is bit off topic, just for those interested

I did edit the ready-to-go slides a lot. Having a pause triggered by the On Exit event is not something  like because it can create problems. All slides have at least one interactive object, which can pause the slide near its end. That project has some quiz slides, but they are NOT based on the quizzing master slides. This means that the theme was not fully realized. When I inserted random quiz slides, which necessarily do use quizzing master slides they looked very different from the nice quiz slides. I have updated the master slides to make them looking approximately like the ready-to-go slides.

There are more problems with this project, which I detected but didn't encounter in this particular project. As I have written in previous posts, never do use the Switch to Destination theme!

SVGs for custom Hotspot question

Intro

About 5 years ago I published this blog explaining the workflow for such a question using shape buttons. The embedded example in that blog was of course a SWF output.  With release 11.5 the possibility to use SVG’s as buttons, and to control the clickable area, combined with multistate object made me feel it is time to update this old post.

Why a custom Hotspot Question?

The default hotspot question slide has several limitations. To me the most frustrating limitations are:
  • hotspots have to be rectangular
  • partial scoring is impossible
With a custom question you can have
  • hotspots with any shape
  • partial scoring, which still can be reported to a LMS
  • full control

Limitations of custom questions

  • developing custom question slides always takes more time
  • although the total score will be correct when using partial scoring, some quizzing system variables will consider each correct hotspot as separate question. You need to keep that in mind for the fields in the score slide (number of questions/correct questions not correct)
  • to the full functionality (Retake/Review) takes a lot of time.

In this post I will show a pretty simple example, no retake/review but with the possibility of having the score stored in the quizzing system variable cpQuizInfoPointsscored.

Example

Have a look at this two-slide file (rescalable HTML5). On the first slide you will be asked to click 4 West-European countries. On a correct click, the country flag appears bottom left. You will not be able to click that country anymore. Move to the second slide (Next button) to see the final score.

Please be a bit patient when you see a ‘pink’ feedback (which is the correct style). It may be due to the amount of SVG’s on the slide, but it takes time to hear the audio and see the flag appear and animated.
If the embedded movie is too small for your device, you click this link.

Setup

Look at the screenshot, which shows the Timeline and the Properties, Actions tab for one of the non-correct SVG-buttons  (Luxemburg). In the Style tab, the option ‘Enable Click in Bounding Box’ is disabled to limit the clickable area to the shape of the SVG itself (see recent post). In the screenshot both groups (countries/flags) are expanded. 

For the SVG buttons (countries) Success message is activated and used to display feedback. Those messages have two different styles: grey for the wrong clicks, and pink for the correct ones. After the correct message, an audio clip sounds and the flag of that country will appear at the bottom left. Both for correct and incorrect SVG’s the state will change to the Visited state when clicked: That visited state is grayscale for incorrect clicks. No advanced actions needed so far.


Actions

The on enter event of the question slide triggers the simple action ‘Hide Gr_Flags’

Success event for the correct countries, needs an Advanced action. In this case a user variable will store the score. It is not really necessary because the clicks are reported and have a score, the system variable cpQuizInfoPointsscored will have the correct scoe as well. Have a look at the action for Belgium:

The first three commands are self-explanatory. Why did first apply the effect (line 4) to the group, and wait 0,1 sec before showing the flag? Because that will avoid flickering, a trick I learned from another user many months ago.

Duplicate that action, and edit it for the three other correct SVG’s.

Why did I not use a Shared Action?

If you are a fan, you know that I mostly use Shared actions when possible. This was also the case for this example.Of course, I created first a shared action. But a strange phenomenon appeared: the Effect was not applied for each instance of the shared action, only for the first one. I wanted the whole group to be animated after a correct answer. Yes, I could have ungrouped and changed the action, but then that meant another advanced action On Enter for the slide (where I now hide the group). Moreover I like to have the effect emphasize the whole group instead of one flag.

Still looking out for the reason of this non-consistent behavior. Really this is the first time I encounter a difference between an advanced and shared action behavior. Be sure, will update this post when I have an answer.