Remote Assessment with Crowdmark Q & A

Questions and answers from the CTE EdTech Remote Assessment with Crowdmark session on Monday March 1, 2021.

1. With the LEARN integration, are there extra steps to ensure that group work is correctly imported into the gradebook?
   • LEARN and Crowdmark do not share their respective group memberships.
     The LEARN system has a 'group assignment' dropbox option:

       	Any member of a group can submit files to a group folder. 
       	Group members are given a common grade.
     

     The Crowdmark system has a 'group assessment' option with the same 'any-member-can-submit' and same-grade-for-group-members features. Students can form their own Crowdmark groups or you can manually make Crowdmark groups. See https://crowdmark.com/help/creating-a-group-assignment/
     A Crowdmark group assessment appears in the LEARN course gradebook as a independent item, not associated with any LEARN activity or feature.
     Please get in touch with UWaterloo Crowdmark help (cmadmin@uwaterloo.ca) know if you want to explore group work in more detail.

2. Can we use our Learn Sandbox to play with this?
   • Yes. Crowdmark can import your LEARN sandbox course. You can also create a non-LEARN Crowdmark course to play with.

3. What does crowdmark do with the 'This course has sections' checkbox?
   • Crowdmark sections are numbers (1, 2, 3, ...) assigned to each student that partition the students in your Crowdmark course into section1, section2, etc. When a new assessment is created, you can make it available for just a section of students, not every student. For example, you might want different lab sections to have the same lab assignment, but different due dates. This is achieved by duplicating the assessment and setting it up for each crowdmark student section.
     See https://crowdmark.com/help/managing-sections-or-multiple-versions-of-an-assigned-assessment/

4. So in the LEARN Quiz tool, I can enable a written response ('text') and the option to attach a PDF file/image. Does Crowdmark have that versatility? Or is it just one or the other?
   • A LEARN quiz question can allow a student to answer with text and file attachement answers in the same question. Crowdmark can too in their 'Text' question type. See section 'Answering text questions": https://crowdmark.com/help/completing-and-submitting-an-assessment/
     In Crowdmark, an attached student image file appears as an image, and non-image files appear as links in the student's answer. The grading tool only annotates answers with highlights and comments for the 'Text' question.
     A student's attached files cannot be annotated in the grade tool for this question type in Crowdmark or in a LEARN quiz. A LEARN dropbox student file submission can be annotated in the grade tool.
     In a LEARN dropbox or in a quiz text question, an instructor can include a file (attached or linked) to the feedback reply. Similarly, feedback files can be dragged into a Crowdmark grade tool comment box as a link.

5. Can you bulk create a larger number of questions (e.g. 50) or does each question need to be manually created?
   • Each question has to be manually created, even if that means just typing a single character into the 'Question content' input box and attaching a document of questions to the assessment. Crowdmark has to know how many questions and the point value of each one.
     Learn has a way to import a .csv file of questions into the Quiz question library. And, a Word document formatted in a specific way can be translated into a .csv question file using tools shown on this Brightspace help page: https://brightspacecommunity.force.com/s/article/Quiz-Question-Converter But, there is no such import feature in Crowdmark.
     Once you have made one 'generic' Crowdmark assessment - one with with many questions and question point values but no question text, you can clone it for re-use.

6. This may be a better question for the open Q&A at the end, but I've heard that it is challenging to 'randomize' questions in Crowdmark. Can you describe how this can be done and what the limitations are? (i.e., Is there something similar to the Question Pool tool in LEARN?)
   • Crowdmark has no concept of a 'random' selection or order of questions. The other UWaterloo quiz platforms LEARN and Mobius both do. This is why Crowdmark is a grading platform more than a quiz platform.
     Several different assessment variations can be made manually in Crowdmark and used with the 'section' feature so that everyone gets one of a small number of different assessments.
     LEARN and Mobius have been used to deliver questions to students (using all the random features they offer - order, selection, algorithmic variations) who view their questions on these platforms but submit their answers to Crowdmark. Instructors who do this know they will be manually grading from a larger set of questions and will have to look up the student's quiz in LEARN or Mobius to know which questions they are grading.

7. Is there a notification in LEARN that a Crowdmark quiz has been released?
   • There isn't.

8. Is it possible for us to manually upload all end-of-term assignments from students and just use Crowdmark to facilitate online marking? (So the students don't need to upload their assignments to Crowdmark, they just submit them in the existing system.)
   • If you don't have to grade 'a-question-at-a-time' but can grade a whole 'assessment-at-a-time' then the LEARN dropbox is better suited. It offers document annotation, rubric, and feedback comments and attachments.
     Crowdmark allows an instructor to manually submit an assessment for one student at a time. If you want to upload student work in bulk, Crowdmark offers the 'Administered" paper-based assessment type. You can read about it here: https://crowdmark.com/help/choosing-an-assessment-type/ You would have to make the assessment outside of Crowdmark, hand it to Crowdmark to stamp some unique student info onto each booklet (see the help page above about the QR codes added to the booklets), download the assessment pdf booklets from Crowdmark, find a way to send a different booklet to each student, have the students print and complete the questions by hand on the printed booklet and give the pages back to you for scanning, and upload to Crowdmark for grading. This assessment type is meant for assessments taking place in a classroom.

9. Can students submit more than once before the deadline? Or do they only get one chance?
   • Student can submit as much and as often as they like before the due date. Or, in the case of a timed assessment, before the timer expires. The due date/time trumps the timer.
     If students have submitted something before the due date then they cannot submit after the due date. If a student has let the due date pass with no submission, they can submit once and the submission is marked as late and the lateness penalty is applied to the grade.
     See: https://crowdmark.com/help/creating-a-timed-assessment/ and https://crowdmark.com/help/how-are-due-dates-and-penalties-enforced/

10. Are there only links between crowdmark and LEARN via the sync lists and output grades? (ie. only these functions?)
    • The Only Crowdmark-Learn linkages are: sync student roster, sync staff roster, import LEARN course from LEARN list, sync grades to LEARN gradebook, login from LEARN.

11. Can "graders" create tags as well?
    - Yes. Instructors, graders and facilitators can grade the same way. See: https://crowdmark.com/help/what-is-the-difference-between-roles/ -

12. Are "tags" not visible to students?
    • Tags are private to the grading team, invisible to students. Comments are feedback to students from the grading team.

13. Is there a way to create a timed 'assignment' (e.g., once started you have two hours)? Or only a calendar deadline (e.g., due March. 10 at 11:59 pm EST)? Also, as a follow-up, if there is a timed assignment option, how would AAS accommodations be applied to specific students?
    • Timed assessment help page: https://crowdmark.com/help/creating-a-timed-assessment/
      Accommodation help page: https://crowdmark.com/help/accommodation-options-for-assigned-assessments/

14. When the option to add checks to the page is showing does Crowdmark add up the checks?
    • Points are attached to comments rather than to the stamp characters checkmark, question mark and X. When there are both positive and negative comment points, the overall point total is not set by the simple sum of comment points. See this help page describing the summation of comment points: https://crowdmark.com/help/adding-comments/

15. Can students upload an essay or other type of assignment that they complete in word or powerpoint etc.? Is this just done by the the image/PDF upload? If so, in order to have separate answers it would then be necessary to add individual questions and students would need to upload multiple files, yes?
    • Students can compose their answers in any software they like, even paper. They then have to turn their document into either a PDF file or a sequence of images. MS programs can export to PDF format. Students can use a phone app to snap images of paper pages.
      When a multi-page PDF file is submitted to a Crowdmark question, all the pages in the PDF are shown in minature. Students can delete and reorder any of the pages.

16. Can students resubmit a specific question?
    • Students can re-submit to any questions they wish to answer as often as they like before the due date/time.

17. How do you know what question the student is answering?
    • Students upload their answers to specific questions on their submission page. Graders click to be taken to the next ungraded instance of the question they are grading. Graders can also look at a student's whole submission and click the question label to grade that question.

18. I've heard you can create like three different full assignments/tests and assign a subset of students to each assignment, but that its really clunky and I don't know how that would work when importing grades into LEARN.
    • See the answer to question 3 above. And, this help page about creating Crowdmark sections: https://crowdmark.com/help/managing-students-in-a-course/
      Each Crowdmark assessment is exported into its own LEARN gradebook item. You can create a LEARN gradebook category to hold all the gradebook items that represent the different sections of the same Crowdmark assessment e.g. Lab1-section1, Lab1-section2, etc. The LEARN gradebook category 'Distribution' setting can be set to drop all of the lowest grade items but 1 of the Lab1-sectionX grades. In effect, this keeps a student's only real grade and drops their 0 grades from the category.

Paul Kates
Mathematics Faculty CTE Liaison
pkates@uwaterloo.ca, x47047, MC 3044
Last modification date: Tue Mar 2 00:09:20 2021.