Introduction

Providing Library Instruction via Angel at times will require use of resources shared in the "Library Repository Pullman" (LOR). It is necessary to have procedures in place for adding content to the LOR and subsequent recommendations for creating assignments so that users of shared resources can have baseline expectations for resources they use. The procedures outlined below were designed to capture data to allow for tracking of use, as well as longitudinal data gathering and analysis that the Instruction Team finds valuable. There may be times when following this guide would not produce the desired structure or results, in such instances, it is recommended to utilize your personal repository of which all angel users have, to accomplish your task without infecting the LOR with resources that do not follow the protocols outlined below. These procedures are not a substitute for Angel training, and it is recommended that anyone working with Angel take advantage of training opportunities. available

Basic workflow

All content (tutorials, and assessments) should be added to the LOR before creating a "Lesson" in a course space. Once all the sources are added to the LOR, they can be added and assigned as necessary in a course.

Documentation Breakdown

The following procedures are broken into three main areas, Adding Content to the LOR, Publishing a Lesson into a Course, and Export Assignment Data. Each area has several subparts related to relevant tasks.

Adding Content to the LOR

Content consists of text, links, embedded images, tutorials and assessments

Tutorials

Adding an Existing Tutorial to Angel

 

  • This procedure is for adding an existing tutorial (from WSU Libraries or the open web) that will be assigned in its entirety (e.g.. CLIP: Generating Search Terms
  1. Begin at the Learning Object Repository  on the leftmost toolbar
  2. Click on a repository name (e.g. Library Repository Pullman or your own personal repository)
  3. Navigate to the Tutorials folder
  4. Click on "Add Content"
  5. Click on "Link" under the Create an Item section
  6. In the Content tab
    1. Add a Title: use the convention "Institution name: Tutorial Name", e.g. "CLIP: Generating Search Terms"
    2. Under Link Settings section
      1. Link target: Same Window
      2. Link URL: web address of the tutorial
  7. In the Access tab
    • The "View Restrictions" start and end dates control what appears to the student. It does not make an assessment or tutorial able to be interacted with, that function is controlled using the Interaction tab.
    • It is recommended to leave the start and end dates in the "Access" tab alone
    1. User Tracking: Everyone
    2. Viewable by: Members
  8. Save

Adding an Existing Multi-part Tutorial to Angel

 

  • This procedure describes how to add an existing multi-part tutorial for which only certain components are assigned (e.g. Griffin Tutorial)
  1. Begin at the Learning Object Repository  on the leftmost toolbar
  2. Click on a repository name (e.g. Library Repository Pullman or your own personal repository)
  3. Navigate to the Tutorials folder
  4. Click on "Add Content"
  5. Click on "Page" under the Create an Item section
  6. In the Content tab
    1. Add a Title: use the convention "Institution name: Tutorial Name", e.g. "UNC: Copyright and Fair Use"
    2. In the Page Text area, add instructions for portions of the tutorial to complete. You can use screenshots or other images to indicate this.
    3. Obs.: Images should be stored inside drupal using https://www.wsulibs.wsu.edu/file/add
    4. Also in the Page Text area, add the link to the tutorial
  7. In the Access tab
    • The "View Restrictions" start and end dates control what appears to the student. It does not make an assessment or tutorial able to be interacted with, that function is controlled using the Interaction tab.
    • It is recommended to leave the start and end dates in the "Access" tab alone
    1. User Tracking: Everyone
    2. Viewable by: Members
  8. Save

Creating an Internal Tutorial in Angel

 

  • This procedure instructs on how to create a tutorial or instructional page in Angel, as opposed to linking to an external tutorial housed on another server
  1. Begin at the Learning Object Repository  on the leftmost toolbar
  2. Click on a repository name (e.g. Library Repository Pullman or your own personal repository)
  3. Navigate to the Tutorials folder
  4. Click on "Add Content"
  5. Click on "Page" to create the desired content element
  6. In the Content tab, a HTML editor appears where the content can be created. This can be plain text, links, images, multimedia files, etc.
    1. Fill in page settings
      1. Title naming convention: "Institution name: Tutorial Name"
    2. Obs.: Images should be stored inside drupal using https://www.wsulibs.wsu.edu/file/add
  7. Make the new object to be viewable by others
    1. Click on "Access" tab
      • The "View Restrictions" start and end dates control what appears to the student. It does not make an assessment or tutorial able to be interacted with, that function is controlled using the Interaction tab.
      • It is recommended to leave the start and end dates in the "Access" tab alone
      1. User Tracking: Everyone
      2. Viewable by: Members
  8. Save

Assessments

Creating a Tutorial Quiz Set in the LOR

 

  • This procedure describes how to add a tutorial quiz set in the LOR. These are the equivalent of the lists of questions for each tutorial in ILE
  1. Begin at the Learning Object Repository  on the leftmost toolbar
  2. Click on a repository name (e.g. Library Repository Pullman or your own personal repository)
  3. Click on "Assessment" folder
  4. Click on "Tutorial Quiz Sets" folder
  5. Click on Add Content
  6. Select "Assessment"
  7. In the Content tab, enter a name for the question set (e.g. "WSU: Country Research Quiz Set")
    • At this point, only the Content tab needs to be set
  8. Save
  9. The "Assessment Editor" appears where you can introduce the questions
  10. Click on Add Question
  11. Select a question type (e.g. Multiple Choice or Multiple Select)
    • The default question type is "Multiple Choice", which allows only one correct answer
    • If you need to choose "Multiple Select", which allows for multiple correct answers
      1. Go back to "Settings"
      2. In the Subtitle field, add the text "Some questions have multiple correct answers"
  12. Enter the question details
  13. Questions are coded with the form XX.YY, where XX is an assessment id, and YY the question number. If you are creating a new question set, get the next available assessment id by following the directions in Quiz Question Unique Identification.
    1. Question text, answers; indicate the correct answer(s). HTML tags can be used to format the text.
    2. Question Title: question code, e.g. 59.01
    3. Points value: because quiz sets are shared, it is important to follow the convention that each multiple choice question is given a value of 2 (two) points

  14. Save
  15. Repeat steps 10-13 for each question

Updating Questions in a Tutorial Quiz Set

 

  • To maintain the ability to analyze quiz questions statistics, questions should not be edited or deleted, as either option results in the loss of data. Instead, follow the instructions below to archive the quiz set and then re-create the quiz set with updated content
  1. Moving the existing quiz set to the Archive
    1. Begin at the Learning Object Repository , on the leftmost toolbar
    2. Click on a repository name (e.g. Library Repository Pullman or your own personal repository)
    3. Click on "Assessment" folder
    4. Click on "Tutorial Quiz Sets" folder
    5. Hover the mouse pointer over the quiz set to modify
    6. Click on "Settings"
    7. Edit the title adding the text "Archived MM.DD.YY - " as a prefix. e.g.: "WSU: Standards Quiz Set" will become "Archived 05.27.11 - WSU: Standards Quiz Set"
    8. Click "Save"
    9. Click on "Utilities" link
    10. Click on "Move Item"
    11. Select "Tutorial Quiz Set Archive" as destination folder
  2. Create an updated quiz set
  3. Continuing at the "Tutorial Quiz Sets" folder:
    1. Click on "Add content"
    2. Click on "Copy Items"
    3. Look for the archived Quiz Set and click on it
    4. For the copied Quiz Set, click on "Settings"
    5. Adjust the title as needed
    6. Save
    7. Modify the quiz questions as necessary
    8. Obs.: To properly number a new question (updated question) do not repeat a question ID number. Instead, use the next number sequentially. e.g. if the initial question set have 25 questions, and question number 14 is in need of updating, this will be deleted and a new question added, with a new question ID number. In this example, the new ID would be 26.
      1. To delete a question, click on the "Delete" link
      2. To add a question, follow the instructions in the previous section

Course Assessments - Combining Tutorial Quiz Sets into an Assessment

Angel Documentation - Course Assessments - Combining Tutorial Quiz Sets into an Assessment

 

  • This procedure describes the process for combining quiz sets into an assessment. To create a pool of questions from which a subset will be drawn see Link this, Course Assessments - Creating a Question Pool
  1. Begin at the Learning Object Repository , on the leftmost toolbar
  2. Click on a repository name (e.g. Library Repository Pullman or your own personal repository)
  3. Navigate to Assessments / Course Assessments
  4. Obs.: Collaboration with courses will span multiple semesters. A clean shared space will be maintained by using the following structure in the Course Assessments directory: 
    Course Name / Course Name Semester Year / content. e.g.: World Civ 111 / World Civ 111 Fall 2011 / World Civ 111 Quiz.
  5. Click on "Add Content"
  6. Click "Folder" to create a folder for the course
    • All assessments, quizzes and essays for the course are to be placed here
  7. Name the folder according to the course name, e.g. "Geology 101"
  8. Save
  9. Click on "Add Content"
  10. Select Assessment
  11. Fill in page settings
    1. Content tab
      1. Title: create a name for the assessment
      2. Click on "Advanced" radio button to enable the HTML editor
      3. Add quiz instructions to the "Page Text" area. These will be visible to the student before beginning the assignment. E. g.: "This quiz is included as part of your grade for the course. Each question has only one correct answer"
    2. Access tab
      • The "View Restrictions" start and end dates control what appears to the student. It does not make an assessment or tutorial able to be interacted with, that function is controlled using the Interaction tab.
      • It is recommended to leave the start and end dates in the "Access" tab alone
      1. In "View Restrictions" section, set the "Viewable By" option to Members
    3. Interaction tab
      1. Do not modify "Delivery settings" at this point, they will be assigned once the assessment has been published into a course
      2. "Display Settings" section
        1. Select "Question at a time"
      3. "Question Set Defaults" section
        1. Check "Randomize the order in which questions are delivered"
        2. Check "Randomize the order of each question's answer options"
      4. "Submission Settings" section
        1. Set the number of Attempts Allowed
    4. Review tab
      1. Deselect all feedback options except Overall score, as displayed below
  12. Save
  13. Select "Add Question"
    • Do not click "Add Question Set"
  14. Select Copy Questions from an assessment (there are other options available, though this should be the standard one)
  15. Use the Choose Assessment dropdown box to select the appropriate quiz set
    • Click "Go" button
  16. Select the questions that you need to copy or use the "select all on this page" option
  17. Click "Ok" button
  18. To combine questions from additional sets, repeat steps 12-16

Course Assessments - Creating a Question Pool

 

  • To assign a subset of a larger question set, a Question Pool is required. A Question Pool can be comprised of many or a single quiz set. The procedure involves using the Question Bank Manager available in the Management Console. After creating a pool or pools of questions, they must be added to an assessment which can then be assigned to a course

  1. Begin at the Learning Object Repository , on the leftmost toolbar
  2. Select the Management Console using the "Manage" tab at the top of the page
  3. Click "Question Bank Manager"
  4. Click on the small down arrow next to Library Repository Pullman
  5. Select "Add Subfolder"
  6. Add a folder name
    • Name folder using the following format Question Bank: Coursename e.g.. Question Bank: World Civ 111
  7. Click the down arrow next to the newly created Question Bank (bottom of the list)
    • The following describes how to create a simple Question Pool similar to that formerly used in ILE, where quiz sets for multiple tutorials were combined into one list from which subsets were drawn from. More sophisticated assessment instruments can be made by creating multiple Question Pools nested within the Question Bank folder created in number 6. Each Question Pool can then be assigned a specific number of questions to draw from. Once Question Pools have been created they can be assigned as part of an assessment. For the purposes of this documentation, what follows is an example of a "simple Question Pool" only.
  8. Click "Add Question"
  9. Click "Copy Questions from Assessment"
  10. Select the Quiz Set to add to the Question Pool from the Choose Assessment dropdown menu
  11. Click "Go" button
  12. Select all the questions to be included in the Question Pool
  13. Click "OK"
  14. Repeat numbers 8 - 13 for each quiz set to be added to the Question Pool

Course Assessments - Incorporating a Question Pool into an Assessment

 

  1. Begin in the Publishable Content/Course folder
  2. Click "Add Content"
  3. Click "Assessment"
  4. Fill in page settings
    1. Content tab
      1. Title: create a name for the assessment
      2. Click on "Advanced" radio button to enable the HTML editor
      3. Add quiz instructions to the "Page Text" area. These will be visible to the student before beginning the assignment. E. g.: "This quiz is included as part of your grade for the course. Each question has only one correct answer"
    2. Access tab
      • The "View Restrictions" start and end dates control what appears to the student. It does not make an assessment or tutorial able to be interacted with, that function is controlled using the Interaction tab.
      • It is recommended to leave the start and end dates in the "Access" tab alone
      1. In "View Restrictions" section, set the "Viewable By" option to Members
    3. Interaction tab
      1. Do not modify "Delivery settings" at this point, they will be assigned once the assessment has been published into a course
      2. "Display Settings" section
        1. Select "Question at a time"
      3. "Question Set Defaults" section
        1. Check "Randomize the order in which questions are delivered"
        2. Check "Randomize the order of each question's answer options"
      4. "Submission Settings" section
        1. Set the number of Attempts Allowed
    4. Review tab
      1. Deselect all feedback options except Overall score, as displayed below
  5. Save
  6. Click "Add Question," do not click "Add Question Set"
    • At this point you should know the number of question that are in the Question Pool created above in addition to the number of questions that will be asked on the assessment
  7. Configure Question Pool Settings
    1. Click "Question Pool"
    2. Add the "Number of questions"
    3. Add the "Question Pool Points"
      • In order to maintain baseline expectations for shared resources, the "Question Pool Points" should be set to 100. Adjustments of the value of the assessment can be made when integrating into the course, that will determine the ultimate point value of the assignment
    4. Click the "Choose Folders" link
    5. Select the checkbox for the Question Bank which should have been created earlier. See Course Assessments - Creating a Question Pool
    6. Click "Done"
    7. Click "Save"

Course Assessments - Creating Essays

 

  • Essays are just a type of question in Angel. The essay question must be created as any other item in a repository, and later copied to an assessment within a course, as explained shortly.
  • Important to keep in mind: essay questions are always graded manually.
  • To Create an Essay Assignment:
  1. Begin at the Learning Object Repository , on the leftmost toolbar
  2. Click on a repository name (e.g. Library Repository Pullman or your own personal repository)
  3. Navigate to Assessments / Course Assessments
  4. Obs.: Collaboration with courses will span multiple semesters. A clean shared space will be maintained by using the following structure in the Course Assessments directory: 
    Course Name / Course Name Semester Year / content. e.g.: World Civ 111 / World Civ 111 Fall 2011 / World Civ 111 Quiz.
  5. Click on "Add Content"
  6. Click "Folder" to create a folder for the course
    • All assessments, quizzes and essays for the course are to be placed here
  7. Name the folder according to the course name, e.g. "Geology 101"
  8. Save
  9. Click on "Add Content"
  10. Select Assessment
  11. Give the essay a Title
  12. Save
  13. Select "Add Question"
  14. Select "Essay"
  15. Add the question
    1. Question title (optional)
    2. Assign 10 points as default value
    3. Add question text
    4. Add answer length (optional)
    5. Obs.: Leaving this field blank will result in an answer box of approximately 10 lines. Consider increasing it to at least 50 when intended response is to be longer, so as to be recognizable by the student
  16. Save

Quiz Question Unique Identification

Question codes are maintained in a file in the LOR, named Assessment / List of Quiz Sets / Assessment IDs

  1. Begin at the Learning Object Repository  on the leftmost toolbar
  2. Click on a repository name (e.g. Library Repository Pullman or your own personal repository)
  3. Navigate to Assessment / List of Quiz Sets
  4. Click on "Assessment IDs" webpage
  5. Verify if there is an existing entry for the question set to be created or modified
  6. If there's no entry, create one:
    1. Click on Settings
    2. In the HTML editor, go to the end of the list
    3. Add an ID and Title, using the next available integer as ID
    4. Save
  7. Take note of the ID to use when creating questions for a quiz set

Publishing a Lesson into a Course

 

  • Publishing provides another method of copying resources from one place to another. As it is a copy, tracking will not be available from the LOR, instead use and interaction data will need to be exported from the course.
  • A benefit of publishing, is that you can create an entire lesson in the LOR and then push it to the course in it's entirety. This method results in retaining a copy of the published content in the Library Repository
  • The process of building a lesson to be published into a course involves copying items in the repository into a folder to be published. Once the lesson content is built in entirety, it can be published into the course. Once in the course adjustments may be needed for proper labeling, access, and interaction settings.

Creating a Folder to Publish from

 

  1. Begin in the "Publishable Content" folder in the LOR
  2. Click on Add Content
  3. Click on "Folder" to create a space for the course. Name the folder after the course e.g.. World Civ 111
  4. Click "Save"
  5. Repeat steps 2 - 4 to create a folder for a specific semester
  6. Obs.: Collaboration with courses will span multiple semesters. A clean shared space will be maintained by using the following structure in the Course Assessments directory: 
    Course Name / Course Name Semester Year. e.g.: World Civ 111 / World Civ 111 Fall 2011.
  7. Click on "Settings" and Select the "Advanced" radio button
  8. Configure settings as described below
    1. Content tab
      1. Set the Title. Name the folder for the course and semester. e.g. World Civ 111 / World Civ 111 Fall 2011.
      2. Use the "Page Text" area to include instructions for the lesson
    2. Access tab
      • The "View Restrictions" start and end dates control what appears to the student. It does not make an assessment or tutorial able to be interacted with, that function is controlled using the Interaction tab.
      • It is recommended to leave the start and end dates in the "Access" tab alone
      1. Set User Tracking to "Everyone"
      2. Set Viewable by to "Members"
  9. Save

Building the Lesson in the LOR to Publish from

 

  • Here, we are going to pull together all the content that will be "published" to the course into one place for easy exporting of the material into the course
  1. Begin in the folder created in the Creating a Folder to Publish From procedure (LOR/Publishable Content/)
  2. Click on Add Content
  3. Click "Copy Items"
  4. Select items from the hierarchy
  5. Repeat as necessary to incorporate all content
  6. Click "Save"

Publish Lesson Content in LOR into a Course

 

  1. Begin in the folder in the LOR that was created for the publishable content (LOR/Publishable Content/Course folder)
  2. The order that the content appears is the order it will be displayed to students. Verify that the order is as intended and adjust as necessary
    1. Click "Rearrange" from top menu
    2. Drag and drop the content into the order intended
    3. Click "Save"
  3. Click "Publish"
  4. Use the checkboxes to select all the content to be published into the course
  5. Click "Continue"
  6. Use the interface provided to find the course for the content to be published into
  7. Select the course
  8. Use the "Copy Content" button to send the content to the course

Setting up the Lesson in the Course

 

  • The procedure for setting up a lesson in the course is made easier when all the content for the lesson has been published together.
  1. Begin in the "lessons" folder in the course the content was published to, and find the "Repository - Library Repository Pullman" folder
  2. Click "Settings"
  3. Adjust the lesson title to something appropriate for the course using the "settings" options
  4. Verify "Access Settings" are set to:
    • Access Tracking, User Tracking = "All Enrolled Users"
    • View Restrictions, Viewable By = "Students"
    • The "View Restrictions" start and end dates control what appears to the student. It does not make an assessment or tutorial able to be interacted with.
    • It is recommended to leave the start and end dates in the "Access" tab alone
  5. Verify Assessment Settings
  6. Verify the order of items in the lesson
  7. Save

Configure Access Dates & Connecting Assessments to Gradebook

 

  1. Navigate to one of the recently created module folders
  2. Click "settings" link under the assignment name
    1. Access tab
      1. Click on "advanced" radio button
      2. Set User Tracking to "Everyone"
      3. Viewable by "Students"
        • The "View Restrictions" start and end dates control what appears to the student. It does not make an assessment or tutorial able to be interacted with, that function is controlled using the Interaction tab.
        • It is recommended to leave the start and end dates in the "Access" tab alone
    2. Interaction tab
      • The "Interaction" tab includes "Delivery Settings" where start and end dates can be set for availability of a tutorial or assessment
      1. Under "Delivery Settings" set the enable and disabled dates for the assessment to be available for student work
      2. Verify "Display Settings" are set as desired. See recommendations for the Interactions tab settings inCourse Assessments - Combining Tutorial Quiz Sets into a Quiz
    3. Assignment tab
      1. Milestone Settings: leave Task Type with default value "(None)"
      2. In the "Gradebook Settings" section, set the values as shown in fig. 5
        1. Assignment: (New Assignment)
        2. Points Possible: The quiz value to be displayed to students when taking the assessment, and what is used for calculations in the gradebook
        3. Select the appropriate category for the assignment to be tied to the gradebook in, or create a new one (this should be discussed with the course instructor)
  3. Save
  4. Repeat as necessary for each assessment

Export Assignment Data

Assignment submissions can be exported from Angel to a CSV file, which contains all the students' answers and a mapping of values 1 and 0 to indicate whether or not they're correct. To process the data in a standard way, we'll create an Excel file with a reformatted version of this data.

Export Posted Submissions

 

  1. Go to Home  and select a course from the Courses box
  2. Click in Lessons
  3. Navigate to the assignment location
  4. Hover the mouse pointer over the assignment for option links to appear underneath
  5. Click on "Utilities" (*)
  6. Click on "Export Posted Submissions"
  7. (*) Obs.: This is for Angel 7.3. A separate "Submissions" option exists in Angel 7.4, which will be used instead of "Utilities" in this case. The new version is planned to be installed by August 3, 2011.
  8. In the Export screen (fig. 8)
    1. File Format: CSV
    2. You can leave all the options checked by default
    3. Click OK button
    4. Save the file to a desired location
  9.  

Obs.: In the CSV file each row (after 4th) belongs to a student submission. After the "GRADED BY" column, there are columns with the answers (each titled with the question text in row 3) and after them, "mapping" columns with values 1 or 0, representing whether or not the student answer was correct, respectively. They are related by the identifiers in row 4, e.g. FIELD_4daeaa8e-ad08-4117-85ab-e020fecc2a60 matches VAL_FIELD_4daeaa8e-ad08-4117-85ab-e020fecc2a60. See figure 9 for illustration 

Generate Excel file

The "standard" Excel file needs to be generated from the CSV, with a worksheet within it for every question. Each question worksheet will pair the answers and the mapping (1 or 0 values) for that question only. This will simplify further analysis of the data.

  1. Open the CSV file
  2. Create a new Excel file
  3. For each question in the CSV file
    1. Create a worksheet in the new file
    2. Copy the columns RESPONSE ID, GRADE, NAME, Login Name and DATE SUBMITTED to the new worksheet
    3. Copy the question column to the new worksheet
    4. Copy the corresponding mapping column to the new worksheet. Remember to verify the identifiers in row 4 for every match (like FIELD_4daeaa8e-ad08-4117-85ab-e020fecc2a60 matchingVAL_FIELD_4daeaa8e-ad08-4117-85ab-e020fecc2a60)
  4. Figure 10 shows a sample Excel file, with one worksheet per question, for the previously displayed CSV file. 

WSU Libraries, PO Box 645610, Washington State University, Pullman WA 99164-5610, 509-335-9671, Contact Us