Guide
      Back to home
      1. Administrator Knowledge Base
      2. Subject Matter Expert Resources
      3. Guide

      Item Writing Guide

      Item writing guidelines for Subject Matter Experts

      1. Purpose

      Thank you for your participation! As a subject matter expert (SME) in your field, you will be helping to create items for an upcoming exam. This exam will help determine whether or not test takers have the necessary knowledge and skills to work in a specified role in your industry.

      2. Security and Confidentiality

      Because these items might be used on an upcoming exam, it is extremely important that you keep the item completely private and unavailable to view by others, including while working on electronic devices such as cell phones.

      3. Overall Process

      There are three main steps to creating items. Most experts participate in all of these activities, so please note that you might be asked to also review items.

      1. Write Items: A SME writes an item.
      2. Item Review: A different SME(s) reviews the item based on research-based item writing principles. The item either “passes” the review or is returned to the item writer for revision. Oftentimes an item is required to pass multiple reviews from different SMEs.
      3. Revision/Revise Items: The initial SME revises the item while considering the feedback from the reviewer. (Initial feedback is common, so please be aware that you might be receiving input from reviewers.)

      4. Choosing a Topic

      From the Authoring Portal, select “Write Items” under your exam name.

      Each Certiverse exam is created based on a list of pre-determined topics, along with an estimated number of items that need to be written for each topic. Certiverse also includes descriptions, which provide detailed information about the topics.

      To start writing items, you must first choose a topic. Make sure to read the description carefully and understand its goal. It is critical that each item aligns with its respective topic.

      Select “Write” to choose your topic and begin writing an item.

      5. Item Writing

      Certiverse has two different ways for you to write your items:

      1. The authoring wizard, which walks you through the process of writing each part of the item.
      2. AI-assisted item writing, where an AI will suggest an item for your review and revision.

      AI-Assisted Item Writing

      We’ve already written most of a prompt for an AI engine. All you need to add is the specific topic of your item. Type exactly what you want the item to be about (e.g., “what SQL statement stores data into a database” or “what civil rights law

      The AI engine will then suggest three items. We expect the AI to continue to get better, but our current research suggests that over 80% of items will need at least minor revision, so you should review the suggestions and pick one that is a good start. The AI is good (not perfect) at making up well-formed sentences that are related to your topic, but you should check the correctness of the answer(s) marked as correct (the “key(s)”) and that the distractors are truly incorrect. You may need to select the best key(s) and then edit the items.

      You can reject an item and it will be replaced (up to 10 tries). You can also “Try Again” and revise your specific item topic.

      When you accept an item, it will be loaded into the Authoring Wizard for your review and revision.

      Tip: Make sure you select the answer(s) that is/are the best key. That way, those answers will be loaded into the “Correct Response(s)” field(s) of the authoring wizard.

      Authoring Wizard 

      The authoring wizard walks you through writing each part of an item. You’ll also use this interface to edit AI-generated content and to review your items. (The authoring wizard can also be switched the the standard item writing page – select “Standard” to switch views. It’s recommended that you use the wizard.)

      The exam may contain one or a variety of the following item types:

      1. Multiple Choice - Single Correct Response - Two Options
      2. a question
      3. one correct answer
      4. one incorrect answer
      1. Multiple Choice - Single Correct Response - Three Options
      2. a question
      3. one correct answer
      4. two incorrect answers
      1. Multiple Choice - Single Correct Response - Four Options
      2. a question
      3. one correct answer
      4. three incorrect answers
      1. Multiple Choice - Single Correct Response - Five Options
      2. a question
      3. one correct answer 
      4. four incorrect answers
      1. Multiple Correct Responses - Dichotomous Scoring - Four Options, Two Correct, 2 Incorrect
      2. a question
      3. two correct answers
      4. two incorrect answers
      1. Multiple Correct Responses - Dichotomous Scoring - Four Options, Three Correct, 1 Incorrect
      2. a question
      3. three correct answers
      4. one incorrect answers
      1. Multiple Correct Responses - Dichotomous Scoring - Five Options, Two Correct, 3 Incorrect
      2. a question
      3. two correct answers
      4. three incorrect answers
      1. Multiple Correct Responses - Dichotomous Scoring - Six Options, Two Correct, 4 Incorrect
      2. a question
      3. two correct answers
      4. four incorrect answers
      1. Multiple Correct Responses - Dichotomous Scoring - Six Options, Three Correct, 3 Incorrect
      2. a question
      3. three correct answers
      4. three incorrect answers
      1. Multiple Correct Responses - Dichotomous Scoring - Seven Options, Two Correct, 5 Incorrect
      2. a question
      3. two correct answers
      4. five incorrect answers
      1. Multiple Correct Responses - Dichotomous Scoring - Seven Options, Three Correct, 4 Incorrect
      2. a question
      3. three correct answers
      4. four incorrect answers
      1. Multiple Correct Responses - Dichotomous Scoring - Eight Options, Two Correct, 6 Incorrect
      2. a question
      3. two correct answers
      4. six incorrect answers
      1. Multiple Correct Responses - Dichotomous Scoring - Eight Options, Three Correct, 5 Incorrect
      2. a question
      3. three correct answers
      4. five incorrect answers
      1. Multiple Correct Responses - Dichotomous Scoring - Eight Options, Four Correct, 4 Incorrect
      2. a question
      3. four correct answers
      4. four incorrect answers

      Additionally, each item requires:

      1. a rationale for each correct and incorrect answer. The rationale helps the reviewer understand why the correct answer is truly correct and the incorrect answers are truly incorrect.
      2. your estimation of the item difficulty; by asking what percentage of test takers will answer it correctly. Your response is based on your own evaluation of the item.
      3. a cited reference, which includes support for the correct answer and rationale. These references help avoid opinion-based items and will be used by reviewers to verify the rationale.

      Writing Principles for the Item

      Writing principles are research-based guidelines authors should follow to help create high-quality items that are more likely to pass the review process. Please ensure that each item:

      1. focuses on significant, non-trivial information related to the topic.
      2. is clear, grammatically correct, and free from unnecessary words or details.
      3. possesses only one meaning (cannot be interpreted multiple ways).
      4. does not include “trickery” intended to misdirect from the correct answer.
      5. is inoffensive and free from bias (age, ethnicity, gender, disabilities, etc.).
      6. contains all the information needed to determine the correct answer and does not require the test taker to make assumptions.
      7. avoids negative phrasing (see Example A).

      Items should focus on higher-order thinking, where test takers apply their knowledge to situations. Items focused on lower-order thinking, such as simple recall of facts, are less useful (see Example B).

      Writing Principles for the Correct and Incorrect Answers

      In addition to focusing on the question, it is also important to write high-quality answers. Please ensure that the correct and incorrect answers:

      1. have a similar format and are of similar length to ensure that the correct answer does not stand out (see Example C).
      2. are independent in content and do not “give away” the correct answer (see Example D).
      3. avoid specific determiners such as “never,” “always,” or “only” to make them false (see Example E).
      4. avoid complex multiple-choice items with answers such as: “all of the above,” “none of the above,” “A and D,” etc.

      Good incorrect answers include common misconceptions and words that “sound official.” They should help distinguish between test takers who have and have not mastered the topic.

      6. Examples

      Note: Correct answer is in bold font.

      EXAMPLE

      INCORRECT

      CORRECT

      A

       

      (Item avoids negative phrasing.)

      With what condition is the fluid imbalance known as edema NOT commonly associated?

       

      a) Viral infections.

      b) Congestive heart failure.

      c) Allergic reactions.

      d) Protein deficiency.

      With what condition is the fluid imbalance known as edema commonly associated?

       

      a) Congestive heart failure.

      b) Viral infections.

      c) Multiple Sclerosis.

      d) Asthma.

      B

       

      (Item focus on higher- order thinking.)

      Who was the second President of the United States?

       

      a) John Adams.

      b) Thomas Jefferson.

      c) James Madison.

      d) James Monroe.

      Historians characterize the 22nd Amendment to the U.S. Constitution as a reaction to the presidency of which person?

       

      a) Franklin D. Roosevelt.

      b) Herbert Hoover.

      c) Calvin Coolidge.

      d) Warren G. Harding.

      C

       

      (Correct and incorrect answers have similar format and length.)

      Which of the following is the best indication of high morale in a supervisor’s unit?

       

      a) The employees are willing to give first priority to attaining group objectives, subordinating any personal desires they may have.

      b) The employees are rarely required to work overtime.

      c) The supervisor enjoys staying late to plan the next day.

      d) The unit gives expensive birthday presents to each other.

      Which of the following is the best indication of high morale in a supervisor’s unit?

       

      a) The employees willingly give first priority to group objectives.

      b) The employees are rarely required to work overtime.

      c) The supervisor enjoys staying late to plan for the next day.

      d) The unit members give expensive birthday presents to each other.

      D

       

      (Answers are independent and do not “give away” the correct answer.)

      When conducting library research in education, which of the following is the best source to use for identifying pertinent journal articles?

       

      a) Current Index to Journals in Education.

      b) A Guide to Sources of Education Information.

      c) Resources in Education.

      d) The International Encyclopedia of Education.

      When conducting library research in education, which of the following is the best source to use for identifying pertinent articles?

       

      a) Current Index to Journals in Education.

      b) A Guide to Sources of Education Information.

      c) Resources in Education.

      d) The International Encyclopedia of Education.

      E

       

      Avoid specific determiners (e.g., never, always, only.)

      What should you do to avoid infection after receiving a puncture wound to the hand?

       

      a) Ensure that no foreign object has been left in the wound.

      b) Always go to the immunization center to receive a tetanus shot.

      c) Be treated with an antibiotic only if the wound is painful.

      d) Never wipe the wound with alcohol unless it is still bleeding.

      What should you do to avoid infection after receiving a puncture wound to the hand?

       

      a) Ensure that no foreign object has been left in the wound.

      b) Go to the immunization center to receive a tetanus shot.

      c) Be treated with an antibiotic when the wound is painful.

      d) Wipe the wound with alcohol unless it is bleeding.

      Contact Us

      If you have any questions or need additional assistance, please contact us by either emailing support@certiverse.com or by submitting a ticket from this article.

      References

      Burton, S. J., Sudweeks, R. R., Merrill, P. F., Wood, B. 1991. How to Prepare Better Multiple-Choice Test Items: Guidelines for University Faculty. Brigham Young University Testing Services and The Department of Instructional Science. https://testing.byu.edu/handbooks/betteritems.pdf

      Haladyna, T. M. & Downing S. M. (1989). A taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 1, 37-50. https://doi.org/10.1207/s15324818ame0201_3

      Rush, B. R., Rankin, D. C., & White, B. J. (2016). The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value. BMC Medical Education, 16, 1-10. Article 250. https://doi.org/10.1186/s12909-016-0773-3

      Example items are adapted, with permission, from the referenced Burton, Sudweeks, Merrill and Wood guide (1991). Example C created by Certiverse 2020.