Accounting for slight variations in choices which are intended to be “the same”
This article outlines a procedure to use in Qualtrics to counterbalance two choices within a single question. This is useful when you are offering a choice between to items which are intended to be the same, with the exception of some experimentally manipulated item. But when you offer two choices, it is obvious your intention if you make them the same, so instead you make them similar. This leads to problems, however. Are differences you see in the data the result of the experimental manipulation or a slight preference for one of the options? You can correct this by counterbalancing, associating your manipulation with one choice to half the participants, and to the other choice with the other half. Here’s how.