Citation Requirement

This tutorial and the accompanying resources utilize methods presented in the following working paper:

Velez, Y., Liu, P., & Clifford, S. (2025). When Information Affects Attitudes: The Effectiveness of Targeting Attitude-Relevant Beliefs. APSA Preprints. doi: 10.33774/apsa-2025-m6rbw

Note: This content is a preprint and has not been peer-reviewed.

If you utilize or adapt the concepts, code, or methodologies from this tutorial in your own work, please ensure you appropriately cite the source.

@misc{velez2025when, title = {When Information Affects Attitudes: The Effectiveness of Targeting Attitude-Relevant Beliefs}, author = {Velez, Yamil and Liu, Patrick and Clifford, Scott}, year = {2025}, howpublished = {APSA Preprints}, doi = {10.33774/apsa-2025-m6rbw}, note = {This content is a preprint and has not been peer-reviewed.} }
Creating an AI-Powered Open-Ended Question

Step 1: Download QSF File

To begin, download the Qualtrics Survey File (QSF) for this setup:

  1. Click on the following link: Download Template
  2. Save the file to your computer. Remember where you save it!

This QSF file contains the basic structure of the survey with placeholders for the AI-generated content.

Step 2: Import QSF into Qualtrics

Qualtrics import illustration

Now, let's import the QSF file into Qualtrics:

  1. Log in to your Qualtrics account.
  2. Click on "Create new project".
  3. Select "Survey" from the options.
  4. Click on "How Do You Want to Start a Survey?".
  5. Choose "Import a QSF File".
  6. Navigate to and select the QSF file you just downloaded.
  7. Click "Create a Project".

Your survey template should now be loaded into Qualtrics.

Step 3: Obtain API Key

API key illustration

To connect with an AI service, you'll need an API key. This is a crucial step; without an API key, no content will be generated.

You have two options:

Option 1: Fireworks AI (Open Source Llama 3.1-Instruct 70B default)

  1. Go to Fireworks AI.
  2. Sign up for an account if you haven't already.
  3. Navigate to the API section.
  4. Generate a new API key.
  5. Copy the API key and keep it secure.

Option 2: OpenAI (GPT-4o Mini)

  1. Go to OpenAI.
  2. Sign up for an account if you haven't already.
  3. Navigate to the API section.
  4. Create a new API key.
  5. Copy the API key and keep it secure.

Step 4: Configure Settings

Embedded Data Fields Configuration

4.1 API Key Configuration

  1. In the Survey Flow, locate the Embedded Data Field "API_KEY".
  2. Replace the placeholder value with your actual API key.

4.2 API Selection and Additional Settings

  1. Locate the Embedded Data Field "open".
  2. Set it to `no` if using OpenAI, or leave it as `yes` if using Fireworks AI (default model is Llama 3.1 70B).
  3. Locate the Embedded Data Field "maxTurns" and set the desired number of conversational turns.
  4. Ensure the Embedded Data Field "chatHistory" is empty to hold the conversation history.

Step 5: Run the Survey

Closed and Open-Ended Block Illustration

Now it's time to test your survey:

  1. Click on the "Preview Survey" button in Qualtrics.
  2. Go through the survey, answering the closed-ended question.
  3. The next screen will be a chatbot exchange about the topic of the closed-ended question and the user's position on it.

The survey should now present AI-generated content tailored to the participant's responses.

Step 1 of 5