<p>It sounds like you're looking for a platform on which to implement <a href="http://en.wikipedia.org/wiki/Computerized_adaptive_testing">computerized adaptive tests</a> (since subsequent questions are contingent on prior responses).</p>

<p>I found <a href="http://www.psychometrics.cam.ac.uk/page/300/concerto-testing-platform.htm">Concerto</a>, which is based in R and MySQL, but allows some flexibility in presentation (it says it uses HTML directly, but you could probably couple it with another language).</p>

CC BY-SA 3.0
2012-04-12T07:07:21.080Z
Looks interesting. Do you know if it is possible to have more than one item displayed onscreen? Furthermore, do you have an idea how to run it in a kind of controlled environment so that users cannot close the browser (better, there is no browser window) and need to perform the test?
Henrik
CC BY-SA 3.0
2012-04-12T07:15:39.523Z
@Henrik I have never used it.  What do you mean by having more than one item on screen? Multiple questions, or something like multiple visual tests at once?  I suppose you could put your own front end on it, or run a browser full screen, with the capability to diminish it disabled through the Windows Registry.  I saw the package more for its ability to do CAT and didn't study the UI elements.
Chuck Sherrington
CC BY-SA 3.0
2012-04-12T09:47:45.997Z
Regarding your first question I mean multiple questions. And I am really interested in a as much as possible out-of-the-box front end for displaying questionaires within an experimental session on screen with maximum control of what the user is allowed to do and minimum programming hassle.
Henrik
CC BY-SA 3.0
2012-04-12T16:53:41.477Z
`maximum control of what the user is allowed to do and minimum programming hassle` As I'm sure you know, these two are often mutually exclusive... I will probe Concerto a little further over the next couple of days.
Chuck Sherrington
CC BY-SA 3.0
2012-04-13T09:34:16.980Z
I think my statement was ambigous. I want to maximally control what the participant is allowed to do (i.e., he should not be able to close the borwser). So I am more interested in a neat frontend than in the adaptive testing abilities. BTW: I tried the demonstrations on the concerto website and only the first one runs (with firefox and ie on win 7).
Henrik
CC BY-SA 3.0
2012-04-13T10:43:37.930Z
@Henrik Your question led me to believe otherwise with `presenting some texts to the participants, then some questions contingent on this text and on what participants entered before` but I could certainly elaborate on the UI bit in the answer (I haven't had a chance to try their examples).  Controlling what the user has access to can be done through using a "kiosk" configuration.
Chuck Sherrington
CC BY-SA 3.0
2012-04-13T10:44:53.403Z
See [this SO question](http://stackoverflow.com/questions/4617303/does-windows-7-have-a-kiosk-mode), [SU question on IE kiosk](http://superuser.com/questions/392478/launch-ie-or-other-browser-in-kiosk-mode-across-multiple-monitors),  and [start with browser](http://superuser.com/questions/351342/how-to-start-computer-with-ie-open-and-cant-do-anything-else) (really any question with the `kiosk` tag that involves Windows on [SU](http://superuser.com/questions/tagged/kiosk) will be useful to you).
Chuck Sherrington
CC BY-SA 3.0
2012-04-13T11:15:17.183Z
Thanks. This is a lot more in the way of what I want to do. I hope there is an easy method to do this that also includes getting the `POST` command from the html-forms. I posted a smiliar question on SO yesterday: http://stackoverflow.com/questions/10121864/how-do-i-display-a-website-with-html-forms-locally-using-python-and-collect-the
Henrik
CC BY-SA 3.0
2012-04-11T23:07:28.990Z
2012-04-11T23:07:28.990Z
Chuck Sherrington
<p>You'll want to look into <a href="http://psychtoolbox.org/HomePage" rel="nofollow">PsychToolbox 3</a>, a very capable Matlab/Octave toolbox intended for running high-precision behavioral psychology experiments. If you don't have access to Matlab, you can use <a href="http://octave.sourceforge.net/" rel="nofollow">Octave</a> (which is available for Windows 7; see link) to run your experiments.</p>

<p>The toolbox itself is a platform for creating interactive experiments, ranging from simple question-based surveys to highly complex, precision visual displays. For the purpose at hand, I would build a simple function that displays text using PsychToolbox's <a href="http://docs.psychtoolbox.org/DrawText" rel="nofollow"><code>DrawText</code></a> or <a href="http://docs.psychtoolbox.org/DrawFormattedText" rel="nofollow"><code>DrawFormattedText</code></a> functions and monitors responses using the <code>Kb*</code> functions; see the <a href="http://docs.psychtoolbox.org/KbDemo" rel="nofollow"><code>KbDemo</code> page</a> for an overview. Each of these has their own demo showing how to use the function (the aptly-named <a href="http://docs.psychtoolbox.org/DrawSomeTextDemo" rel="nofollow"><code>DrawSomeTextDemo</code></a> and <a href="http://docs.psychtoolbox.org/DrawFormattedTextDemo" rel="nofollow"><code>DrawFormattedTextDemo</code></a> functions, and the <code>KbDemo</code> one mentioned earlier); simply check the source code of the demos to see how they work.</p>

<p>I will mention that PsychToolbox has learning curve, and requires familiarity with Matlab/Octave programming. That being said, with the above functions, it should be pretty easy to create a generic "survey" function which accepts however many lines of text (one question, however many answers), displays them to the user, and monitors and records feedback.</p>

CC BY-SA 3.0
2012-04-12T07:02:27.187Z
My quick look at the homepage of PsychToolbox3 somehow gives the impression that it is another tool for running reaction time based experiments (i.e., a focus on accurate timing). Can you elaborate a little on which functions are intended to be used to display questionnaires and similar stuff?
Henrik
CC BY-SA 3.0
2012-04-12T14:28:12.870Z
Thanks for this expansion. However, from what I get PsychToolbox does not really differ from PsychoPy or other related approaches (e.g., Tscope). You still need to completely draw the whole screen elementwise. This is not what I consider easy to use for presenting questionnaires. Especially not if you want something like radiobuttons, checkboxes and input fields and want poeple to use the mouse. I especially are looking for some program that uses a different approach.
Henrik
CC BY-SA 3.0
2012-04-12T15:07:39.517Z
@Henrik - Makes sense, and I agree, if you don't want to program this may not be for you. That being said, free setups that don't require programming aren't that common. More often than not, with the proprietary solutions, you're paying for the interface.
eykanal
CC BY-SA 3.0
2012-04-13T09:36:13.920Z
Its not that I dont want to program. I just want a way to present items (e.g., likert type) within an experimental session without the need to always program each question per hand in wxPython (which means positioning the text, the buttons, the box, ...).
Henrik
CC BY-SA 3.0
2012-04-12T02:18:09.730Z
2012-04-12T12:03:13.410Z
2012-04-12T12:03:13.410Z
eykanal
<p>I am <strong>looking for free software to build experiments that do not heavily rely on accurate timing</strong> (where I use <a href="http://www.psychopy.org/" rel="nofollow noreferrer">PsychoPy</a> and heard a lot of good things about <a href="http://www.cogsci.nl/software/opensesame" rel="nofollow noreferrer">OpenSesame</a>) but more on an easy way to implement different types of items and response options.</p>
<p>That is, for example, an experiment which starts in presenting some texts to the participants, then some questions contingent on this text and on what participants entered before (potentially more than one question per page with different response options) followed by some standard questionnaires. How would you implement such a study?</p>
<hr />
<h3>Background and motivation</h3>
<p>I used to do such things using <a href="http://www.empirisoft.com/medialab.aspx" rel="nofollow noreferrer">MediaLab</a> which unfortunately is proprietary software. However, implementing items and questionnaires was easy. You could program a simple html page (even using variables and placeholders) using html forms and MediaLab would present this page using ie engine in fullscreen and collect the responses. Those responses could then be used or handed over to DirectRT for response time experiments.</p>
<p>Currently I am using <a href="http://www.psychopy.org/" rel="nofollow noreferrer">PsychoPy</a> to control the screen and data collection and code all questionnaires and such stuff per hand using <a href="http://www.wxpython.org/" rel="nofollow noreferrer">wxPython</a>. It works, but designing questionnairs in wxPython is a lot less handy and more difficult than programming html pages with html forms and definitely nothing for people not too profficient in coding.</p>
<p>Any ideas or hints would be really appreciated.</p>
<p>Note that I read <a href="https://cogsci.stackexchange.com/q/508/604">&quot;What's the best program for creating computer based psychology experiments under OS X?&quot;</a>, but my question concerns specific implementations. Furthermore, I work on Windows 7.</p>

CC BY-SA 3.0
2012-04-11T23:23:44.453Z
Does the software need to be able to run on the Internet?
Jeromy Anglim
CC BY-SA 3.0
2012-04-12T00:45:47.253Z
How important is free? There are online survey providers (e.g., surveymonkey, questionpro, and many more) that support sophisticated questionnaire delivery (e.g., conditional logic, branching, etc.). Such sites  typically have some low-cost options and even a limited free option.
Jeromy Anglim
CC BY-SA 3.0
2012-04-12T06:54:20.893Z
@JeromyAnglim I really prefer free and especially non-pure web based as I intend to use it in combination with other approaches like PsychoPy.
Henrik
CC BY-SA 4.0
2022-03-14T18:43:22.257Z
If you want to use HTML forms in PsychoPy, then there is this solution: https://discourse.psychopy.org/t/custom-web-component-for-online-experiments-forms-surveys-questionnaires-and-other-web-based-content/14423
Arnon Weinberg
<methodology><software><experimental-psychology>
Free software to run questionnaire based experiments
Henrik
# Free software to run questionnaire based experiments

(Asked by: username_0 on 2012-04-11)

I am **looking for free software to build experiments that do not heavily rely on accurate timing** (where I use [PsychoPy](http://www.psychopy.org/) and heard a lot of good things about [OpenSesame](http://www.cogsci.nl/software/opensesame)) but more on an easy way to implement different types of items and response options.

That is, for example, an experiment which starts in presenting some texts to the participants, then some questions contingent on this text and on what participants entered before (potentially more than one question per page with different response options) followed by some standard questionnaires. How would you implement such a study?

---

### Background and motivation

I used to do such things using [MediaLab](http://www.empirisoft.com/medialab.aspx) which unfortunately is proprietary software. However, implementing items and questionnaires was easy. You could program a simple html page (even using variables and placeholders) using html forms and MediaLab would present this page using ie engine in fullscreen and collect the responses. Those responses could then be used or handed over to DirectRT for response time experiments.

Currently I am using [PsychoPy](http://www.psychopy.org/) to control the screen and data collection and code all questionnaires and such stuff per hand using [wxPython](http://www.wxpython.org/). It works, but designing questionnairs in wxPython is a lot less handy and more difficult than programming html pages with html forms and definitely nothing for people not too profficient in coding.

Any ideas or hints would be really appreciated.

Note that I read ["What's the best program for creating computer based psychology experiments under OS X?"](https://cogsci.stackexchange.com/q/508/604), but my question concerns specific implementations. Furthermore, I work on Windows 7.

**Tags:** methodology, software, experimental-psychology

> **Comments:**
> **username_1**: Does the software need to be able to run on the Internet?  
> **username_1**: How important is free? There are online survey providers (e.g., surveymonkey, questionpro, and many more) that support sophisticated questionnaire delivery (e.g., conditional logic, branching, etc.). Such sites  typically have some low-cost options and even a limited free option.  
> **username_0**: @JeromyAnglim I really prefer free and especially non-pure web based as I intend to use it in combination with other approaches like PsychoPy.  
> **username_2**: If you want to use HTML forms in PsychoPy, then there is this solution: https://discourse.psychopy.org/t/custom-web-component-for-online-experiments-forms-surveys-questionnaires-and-other-web-based-content/14423  

---

## 2 answers

### Answer by username_3 on 2012-04-11. Score: 6

It sounds like you're looking for a platform on which to implement [computerized adaptive tests](http://en.wikipedia.org/wiki/Computerized_adaptive_testing) (since subsequent questions are contingent on prior responses).

I found [Concerto](http://www.psychometrics.cam.ac.uk/page/300/concerto-testing-platform.htm), which is based in R and MySQL, but allows some flexibility in presentation (it says it uses HTML directly, but you could probably couple it with another language).

> **Comments:**
> **username_0**: Looks interesting. Do you know if it is possible to have more than one item displayed onscreen? Furthermore, do you have an idea how to run it in a kind of controlled environment so that users cannot close the browser (better, there is no browser window) and need to perform the test?  
> **username_3**: @username_0 I have never used it.  What do you mean by having more than one item on screen? Multiple questions, or something like multiple visual tests at once?  I suppose you could put your own front end on it, or run a browser full screen, with the capability to diminish it disabled through the Windows Registry.  I saw the package more for its ability to do CAT and didn't study the UI elements.  
> **username_0**: Regarding your first question I mean multiple questions. And I am really interested in a as much as possible out-of-the-box front end for displaying questionaires within an experimental session on screen with maximum control of what the user is allowed to do and minimum programming hassle.  
> **username_3**: `maximum control of what the user is allowed to do and minimum programming hassle` As I'm sure you know, these two are often mutually exclusive... I will probe Concerto a little further over the next couple of days.  
> **username_0**: I think my statement was ambigous. I want to maximally control what the participant is allowed to do (i.e., he should not be able to close the borwser). So I am more interested in a neat frontend than in the adaptive testing abilities. BTW: I tried the demonstrations on the concerto website and only the first one runs (with firefox and ie on win 7).  
> **username_3**: @username_0 Your question led me to believe otherwise with `presenting some texts to the participants, then some questions contingent on this text and on what participants entered before` but I could certainly elaborate on the UI bit in the answer (I haven't had a chance to try their examples).  Controlling what the user has access to can be done through using a "kiosk" configuration.  
> **username_3**: See [this SO question](http://stackoverflow.com/questions/4617303/does-windows-7-have-a-kiosk-mode), [SU question on IE kiosk](http://superuser.com/questions/392478/launch-ie-or-other-browser-in-kiosk-mode-across-multiple-monitors),  and [start with browser](http://superuser.com/questions/351342/how-to-start-computer-with-ie-open-and-cant-do-anything-else) (really any question with the `kiosk` tag that involves Windows on [SU](http://superuser.com/questions/tagged/kiosk) will be useful to you).  
> **username_0**: Thanks. This is a lot more in the way of what I want to do. I hope there is an easy method to do this that also includes getting the `POST` command from the html-forms. I posted a smiliar question on SO yesterday: http://stackoverflow.com/questions/10121864/how-do-i-display-a-website-with-html-forms-locally-using-python-and-collect-the  

---

### Answer by username_4 on 2012-04-12. Score: 5

You'll want to look into [PsychToolbox 3](http://psychtoolbox.org/HomePage), a very capable Matlab/Octave toolbox intended for running high-precision behavioral psychology experiments. If you don't have access to Matlab, you can use [Octave](http://octave.sourceforge.net/) (which is available for Windows 7; see link) to run your experiments.

The toolbox itself is a platform for creating interactive experiments, ranging from simple question-based surveys to highly complex, precision visual displays. For the purpose at hand, I would build a simple function that displays text using PsychToolbox's [`DrawText`](http://docs.psychtoolbox.org/DrawText) or [`DrawFormattedText`](http://docs.psychtoolbox.org/DrawFormattedText) functions and monitors responses using the `Kb*` functions; see the [`KbDemo` page](http://docs.psychtoolbox.org/KbDemo) for an overview. Each of these has their own demo showing how to use the function (the aptly-named [`DrawSomeTextDemo`](http://docs.psychtoolbox.org/DrawSomeTextDemo) and [`DrawFormattedTextDemo`](http://docs.psychtoolbox.org/DrawFormattedTextDemo) functions, and the `KbDemo` one mentioned earlier); simply check the source code of the demos to see how they work.

I will mention that PsychToolbox has learning curve, and requires familiarity with Matlab/Octave programming. That being said, with the above functions, it should be pretty easy to create a generic "survey" function which accepts however many lines of text (one question, however many answers), displays them to the user, and monitors and records feedback.

> **Comments:**
> **username_0**: My quick look at the homepage of PsychToolbox3 somehow gives the impression that it is another tool for running reaction time based experiments (i.e., a focus on accurate timing). Can you elaborate a little on which functions are intended to be used to display questionnaires and similar stuff?  
> **username_0**: Thanks for this expansion. However, from what I get PsychToolbox does not really differ from PsychoPy or other related approaches (e.g., Tscope). You still need to completely draw the whole screen elementwise. This is not what I consider easy to use for presenting questionnaires. Especially not if you want something like radiobuttons, checkboxes and input fields and want poeple to use the mouse. I especially are looking for some program that uses a different approach.  
> **username_4**: @username_0 - Makes sense, and I agree, if you don't want to program this may not be for you. That being said, free setups that don't require programming aren't that common. More often than not, with the proprietary solutions, you're paying for the interface.  
> **username_0**: Its not that I dont want to program. I just want a way to present items (e.g., likert type) within an experimental session without the need to always program each question per hand in wxPython (which means positioning the text, the buttons, the box, ...).  

---

