<p>You could try www.gorilla.sc  It's cloud based software specifically designed for running cognitive psychology experiments online without needing to code.  It's not free, but it is affordable - £0.75 per participant.</p>

<p>Counterbalancing of various sorts is built in; randomisers, stimuli set counterbalancers, order counterbalancers.  It sounds like the simple <strong>randomiser</strong> set to <strong>balanced</strong> will do what you want.  As long as you stay on top of rejecting Ns that drop out, you'll get a balance in each condition.</p>

<p>There is a GUI questionnaire builder, task builder and experiment tree builder.  So you don't need to do any coding.  The vast majority of cognitive science task can be created in it very easily.</p>

<p>Full disclosure, I created Gorilla.</p>

CC BY-SA 3.0
2017-12-04T15:34:48.907Z
Thanks! This is really useful! We'll have to look at our budget though, we didn't calculate these costs in so far. :)
puslet88
CC BY-SA 3.0
2017-12-04T16:58:19.843Z
Typically, people use budget set aside for participant fees and commissions or for online hosting of their experiment.  From your OP it's sounds like your plan was to use open source software.  This usually requires dialing up your own server to host the experiment, which nearly always has a cost.  So, with hope that will have been budgeted for.  With Gorilla, you won't need to provide your own server as that is all taken care of for you in the 'per respondent' fee. :)
Jo Evershed
CC BY-SA 3.0
2017-12-05T19:41:01.097Z
Yes, I definitely see the appeal! In this case it's just a small start-up experiment so we haven't planned for anything but the bare essentials. But I know what you mean, and we'll definitely know to keep in mind in the future. :)
puslet88
CC BY-SA 3.0
2017-12-04T11:01:01.803Z
2017-12-04T11:01:01.803Z
Jo Evershed
<p>Qualtrics (survey software) and Inquisit (more sophisticated stimulus presentation software) both run online. Both tools support counterbalancing. Inquisit has extensive support for both randomisation of stimuli in a variety of ways and between-subjects allocations to orders and so on. They both require payment, but some universities have subscriptions to one or both services (especially Qualtrics).</p>

<p>I'm not sure about how you'd solve the issue of drop out to get perfectly even numbers in each group ordering. </p>

CC BY-SA 3.0
2017-12-04T15:35:21.187Z
Thanks! We'll look into them and our university subscriptions, maybe we find a match!
puslet88
CC BY-SA 3.0
2017-12-04T10:07:41.550Z
2017-12-04T10:07:41.550Z
Jeromy Anglim
<p>There are a number of different interfaces to make an place experiments online, e.g. in questions <a href="https://cogsci.stackexchange.com/questions/699/software-for-online-psychological-experiments-that-dont-require-users-to-downlo">Software for online psychological experiments that don&#39;t require users to download anything</a> or <a href="https://cogsci.stackexchange.com/questions/109/open-source-software-for-running-internet-psychological-experiments-that-collect">Open source software for running Internet psychological experiments that collect reaction time data</a>.</p>

<p>We have an experimental design that we'd share online, but we've found that one feature may be quite rare - namely counterbalancing. We have 4 conditions, and we'd like participants to be placed in these conditions so that we have more or less equal amount in each by the end. This can be difficult for web studies since it should only count the participants that successfully completed the trial.</p>

<p>Free and open source would be best, we don't have a designated budget to pay for setting up an experiment.</p>

CC BY-SA 3.0
2017-12-02T22:14:20.683Z
Maybe an off suggestion, but I think to add a counterbalancing approach online is going to be challenging; couldn't you simply do that post hoc?
AliceD
CC BY-SA 3.0
2017-12-03T10:18:21.667Z
Thanks, yea one option would be to have a setup for each type and manually allocate participants. It's not impossible to check the results file for how many there are of each condition and allocate new participants accordingly. It won't be perfect but will be balanced.
Simply post-hoc is difficult if we have limited resources and need to make sure we've collected the right conditions with the money.
puslet88
<methodology><experimental-psychology><software><internet>
Platforms to make online experiments with counterbalancing
puslet88
# Platforms to make online experiments with counterbalancing

(Asked by: username_0 on 2017-12-02)

There are a number of different interfaces to make an place experiments online, e.g. in questions [Software for online psychological experiments that don't require users to download anything](https://cogsci.stackexchange.com/questions/699/software-for-online-psychological-experiments-that-dont-require-users-to-downlo) or [Open source software for running Internet psychological experiments that collect reaction time data](https://cogsci.stackexchange.com/questions/109/open-source-software-for-running-internet-psychological-experiments-that-collect).

We have an experimental design that we'd share online, but we've found that one feature may be quite rare - namely counterbalancing. We have 4 conditions, and we'd like participants to be placed in these conditions so that we have more or less equal amount in each by the end. This can be difficult for web studies since it should only count the participants that successfully completed the trial.

Free and open source would be best, we don't have a designated budget to pay for setting up an experiment.

**Tags:** methodology, experimental-psychology, software, internet

> **Comments:**
> **username_1**: Maybe an off suggestion, but I think to add a counterbalancing approach online is going to be challenging; couldn't you simply do that post hoc?  
> **username_0**: Thanks, yea one option would be to have a setup for each type and manually allocate participants. It's not impossible to check the results file for how many there are of each condition and allocate new participants accordingly. It won't be perfect but will be balanced. Simply post-hoc is difficult if we have limited resources and need to make sure we've collected the right conditions with the money.  

---

## 2 answers

### Answer by username_2 on 2017-12-04. Score: 3

You could try www.gorilla.sc It's cloud based software specifically designed for running cognitive psychology experiments online without needing to code. It's not free, but it is affordable - £0.75 per participant.

Counterbalancing of various sorts is built in; randomisers, stimuli set counterbalancers, order counterbalancers. It sounds like the simple **randomiser** set to **balanced** will do what you want. As long as you stay on top of rejecting Ns that drop out, you'll get a balance in each condition.

There is a GUI questionnaire builder, task builder and experiment tree builder. So you don't need to do any coding. The vast majority of cognitive science task can be created in it very easily.

Full disclosure, I created Gorilla.

> **Comments:**
> **username_0**: Thanks! This is really useful! We'll have to look at our budget though, we didn't calculate these costs in so far. :)  
> **username_2**: Typically, people use budget set aside for participant fees and commissions or for online hosting of their experiment.  From your OP it's sounds like your plan was to use open source software.  This usually requires dialing up your own server to host the experiment, which nearly always has a cost.  So, with hope that will have been budgeted for.  With Gorilla, you won't need to provide your own server as that is all taken care of for you in the 'per respondent' fee. :)  
> **username_0**: Yes, I definitely see the appeal! In this case it's just a small start-up experiment so we haven't planned for anything but the bare essentials. But I know what you mean, and we'll definitely know to keep in mind in the future. :)  

---

### Answer by username_3 on 2017-12-04. Score: 2

Qualtrics (survey software) and Inquisit (more sophisticated stimulus presentation software) both run online. Both tools support counterbalancing. Inquisit has extensive support for both randomisation of stimuli in a variety of ways and between-subjects allocations to orders and so on. They both require payment, but some universities have subscriptions to one or both services (especially Qualtrics).

I'm not sure about how you'd solve the issue of drop out to get perfectly even numbers in each group ordering.

> **Comments:**
> **username_0**: Thanks! We'll look into them and our university subscriptions, maybe we find a match!  

---

