Conditional biases and unaccounted for interactions in psychology experiments
Find out more
Unbeknown to you systematic conditional biases in your computer-based psychology experiment could account for your whole experimental effect. An unintended consequence of unknown biases is the classic file drawer problem in psychological research.

Replication lies at the heart of good science and the scientific method. Repeatability is key. If you think you are holding all other variables constant but later find out many more variables were at play where does that leave your findings and your reputation?

One classic example from our own testing was in a cross modal priming study where it was discovered that on trials where a visual stimulus was presented alongside audio there was a systematic discrepancy in measured reaction time. This error was due to the fact that the software running the experiment could not reliably process the onset of a reaction time while a sound was playing. Without independently checking with a Black Box ToolKit v2 the researcher would have assumed that the experimental effect was real as might you in your own study.

In the case of the cross model priming study the equipment was setup as shown below (Apple Mac). Three Black Box ToolKit v2 opto-detectors monitored for any one of three stars appearing on screen, two digital microphones checked for a sound presentation and an Active Switch Closure was wired into the studies regular response box. When a visual stimulus appeared a response was generated after 300 milliseconds. As can be seen from the summary results there was a clear systematic conditional bias!


Schematic of how the studies timing was validated using a Black Box ToolKit.


Measure Expected RT (mS) Observed Mean RT (mS)
Star duration
Tone duration
Tone synchrony to star:
Reaction Time (no tone)
Reaction Time (tone present)

Summary results from Plant, Hammond & Turner (2004),
"Self-validating presentation and response timing in cognitive paradigms: how and why?"

Self-validating presentation and response timing in cognitive paradigms: how and why?, Richard R Plant, Nick Hammond, Garry Turner. Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc. 06/2004; 36(2):291-303. Archive of Replication Attempts in Experimental Psychology