BackgroundMillisecond timing error means that your experiment is not working as you intended and that your results might be invalid.
- Are you always carrying out the experiments you assume you are?
- Are you aware of millisecond timing error in your own experiments?
- Are you confident you can replicate experiments using different hardware and software in another lab?
Are you putting your reputation at risk?
We can helpIf you wish to discuss how any of our products could help you improve your research feel free to email us. Alternatively contact us by mail, phone or fax.
Why you should be worried about millisecond timing accuracy
There are a myriad of factors that can have a negative impact on millisecond accurate presentation and response timing in computer-based psychology, neuroscience and vision experiments. Such errors can mean your experiment may not be doing what you think!
Remember modern computer hardware may be faster but millisecond timing accuracy gets harder to achieve year-on-year. A common misconception is that "millisecond precision" equals "millisecond accuracy". Precision simply means that timings are reported in units of a millisecond, not that they are accurate!
For example, all TFT monitors and projectors suffer from input lag. If you present an image it will take longer to appear on screen than you might think. On some models this can be upwards of 100 milliseconds. Whatever experiment generator you use, it only knows when it requested a stimulus image be shown and not the time when it physically appeared. Because of soundcard startup latency audio timing is even more variable. But don't just take our word for it read PST's own findings when using E-Prime.
The table below shows a range of factors that can negatively impact timing in your experiment. This is before one considers the effects of poor scripting practices or human error when constructing experiments!
|Source of error||Modality||Tell me why|
|Input lag on TFT monitors||TFT input lag is the time it takes an image to appear on screen versus the time when the signal was sent by the computer into the monitor cable.|
|TFT panel response time||TFT response time is not the same as input lag but instead is a measure of how long it takes the actual panel to switch between two colours (usually from grey to grey as that gives manufacturers the fastest time).|
|Image color and intensity||All TFT monitors fade or dim over time which means they won't be as bright as they were as time goes by. Typically they fade around 40% each year.|
|Sound card start-up latency||Soundcard startup latency is the time it takes from telling the computer to play a sound to when the sound actually comes out of the speakers or headphones.|
|Keyboard and mouse||Keyboards and mice are not generally designed to be fast input devices and aren't designed to act as response devices in your experiment. Reported timing varies due to polling loops and the electronics in the keyboard or mouse.|
|Voice keys for vocal responses||Voice keys operate on the basis of "crossing thresholds". Often the researcher is unaware of what this threshold actually is and whether the setting is the same as one used previously.|
|Synchronization with other equipment (EEG, MRI, MEG, biofeedback)||Synchronization between other equipment and the PC or Mac you are running your experiment on is a typical source of timing error. It can be difficult to align an event marker with an EEG machine or sync signal from an MRI machine accurately.|
|Operating system issues||Modern operating systems are more complex than they've ever been (hence why your new computer doesn't run your word processor any faster than your old one!). Layers and layers of software APIs make it more difficult for software to interact directly with the actual hardware and obtain accurate timing.|
|Software settings, software bugs and device drivers||Software settings, software bugs and device drivers: Complexity comes at a price and it's questionable how many researchers actually use the same settings for each study. Modern software will have bugs in it which under certain circumstances will lead to unintentional timing errors. Hardware drivers should be simple but often manufacturers add as many bells and whistles as they can fit in. Settings and bugs in hardware drivers can also have a dramatic impact.|
|Conditional biases and unaccounted for interactions||Unbeknown to you systematic conditional biases could account for your whole experimental effect. The classic file drawer problem in psychological research can be an unintended consequence of unknown biases.|
|The laws of nature and the laws of physics||The laws of nature/physics will work against you. A loudspeaker is a moving coil that has to move back and forth to generate sound waves and to do this it must have time to mechanically start vibrating. The speed of sound in air is roughly 1 millisecond per ft (approx 30cm). Thus if you position your speakers around the room say 10ft from your subject then there will be a 10 millisecond sound lag. Pneumatic headphones in MRI studies - if you have 30ft or 10m or rubber tubing then the sound will take 30 milliseconds to reach your subject.|
When computers first started to be used for psychological research many investigators were more concerned about millisecond timing accuracy and went to extraordinary lengths to ensure that their equipment performed as expected. Because they were working with much slower systems they needed to understand how the hardware worked and generally write the software that conducted the experiments themselves. Although slower, simpler operating systems meant that you could directly access the hardware and you could be fairly certain of what was happening. Operating systems were generally single-tasking, i.e. did one thing at once, and displays were based on CRTs. Merely playing a sound could be a challenge!
Somewhat counter intuitively systems from the early 1990's are typically more accurate in absolute terms than the majority of today's high end multi-core computers and multitasking operating systems with regard to getting millisecond accurate results in studies of human performance.
Because it has become easier to design and run experiments today than it ever was very few current researchers have had to write their own software or become interested in how the hardware actually works. In much the same way that few car drivers could routinely maintain their own car. Computers are now commodity items that can be purchased in the supermarket alongside your grocery shop.
There are common sources of error which mean you are unlikely to be presenting what you think you are presenting and conversely measuring what you think you are measuring. All computers suffer from all of these sources of timing errors. There are no exceptions! To be sure of your experimental timing you should be using a Black Box ToolKit v3 to discover these and minimise where possible. Timing error will vary according to the make and model of TFT, sound card, keyboard, mouse or any other hardware, software or operating system you use. Remember that errors are additive in the respect that errors can combine together.