Is it acceptable to use high granularity for the stimulus range?

[Matt J Dunn]

When defining stimRange, is it acceptable to use a continuous variable? i.e. to allow high granularity in stimRange, rather than using a small number of discrete intensity steps.

I understand that historically psychophysicists tended to use a limited number of fixed step sizes across the testing range, and this means that when plotting the psychometric function you end up with several presentations at the same intensity. Of course, if one uses a ‘more continuous’ variable, then we typically end up with only one presentation at each intensity, meaning that for any given intensity, nCorrect is either 0% or 100%. However, I would imagine that this wouldn’t cause any mathematical problems with the psychometric fitting process? And, it seems to me that it’s more powerful to give the Psi procedure greater choice in terms of possible values to choose from stimRange?

I’ve been struggling to understand why psychophysicists continue to use discrete steps for stimRange, so any light you can shed on this would be greatly appreciated.

[Nick Prins]

The psi-method derives an estimate of the free parameters based on the mean of the posterior distribution. The validity of that does not depend at all on whether observations are all taken at unique intensities or whether some of them share intensities. Increasing the number of stimulus intensities will give the method more precision in choosing the most informative stimulus intensity but will also use more RAM and will slow computation down. Note also that a finer grid in the parameter space represented in the posterior (‘priorAlphaRange’, etc.) will result in a better estimate of the posterior and thus will also make the psi-method better at selecting the ideal stimulus intensity. It is not straightforward to determine whether increasing precision of parameter grid or precision of stimulus range will be the more beneficial strategy. In practice there will be a point where increasing the precision in stimulus intensity (or in the parameter grid) any further will not appreciably improve estimates of parameters.
The running-fit family of adaptive methods (e.g., Quest) in the PAL_AMRF routines also is not bothered by single observations for some or all stimulus intensities. As a matter of fact, the stimulus intensity used by these routines is on a continuous scale.
Do note that things might go wrong if the data are subsequently analyzed. Bayesian methods (PAL_PFHB_fitModel) aren’t bothered by spreading observations thin across many intensities. However, when using maximum-likelihood methods, deriving a standard error using a non-parametric bootstrap (PAL_PFML_BootstrapNonParametric or PAL_PFML_BootstrapNonParametricMultiple) will suffer when there are few observations per intensity level. To see this, note that in the extreme case of a single observation per intensity level, all bootstrap samples will exactly replicate the data and the resulting standard error estimates will equal 0. When using a (maximum-likelihood) Goodness-of-fit test (PAL_PFML_GoodnessOfFit or PAL_PFML_GoodnessOfFitMultiple) use of the asymptotic chi-square distribution (e.g., deriving a p-value by looking up Deviance value in chi-square table or using Matlab’s chi2cdf function) becomes more and more problematic the thinner your observations are spread across many stimulus intensities. Use bootstrap simulations instead. Parametric bootstraps and model comparisons between parametric models do not care about how thin your observations are spread.
Data visualisation is difficult when observations are spread thin. In the extreme case of unique intensities for all observations, all proportions correct (or ‘left’ responses or whatever) will be either at the ceiling (100%) or the floor (0%) of your graph. That’s not a pretty or easily interpretable sight. Things like visually comparing the fitted function against your data or comparing your data against posterior predictive distributions will be difficult if not impossible.