Below are some questions that
might arise, with answers
What do I need to be able to run the Palamedes routines?
A: If you are using Matlab®: The basic
Matlab program. No additional Matlab toolboxes are required. Palamedes is backwards compatible going back at least as far
as Matlab release 14 (2004) [This statement was false for a period of time. Backward compatibility restored in Palamedes version
1.6.3, released February, 2014. Compatibility was checked against Matlab release 14 version 126.96.36.19904 without any of Matlab's
additional toolboxes]. If you are using GNU Octave: The basic Octave program. No additional packages are required. Palamedes
should work under Octave version 3.6.1 and higher. Compatibility with older versions is unknown. Compatibility was checked
against Octave version 3.6.1 under windows and version 3.8.0 under Ubuntu linux. Note that some figures produced by the programs
in the PalamedesDemos folder are not rendered as intended when you run them under Octave. Note also that some routines run
quite a bit slower under Octave compared to Matlab.
- Q: What is GNU Octave? How do I get it?
Octave is an open-source program similar to Matlab.It can be freely downloaded here: www.gnu.org/software/octave.
A: Download and
unzip the Palamedes folder and the PalamedesDemos folder to your computer (likely your computer contains the tools to unzip
the files already). Add the Palamedes folder to the Matlab or Octave search path (on Matlab menu, go to ‘file’,
then ‘set path’. In Octave type: addpath followed by the path, e.g., addpath c:\nick\Palamedes (Windows) or addpath
/home/nick/Palamedes (linux)). This will allow you to call the Palamedes routines from any directory. In case you wish to
be able to run the demos from any directory, add the PalamedesDemos folder also.
Q: How do I get started? How do I figure out how the routines work and what they
A: A good place to start and find out what you can do with Palamedes is our overview page. It describes the various types of procedures and analyses Palamedes is capable of and lists for each the relevant Demo
programs (in the PalamedesDemos folder). All of the scripts in the Demos folder will take you through a complete analysis
and are heavily commented. Do not be put off by the amount of code in these demo scripts: The bulk of the code serves to create
presentable graphs. All user-end Palamedes routines contain help comments (type ‘help’ followed by the name of
the routine). These help comments in all user-end routines contain example code that should execute when typed into the command
Q: I have existing code that calls on Palamedes routines. If I update to a newer
version of Palamedes will this code still work?
A: Yes. We take great care to
make newer versions of Palamedes fully backwards compatible with older ones. We have felt it necessary to violate this guiding
principle knowingly only once before, resulting in a minor, easily fixed incompatibility. If you know of any other incompatibilites, please let us know!
Q: I have used Palamedes routines in my research. Should I cite you in my report?
Yes. Prins, N. & Kingdom, F.A.A. Palamedes: Matlab routines for analyzing psychophysical data. www.palamedestoolbox.org
Q: I don’t understand the theory behind the routines (e.g., what
is Maximum Likelihood?). Do you have time to explain it to me?
Q: Will you guarantee that the Palamedes routine will work free of error?
Q: I have made changes to the code and now it
doesn’t work anymore, can you tell me what I did wrong?
Q: A routine crashed. Reading the elaborate help you provided seems like an awful
lot of work. Do you have time to figure out and tell me what I did wrong?
Q: I wrote a few routines that do this or that. Would you incorporate
them into your toolbox?
Q: I need to perform an analysis on some data that your toolbox can not do. Will you add routines
that can solve my problem?
A: Possibly. Tell us what it is that you’d like to see added and
we’ll take it into consideration.
Will you release updates to Palamedes from time to time?
A: Yes. Check this website for updates. The
current version number will be displayed on the website. Type PAL_version or PAL_info to find out which version you have.
Your Weibull isn't a Weibull (or is it)?
A: Yes, it is. You are probably looking for the Gumbel (or log-Weibull):
PAL_Gumbel. You might also want to consider the Quick or logQuick functions. Click here for more information.
- Q: What is the function value (e.g., probability correct) of the PFs (Logistic,
Weibull, etc.) at threshold?
A: That depends on the form of the PF (Logistic, Weibull, etc.), the 'guess-rate'
and the 'lapse rate'. To find the function value at threshold simply evaluate the function at threshold. For example, the
function value at threshold for the Logistic using guess rate = 0.5 and lapse rate = 0.02 can be found by typing e.g.,:
2 .5 .02],1);
- Q: Occassionally, PAL_PFLR_ModelComparison and/or PAL_PFML_GoodnessOfFitMultiple return
negative values for the Transformed Likelihood Ratio (or Deviance). Palamedes does not indicate that any of the fits failed.
That can't be right, can it?
A: Either one of three things happened.
the negative TLRs (or Devs) are near zero, it is likely that the corresponding lesser and fuller model have actual Log Likelihoods
(LL) that are nearly identical. Palamedes only approximates LL values. If the approximation of LL for the fuller model happens
to be a bit lower than the actual LL value and the approximation of the LL for the lesser model happens to be a bit higher
than the actual value the TLR value might be (slightly) negative. The solution is either to accept the negative TLR (or Dev)
value to be the result of imprecision or to decrease the acceptable degree of imprecision by using the 'searchOptions' option
and decrease the value of the '.TolFun' field in the options structure (this will generally require you to increase the maximum
allowed iterations and function evaluations as well. It will also slow down the routine). See PAL_PFLR_Demo for an example
(2) If the negative values are not near zero, the fitting procedure may have ended up in
a local maximum in the likelihood function. (These mis-fits can usually be spotted easily by inspection of the parameter estimates
which will generally be nowhere near any realistic value). A solution that might work here is to use different guesses for
the initial search values in the call to the function. Also use a sensible range of values when using the 'rangeTries' option.
If this does not solve the problem, either reduce the number of free parameters or gather more data.
Your 'lesser' model is not nested under your 'fuller' model. If the lesser model is nested under the fuller model, the fuller
model can do everything the lesser model can and more. That is, the fuller model can always at least match the fit and the
Log Likelihood of the lesser model and the TLR can not be negative. Unless the lesser model is nested under the fuller model,
the likelihood ratio test is inappropriate. In case you want to compare two models that are not nested, use one of the information
criteria (e.g., Akaike's Information Criterion (AIC) or Bayesian Information Criterion (BIC)).
Q: Is it possible to restrict the range of possible value for the parameter
estimates when fitting multiple PFs?
A: Yes. With the introduction of custom reparametrization of
parameters in Palamedes Version 1.1.0 this is possible. For example, to restrict a slope estimate to be positive, use the
slope = exp(b)and have Palamedes find the best-fitting
estimate of b. See the example code in PAL_PFLR_CustomDefine.m for an example. The atan function can be used to restrict parameter
estimates to a range bound by a lower and upper value.
- Q:Routines crash because I don't have a function
called 'iscolumn', what do I do?
A: You are running an old version of Matlab (older than R2010b)
AND an old version of Palamedes (older than 1.6.3). Upgrade Palamedes and/or Matlab.
- Q: When I fit a psychometric
function, should I allow the lapse rate to vary/estimate the lapse rate?
A: Maybe. But exercise
caution (especially when you've used an adaptive method to collect your data) and consider various options. Bad things can
happen when you don't. If you decide to fix the lapse rate, don't fix it at 0 (but rather at e.g., 0.02). See Prins (2012; www.journalofvision.org/content/12/6/25) for more information.
- Q: What is the 'Psi-marginal' method? Should I use it?
'Psi-marginal' is a term we came up with to denote a modification of Kontsevich and Tyler's (1999) Psi method we implemented
in Palamedes. Any combination of the four parameters of a PF can be included in the procedure. For each of the parameters,
you can specify whether it should be treated as a parameter of primary interest whose estimation should be optimized, as a
nuisance parameter whose estimation should be subservient to the estimation of the parameters of primary interest, or as a
fixed parameter. For example, in order to use the original Psi-method specify threshold and slope to be of primary
interest and fix the guess and lapse rates (this is the default in PAL_AMPM_setupPM). QUEST is also (more or less) a particular
set up of Psi-marginal (threshold is of primary interest, all other parameters are fixed). Whatever you do, do not use the
original Psi-method to collect your data, then refit while allowing the lapse rate to vary (Prins, 2012; www.journalofvision.org/content/12/6/25). For more information on Psi-marginal see Prins (2013; www.journalofvision.org/content/13/7/3).
- Q: After downloading a new version of Palamedes, a script that I wrote that used the PAL_AMPM (Psi
method) routines crashes. What do I do?
A: As of Palamedes version 1.5.0 the posterior (i.e., PM.pdf;
created using PAL_AMPM_setupPM and updated trial-by-trial by PAL_AMPM_updatePM) varies alpha values across the first dimension
(rows) of a 2-D PM.pdf and beta values across the second dimension (columns). This was vice versa in earlier versions. This
change will affect behavior of scripts only in case (1) the script supplied a prior to PAL_AMPM_setupPM instead of using the
default uniform prior or (2) if script performed some operation on PM.pdf (e.g., plotted PM.pdf using, for example, 'surf'
or 'image'). To make scripts written for previous versions compatible with Palamedes 1.5.0, simply transpose a user-supplied
prior (i.e., prior = prior';) and transpose PM.pdf in the plot command [e.g., surf(PM.pdf') instead of surf(PM.pdf)]. Click
here for more information.
- Q: When I run my analysis, Matlab issues 'log of zero' and/or 'divide by
zero' warnings from within palamedes routines. What do I do?
A: You have options. (1) Ignore the warnings
(they are inconsequential, the Palamedes routines deal with the dividing by zero or taking the log of zero). (2) Upgrade Matlab
(newer versions of Matlab do not issue the warning). (3) Disable the warnings. Type help warning to get information on how
to disable warnings.
- Q: My fit of a psychometric function doesn't give me the results I was expecting/hoping
for (way-out-there parameter estimates, exitflag is set to zero, bootstrap simulations fail to converge etc). What am I doing
A: Not sure. The problem might even be at our end (but it probably isn't). This page gives some
insight and some tips: www.palamedestoolbox.org/understandingfitting.html