|
Below are some questions that
might arise, with answers
Q: What do I need to be able to run the Palamedes routines? A: If you
are using Matlab®: The basic Matlab program. No additional Matlab toolboxes are required. Palamedes is backwards compatible
going back to older versions of Matlab. Every once in a long while we check compatibility with older versions of Matlab and
fix any incompatibilities we find (if any). If you are using GNU Octave: The basic Octave program. No additional packages
are required. Note that some figures produced by the programs in the PalamedesDemos folder are not rendered as intended when
you run them under Octave. Note also that some routines run quite a bit slower under Octave compared to Matlab. If you want
to use the Bayesian fitting routines in the PAL_PFHB branch of Palamedes, you will also need to install either JAGS (http://mcmc-jags.sourceforge.net/)
or cmdStan ('command Stan'; https://mc-stan.org/users/interfaces/cmdstan.html). Please
cite JAGS or Stan when you use them with Palamedes to analyse data. - Q: What is GNU Octave?
How do I get it?
A: Octave is an open-source program similar to Matlab.It
can be freely downloaded here: www.gnu.org/software/octave. A: Download and
unzip the Palamedes folder and the PalamedesDemos folder to your computer (likely your computer contains the tools to unzip
the files already). Add the Palamedes folder to the Matlab or Octave search path (on Matlab menu, go to ‘file’,
then ‘set path’. In Octave type: addpath followed by the path, e.g., addpath c:\nick\Palamedes (Windows) or addpath
/home/nick/Palamedes (linux)). This will allow you to call the Palamedes routines from any directory. In case you wish to
be able to run the demos from any directory, add the PalamedesDemos folder also.
Q: How do I get started? How do I figure out how the
routines work and what they do? A: A good place to start and find out what you
can do with Palamedes is our overview page. It describes the various types of procedures and analyses Palamedes is capable of and lists for each the relevant Demo
programs (in the PalamedesDemos folder). All of the scripts in the Demos folder will take you through a complete analysis
and are heavily commented. Do not be put off by the amount of code in these demo scripts: The bulk of the code serves to create
presentable graphs. All user-end Palamedes routines contain help comments (type ‘help’ followed by the name of
the routine). These help comments in all user-end routines contain example code that should execute when typed into the command
window. Q: I have existing code that calls on Palamedes routines. If
I update to a newer version of Palamedes will this code still work?
A: Yes. We take great care to make newer versions of Palamedes fully backwards compatible with older ones. We have
felt it necessary to violate this guiding principle knowingly only once before, resulting in a minor, easily fixed incompatibility. If you know of any other incompatibilites, please let us know!
Q: I have used Palamedes routines in my research. Should I cite you in my report? A: Yes. Prins, N & Kingdom, F. A. A. (2018) Applying the Model-Comparison Approach to Test Specific Research
Hypotheses in Psychophysical Research Using the Palamedes Toolbox. Frontiers in Psychology, 9:1250. doi: 10.3389/fpsyg.2018.01250. If you used PAL_PFHB_fitModel you should also cite JAGS or Stan (check with them to see how to properly cite them). Q: I don’t understand the theory behind the routines
(e.g., what is Maximum Likelihood?). Do you have time to explain it to me? A:
No. Q: Will you guarantee that the
Palamedes routine will work free of error? A: No. Q: I have made changes to the code and now it doesn’t
work anymore, can you tell me what I did wrong? A: No. Q: A routine crashed. Reading the elaborate help you
provided seems like an awful lot of work. Do you have time to figure out and tell me what I did wrong? A: No. Q: I wrote a few routines that
do this or that. Would you incorporate them into your toolbox? A: No. Q: I need to perform an analysis on some data that your
toolbox can not do. Will you add routines that can solve my problem? A: Possibly.
Tell us what it is that you’d like to see added and we’ll take it into consideration. Q: Will you release updates to Palamedes from time to
time? A: Yes. Check this website for updates. The current version number will
be displayed on the website. Type PAL_version or PAL_info to find out which version you have. - Q:
Your Weibull isn't a Weibull (or is it)?
A: Yes, it is. You are probably looking
for the Gumbel (or log-Weibull): PAL_Gumbel. You might also want to consider the Quick or logQuick functions. Click here for more information. - Q: What is the function value (e.g., probability correct)
of the PFs (Logistic, Weibull, etc.) at threshold?
A: That depends on the form
of the PF (Logistic, Weibull, etc.), the 'guess-rate' and the 'lapse rate'. To find the function value at threshold simply
evaluate the function at threshold. For example, the function value at threshold for the Logistic using guess rate = 0.5 and
lapse rate = 0.02 can be found by typing e.g.,: y=PAL_Logistic([1 2 .5 .02],1); - Q: Occassionally, PAL_PFLR_ModelComparison and/or PAL_PFML_GoodnessOfFitMultiple return negative values for the Transformed
Likelihood Ratio (or Deviance). Palamedes does not indicate that any of the fits failed. That can't be right, can it?
A: Either one of three things happened. (1) If the negative TLRs (or
Devs) are near zero, it is likely that the corresponding lesser and fuller model have actual Log Likelihoods (LL) that are
nearly identical. Palamedes only approximates LL values. If the approximation of LL for the fuller model happens to be a bit
lower than the actual LL value and the approximation of the LL for the lesser model happens to be a bit higher than the actual
value the TLR value might be (slightly) negative. The solution is either to accept the negative TLR (or Dev) value to be the
result of imprecision or to decrease the acceptable degree of imprecision by using the 'searchOptions' option and decrease
the value of the '.TolFun' field in the options structure (this will generally require you to increase the maximum allowed
iterations and function evaluations as well. It will also slow down the routine). See PAL_PFLR_Demo for an example of usage. (2) If the negative values are not near zero, the fitting procedure may have ended up in a local maximum in the likelihood
function. (These mis-fits can usually be spotted easily by inspection of the parameter estimates which will generally be nowhere
near any realistic value). A solution that might work here is to use different guesses for the initial search values in the
call to the function. Also use a sensible range of values when using the 'rangeTries' option. If this does not solve the problem,
either reduce the number of free parameters or gather more data. (3) Your 'lesser'
model is not nested under your 'fuller' model. If the lesser model is nested under the fuller model, the fuller model can
do everything the lesser model can and more. That is, the fuller model can always at least match the fit and the Log Likelihood
of the lesser model and the TLR can not be negative. Unless the lesser model is nested under the fuller model, the likelihood
ratio test is inappropriate. In case you want to compare two models that are not nested, use one of the information criteria
(e.g., Akaike's Information Criterion (AIC) or Bayesian Information Criterion (BIC)). Q: Is it possible to restrict the range of possible
value for the parameter estimates when fitting multiple PFs? A: Yes. With the
introduction of custom reparametrization of parameters in Palamedes Version 1.1.0 this is possible. For example, to restrict
a slope estimate to be positive, use the reparametrization:
slope
= exp(b) and have Palamedes find the best-fitting estimate of b. See the example code in
PAL_PFLR_CustomDefine.m for an example. The atan function can be used to restrict parameter estimates to a range bound by
a lower and upper value.
- Q:Routines crash because I don't have a function called
'iscolumn', what do I do?
A: You are running an old version of Matlab (older
than R2010b) AND an old version of Palamedes (older than 1.6.3). Upgrade Palamedes and/or Matlab. - Q: When I fit a psychometric function, should I allow the lapse rate to vary/estimate the lapse rate?
A: Maybe. It is a complicated issue. Check this page out for a bit more information. - Q: What is the 'Psi-marginal'
method? Should I use it?
A: 'Psi-marginal' is a term we came up with to denote
a modification of Kontsevich and Tyler's (1999) Psi method we implemented in Palamedes. Any combination of the four
parameters of a PF can be included in the procedure. For each of the parameters, you can specify whether it should be treated
as a parameter of primary interest whose estimation should be optimized, as a nuisance parameter whose estimation should be
subservient to the estimation of the parameters of primary interest, or as a fixed parameter. For example, in order to
use the original Psi-method specify threshold and slope to be of primary interest and fix the guess and lapse rates (this
is the default in PAL_AMPM_setupPM). QUEST is also (more or less) a particular set up of Psi-marginal (threshold is of primary
interest, all other parameters are fixed). Whatever you do, do not use the original Psi-method to collect your data, then
refit while allowing the lapse rate to vary (Prins, 2012; www.journalofvision.org/content/12/6/25). For more information on Psi-marginal see Prins (2013; www.journalofvision.org/content/13/7/3).
- Q: After downloading a new version of Palamedes, a script that I wrote that
used the PAL_AMPM (Psi method) routines crashes. What do I do?
A: As of Palamedes
version 1.5.0 the posterior (i.e., PM.pdf; created using PAL_AMPM_setupPM and updated trial-by-trial by PAL_AMPM_updatePM)
varies alpha values across the first dimension (rows) of a 2-D PM.pdf and beta values across the second dimension (columns).
This was vice versa in earlier versions. This change will affect behavior of scripts only in case (1) the script supplied
a prior to PAL_AMPM_setupPM instead of using the default uniform prior or (2) if script performed some operation on PM.pdf
(e.g., plotted PM.pdf using, for example, 'surf' or 'image'). To make scripts written for previous versions compatible with
Palamedes 1.5.0, simply transpose a user-supplied prior (i.e., prior = prior';) and transpose PM.pdf in the plot command [e.g.,
surf(PM.pdf') instead of surf(PM.pdf)]. Click here for more information.
- Q: When I run my analysis, Matlab issues 'log of zero'
and/or 'divide by zero' warnings from within palamedes routines. What do I do?
A:
You have options. (1) Ignore the warnings (they are inconsequential, the Palamedes routines deal with the dividing by zero
or taking the log of zero). (2) Upgrade Matlab (newer versions of Matlab do not issue the warning). (3) Disable the warnings.
Type help warning to get information on how to disable warnings. - Q: My fit of a
psychometric function doesn't give me the results I was expecting/hoping for (way-out-there parameter estimates, exitflag
is set to zero or a negative value, bootstrap simulations fail to converge, standard errors are reported as NaN, etc). What
am I doing wrong?
A: Not sure. The problem might even be at our end (but it probably
isn't). Our money is on you having too much model and too little data. This page gives some insight and some tips: www.palamedestoolbox.org/understandingfitting.html
|