HomeOverviewDownload PalamedesSupporting documentationFrequently asked questionsSubmit comments and questionsSubmit a bug reportNews and updates (last update: October 8, 2020Why "Palamedes"?About usOptions besides PalamedesDemos figure galleryModel Comparison ExamplesConfused about Weibull?Failed Fits?Hierarchical Bayesian PF Fitting

This serves as a quick guide to four possible scenarios that relate to whether the likelihood function has a true global maximum. A detailed exposition on the issues discussed here may be founhere or in Prins (2019). Note that fitting fails described here aren't exactly 'fitting fails', rather they are data fails (scenarios -1, -2, and -3 occur when there is no maximum in the likelihood function that can be found). If your likelihood function contains a global maximum and you use an appropriate search grid (type 'help PAL_PFML_Fit'), Palamedes will find that global maximum. The 4 different scenarios:
 

Scenario 1: all is well, your likelihood function has a global maximum.

 

scenario1.jpeg

Scenario -1:
 
scenarion1.jpeg
 
Step function with one intermediate point fits better than any sigmoid (but can be approached asymptotically by a sigmoid). Solid blue line is best-fitting step function with guess and lapse rate fixed. Broken lines show best-fitting with guess and/or lapse rate freed.
 
Problem: Stimulus placement is too wide and/or too few observations (trials) were made.
 
Even though the fit technically failed (because data are deficient, mind you: It's you, not us) information is still there: 
Location ('threshold') parameter is likely near location of step.
Slope parameter is likely very high (relative to spacing of placement anyway). 
 
 
 
Solutions:
-Redo experiment with narrower stimulus placement (consider adaptive placement).
-Add observations.
-Fix slope at reasonable value.
-Go Bayesian: PAL_PFHB_modelFit.
 
 

Scenario -2: 

scenarion2.jpeg

Similar to scenario -1 but without intemediate point. Solid blue line is best-fitting step function with guess and lapse rate fixed. Broken lines show best-fitting with guess and/or lapse rate freed.

 
Problem: Stimulus placement is too wide and/or too few observations (trials) were made.
 
Even though the fit technically failed (because data are deficient, mind you: It's you, not us) information is still there: 
Location ('threshold') parameter is likely near location of step.
Slope parameter is likely very high (relative to spacing of placement anyway). 
 
 
 
Solutions:
-Redo experiment with narrower stimulus placement (consider adaptive placement).
-Add observations.
-Fix slope at reasonable value.
-Go Bayesian: PAL_PFHB_modelFit.
 
 
 

Scenario -3:
scenarion3.jpeg

 
Constant function valued at the mean proportion correct across all trials fits better than any increasing sigmoid (but can be approached asymptotically by a sigmoid). Solid blue line is best-fitting constant function. Any constant function valued between γ and 1 - λ (not just at the proportion correct corresponding to the PF's value at threshold!) can be approached asymptotically by a sigmoid.
 
Problem: Stimulus placement is too narrow and/or too few observations (trials) were made. Scenario -3 may result when using adaptive method that aims to place stimuli near location ('threshold') parameter (e.g., up/down, best Pest, Quest).
 
Very little information here as to what value of location parameter may be but slope parameter is likely low (relative to the spacing of stimulus placements anyway).
 
 
 
Solutions:
-Redo experiment with broader stimulus placement (consider adaptive placement strategy that targets slope as well as location, e.g., psi-method).
-Add observations.
-Fix slope at reasonable value.
-Go Bayesian: PAL_PFHB_modelFit.