[ADMB Users] importance sampling

[dave fournier]

> On 13-03-15 11:04 AM, dave fournier wrote:
> >/  
> />/  A while back  it became clear that the Laplace approximation is not
> />/  good enough for some simple RE models. With no crossed effects one
> />/  can use Gauss-Hermite quadrature (This can be done for more than one
> />/  RE per group in ADMB although I have seen reports on the notoriously
> />/  inaccurate R list claiming that ADMB can only do GH for one RE per
> />/  block).
> /
>    Dave: those reports were from me, following the published documentation.
>
>    When I go to
>
>    http://admb-project.org/documentation/manuals/admb-user-manuals
>
> and open
>
> http://admb-project.googlecode.com/files/admbre-10.0-rev1.pdf
>
> and search for "Gauss-Hermite quadrature", I find this:
>
> >/  In the situation where the model is separable of type "Block diagonal Hessian," with only a
> />/  single random effect in each block (see Section 4), Gauss-Hermite quadrature is available as
> />/  an option to the Laplace approximation and to the -is (importance sampling) option. It is
> />/  invoked with command line option -gh N, where N is the number of quadrature points.
> /
>    I also checked in the LaTeX code of the latest version from the SVN
> repository -- this text is current.
>
>    Perhaps the documentation can be corrected?  If you tell me what it
> should say, I can submit a documentation patch ...
>
>    Ben
>
>
>
> >/  
> />/  However for models with crossed effects in general GH would be
> />/  impractical.  The alternative is importance sampling.  We should
> />/  encourage users to routinely do importance sampling to test the
> />/  stability of the estimates.  We could make this simple with a command
> />/  like
> />/  
> />/  -is 25  121  50
> />/  
> />/  this would do 50 fits with 25 random points and starting with the
> />/  random number seed 121. The seed would get changed for each fit.  A
> />/  report would get generated showing the variability of the LL and
> />/  parameter estimates.
> />/  
> />/  One problem is that since lme? in R can not do this the idea will
> />/  probably get surpressed on the R list.  Why do I care?  Well users of
> />/  glmmadmb should be encouraged to take advantage of importance
> />/  sampling.  I worry that the kind of sloppy practice encouraged in the
> />/  R community  extends to glmmadmb which uses my software.
> /
>     You'll also notice that I *suggested* using ADMB with importance
> sampling on the R mailing list in the first place (in the same message
> that you're referring to obliquely above) ...

I extended the software to have multiple random effects per group
some time back. People seem to be aware of this as I have been sent
code to examine with exactly this setup.

As for the importance sampling,  we all became aware that for a simple RE
model the Laplace approximation was not good enough to get reliable
estimates from either lme? or ADMB.  At that time I discovered
that the GH quadrature in lme? was broken and you decided to fix it
quietly without having the users worry their little heads about it.
Well fine.  However GH can not be used for crossed effects due to
computational limitations.  One has every reason to expect therefore
that neither ADMB or lme? can fit these models reliably based on the
LA.  However ADMB can do importance sampling for these models,
and IS proved to be adequate for the simple model referred to above.

Now in many cases I have found by using IS that for negative binomial
models the LA appears to be good enough.  The kind of problem
for which the LA did not perform well was a binary response model.

So I conclude that lme? should not be relied on for binary response models
with crossed effects.  However I doubt that anyone will  deal with this 
issue on
the R lists.

In any event it should be simple with ADMB to use IS to investigate the
adequacy of the LA for any RE model. This should be a routine part of
any serious analysis.






>    Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.admb-project.org/pipermail/users/attachments/20130315/b9d7f954/attachment.html>