[ADMB Users] NLMM Model Selection

[Chris Gast]

Thanks, Mark.  I can look into that option, too.  I was hoping to be able to
output it from the initial optimization, but that doesn't appear possible.

As for the question of why: recent references seem to indicate that the
conditional AIC might be more appropriate for model selection when inference
is to be made on the same "groups" from the original data, and marginal AIC
when extending inference to other "groups" of the same type.  I think Dr.
Bolker said it more cleanly, so I'll paste a portion of his discussion here:

There is a fundamental distinction between the 'marginal AIC'
(for population-level predictions, i.e. where you want to predict
future values for a different set of random effects than those measured)
and the 'conditional AIC' (for group-level predictions where you want
to predict future values for the same random effects measured).

Of course AIC has some difficulty here, related to the boundary condition
problems also encountered with LRTs on constrained parameter spaces (sigma
necessarily > 0). I'm still investigating the model selection issues myself,
so can't offer much more clarity.


Chris






-----------------------------
Chris Gast
cmgast at gmail.com


On Tue, Feb 22, 2011 at 3:35 PM, Mark Maunder <mmaunder at iattc.org> wrote:

>  Could this be calculated by modifying your code from
>
>
>
> Random_effects_vector
>
>
>
> To
>
>
>
> Init_vector
>
>
>
> And then running the model with the command line option –maxfn 0 and using
> the par file from the random effects model and the pin file
>
>
>
> This would use the full joint likelihood without integrating over the
> random effects and evaluating it at the MLE of the fisxed effects and the
> empirical bayes estimates of the random effect realizations.
>
>
>
> Not sure why you would do this though.
>
>
>
> Mark
>
>
>
>
>
>
>
> Mark Maunder
>
>
>
>
> Head of the Stock Assessment
> Program
>
> Inter-American  Tropical Tuna
> Commission
>
>  President
>
> ADMB Foundation
>
>
>
> 8604 La Jolla Shores Drive
> La Jolla, CA, 92037-1508, USA
>
> Tel: (858) 546-7027
> Fax: (858) 546-7133
> mmaunder at iattc.org
>
> http://www.fisheriesstockassessment.com/TikiWiki/tiki-index.php?page=Mark+Maunder
>
>
>
> Visit the AD Model Builder project at
>  http://admb-project.org/
>
>
>
> See the following website for information on fisheries stock assessment
>
> http://www.fisheriesstockassessment.com/
>
>
>
>
>
> *From:* Chris Gast [mailto:cmgast at gmail.com]
> *Sent:* Tuesday, February 22, 2011 3:25 PM
> *To:* Mark Maunder
>
> *Cc:* users at admb-project.org
> *Subject:* Re: [ADMB Users] NLMM Model Selection
>
>
>
> Conditional on the values of the main effects. I'm working from a reference
> provided by Dr. Bolker:
>
> Greven, Sonja, and Thomas Kneib. 2010. On the Behaviour of Marginal
> and Conditional
> Akaike Information Criteria in Linear Mixed
> Models. Biometrika 97, no. 4: 773-789.
> http://www.bepress.com/jhubiostat/paper202/.
>
>
>
> where the marginal likelihood (for a LMM, not a GLMM or other nonlinear
> mixed model) is as discussed above, y~N(XB,V) (REs have been integrated
> over) and the conditional likelihood (at its optimum) is evaluated at the
> MLEs and EB estimates of REs, y | b ~ B*XB + Zb, I*sigma^2) where the b's
> are the REs.
>
>
>
> So ADMB works with the marginal likelihood, and the optimum value is
> provided in the .par file.  However, it would be nice (at least at this, the
> exploratory model-selection phase of my work) to also have the conditional
> likelihood value, which would be obtained (I believe) by plugging the RE
> estimates and MLEs back into the likelihood function and evaluating. This
> isn't done automatically by ADMB, but (I think) could be done by changing
> the .par file to a .pin file, and re-evaluating the model and outputting the
> first objective function value.  I haven't tested this idea yet, though.
>  I'll probably play with it a bit with a GLMM for which I can compare AIC
> values with other software to try and understand what precisely each is
> doing.
>
>
>
>
>
> Chris
>
>
>
>
>
>
> -----------------------------
> Chris Gast
> cmgast at gmail.com
>
>  On Tue, Feb 22, 2011 at 9:30 AM, Mark Maunder <mmaunder at iattc.org> wrote:
>
> Conditional on what?
>
>
>
> Mark Maunder
>
> Head of the Stock Assessment Program
> Inter-American  Tropical Tuna Commission
>
> President
> ADMB Foundation
>
> 8604 La Jolla Shores Drive
> La Jolla, CA, 92037-1508, USA
>
> Tel: (858) 546-7027
> Fax: (858) 546-7133
> mmaunder at iattc.org
>
> http://www.fisheriesstockassessment.com/TikiWiki/tiki-index.php?page=Mark+Maunder
>
> Visit the AD Model Builder project at
>  http://admb-project.org/
>
> See the following website for information on fisheries stock assessment
> http://www.fisheriesstockassessment.com/
>
>
>
> -----Original Message-----
> From: users-bounces at admb-project.org [mailto:
> users-bounces at admb-project.org] On Behalf Of H. Skaug
> Sent: Tuesday, February 22, 2011 8:19 AM
> To: Chris Gast
> Cc: users at admb-project.org
> Subject: Re: [ADMB Users] NLMM Model Selection
>
> On Tue, Feb 22, 2011 at 5:10 PM, Chris Gast <cmgast at gmail.com> wrote:
> > I'm sorry, I must be getting confused.  My message from yesterday
> (question
> > 1) asked if the objective function value in the .par file was the
> > post-integration marginal likelihood, to which you responded in the
> > negative, but now it seems as if you're saying what I had originally
> > postulated is true. Maybe it's just a terminology difference.  It is my
> > understanding that the .par file contains the optimum loglikelihood value
> > obtained, where the loglikelihood value is marginalized over the random
> > effects.  Perhaps there is some miscommunication?  I think we're talking
> > about the same thing.
>
> Yes, we are. I misunderstood your original question.
>
> > Since the marginal likelihood is available in the .par file, is there a
> way
> > to output the conditional likelihood following optimization?
>
> No, not any direct way that I am aware of. It is possible that this
> is what you get before the random effects kick in (assuming those
> active in phase 2 or later).
>
> hans
>
> _______________________________________________
> Users mailing list
> Users at admb-project.org
> http://lists.admb-project.org/mailman/listinfo/users
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.admb-project.org/pipermail/users/attachments/20110222/c9e14a17/attachment.html>