Thanks, Mark. I can look into that option, too. I was hoping to be able to output it from the initial optimization, but that doesn't appear possible.<div><br></div><div>As for the question of why: recent references seem to indicate that the conditional AIC might be more appropriate for model selection when inference is to be made on the same "groups" from the original data, and marginal AIC when extending inference to other "groups" of the same type. I think Dr. Bolker said it more cleanly, so I'll paste a portion of his discussion here:</div>
<div><br></div><div><span class="Apple-style-span" style="border-collapse: collapse; font-family: arial, sans-serif; font-size: 13px; ">There is a fundamental distinction between the 'marginal AIC' (for population-level predictions, i.e. where you want to predict future values for a different set of random effects than those measured) and the 'conditional AIC' (for group-level predictions where you want to predict future values for the same random effects measured).</span></div>
<div><span class="Apple-style-span" style="border-collapse: collapse; font-family: arial, sans-serif; font-size: 13px; "><br></span></div><div><font class="Apple-style-span" face="arial, sans-serif"><span class="Apple-style-span" style="border-collapse: collapse;">Of course AIC has some difficulty here, related to the boundary condition problems also encountered with LRTs on constrained parameter spaces (sigma necessarily > 0). I'm still investigating the model selection issues myself, so can't offer much more clarity.</span></font></div>
<div><font class="Apple-style-span" face="arial, sans-serif"><span class="Apple-style-span" style="border-collapse: collapse;"><br></span></font></div><div><font class="Apple-style-span" face="arial, sans-serif"><span class="Apple-style-span" style="border-collapse: collapse;"><br>
</span></font></div><div><font class="Apple-style-span" face="arial, sans-serif"><span class="Apple-style-span" style="border-collapse: collapse;">Chris</span></font></div><div><font class="Apple-style-span" face="arial, sans-serif"><span class="Apple-style-span" style="border-collapse: collapse;"><br>
</span></font></div><div><font class="Apple-style-span" face="arial, sans-serif"><span class="Apple-style-span" style="border-collapse: collapse;"><br></span></font></div><div><font class="Apple-style-span" face="arial, sans-serif"><span class="Apple-style-span" style="border-collapse: collapse;"><br>
</span></font></div><div><font class="Apple-style-span" face="arial, sans-serif"><span class="Apple-style-span" style="border-collapse: collapse;"><br></span></font></div><div><font class="Apple-style-span" face="arial, sans-serif"><span class="Apple-style-span" style="border-collapse: collapse;"><br clear="all">
</span></font><br>-----------------------------<br>Chris Gast<br><a href="mailto:cmgast@gmail.com">cmgast@gmail.com</a><br>
<br><br><div class="gmail_quote">On Tue, Feb 22, 2011 at 3:35 PM, Mark Maunder <span dir="ltr"><<a href="mailto:mmaunder@iattc.org">mmaunder@iattc.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">Could this be calculated by modifying your code from</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">Random_effects_vector</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">To</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">Init_vector</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">And then running the model with the command line option –maxfn 0 and using the par file from the random effects model and the pin file</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">This would use the full joint likelihood without integrating over the random effects and evaluating it at the MLE of the fisxed effects and the empirical bayes
estimates of the random effect realizations.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">Not sure why you would do this though.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">Mark</span></p><div class="im">
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">Mark Maunder </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">Head of the Stock Assessment Program
</span><span style="color:#1F497D"></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">Inter-American Tropical Tuna Commission
<br>
<br>
</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">President</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">ADMB Foundation</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">8604 La Jolla Shores Drive<br>
La Jolla, CA, 92037-1508, USA<br>
<br>
Tel: (858) 546-7027<br>
Fax: (858) 546-7133<br>
<a href="mailto:mmaunder@iattc.org" target="_blank">mmaunder@iattc.org</a><br>
</span><span style="font-size:10.0pt;color:#1F497D"><a href="http://www.fisheriesstockassessment.com/TikiWiki/tiki-index.php?page=Mark+Maunder" target="_blank">http://www.fisheriesstockassessment.com/TikiWiki/tiki-index.php?page=Mark+Maunder</a></span><span style="font-size:11.0pt;color:#1F497D"></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">Visit the AD Model Builder project at<br>
<a href="http://admb-project.org/" target="_blank"><span style="color:#810081">http://admb-project.org/</span></a></span><span style="color:#1F497D"></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D">See the following website for information on fisheries stock assessment</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"><a href="http://www.fisheriesstockassessment.com/" target="_blank">http://www.fisheriesstockassessment.com/</a></span><span style="font-size:11.0pt;color:#1F497D"></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1F497D"> </span><span style="font-size:11.0pt;color:#1F497D"></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:#1F497D"> </span></p>
</div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt">From:</span></b><span style="font-size:10.0pt"> Chris Gast [mailto:<a href="mailto:cmgast@gmail.com" target="_blank">cmgast@gmail.com</a>]
<br>
<b>Sent:</b> Tuesday, February 22, 2011 3:25 PM<br>
<b>To:</b> Mark Maunder</span></p><div><div></div><div class="h5"><br>
<b>Cc:</b> <a href="mailto:users@admb-project.org" target="_blank">users@admb-project.org</a><br>
<b>Subject:</b> Re: [ADMB Users] NLMM Model Selection</div></div><p></p>
</div><div><div></div><div class="h5">
<p class="MsoNormal"> </p>
<p class="MsoNormal">Conditional on the values of the main effects. I'm working from a reference provided by Dr. Bolker: <br clear="all">
</p>
<div>
<p class="MsoNormal"><span><span style="font-size:10.0pt">Greven, Sonja, and Thomas Kneib. 2010. On the Behaviour of Marginal</span></span><span style="font-size:10.0pt"><br>
<span>and Conditional</span><br>
<span>Akaike Information Criteria in Linear Mixed</span><br>
<span>Models. Biometrika 97, no. 4: 773-789.</span><br>
<span><a href="http://www.bepress.com/jhubiostat/paper202/" target="_blank"><span style="color:#1C51A8">http://www.bepress.com/jhubiostat/paper202/</span></a>.</span></span></p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal"><span><span>where the marginal likelihood (for a LMM, not a GLMM or other nonlinear mixed model) is as discussed above, y~N(XB,V) (REs have been integrated over) and the
conditional likelihood (at its optimum) is evaluated at the MLEs and EB estimates of REs, y | b ~ B*XB + Zb, I*sigma^2) where the b's are the REs.</span></span></p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal"><span><span>So ADMB works with the marginal likelihood, and the optimum value is provided in the .par file. However, it would be nice (at least at this, the exploratory
model-selection phase of my work) to also have the conditional likelihood value, which would be obtained (I believe) by plugging the RE estimates and MLEs back into the likelihood function and evaluating. This isn't done automatically by ADMB, but (I think)
could be done by changing the .par file to a .pin file, and re-evaluating the model and outputting the first objective function value. I haven't tested this idea yet, though. I'll probably play with it a bit with a GLMM for which I can compare AIC values
with other software to try and understand what precisely each is doing.</span></span></p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal"><span><span>Chris</span></span></p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span><br>
</span>-----------------------------<br>
Chris Gast<br>
<a href="mailto:cmgast@gmail.com" target="_blank">cmgast@gmail.com</a><br>
<br>
</p>
<div>
<p class="MsoNormal">On Tue, Feb 22, 2011 at 9:30 AM, Mark Maunder <<a href="mailto:mmaunder@iattc.org" target="_blank">mmaunder@iattc.org</a>> wrote:</p>
<p class="MsoNormal">Conditional on what?<br>
<br>
<br>
<br>
Mark Maunder<br>
<br>
Head of the Stock Assessment Program<br>
Inter-American Tropical Tuna Commission<br>
<br>
President<br>
ADMB Foundation<br>
<br>
8604 La Jolla Shores Drive<br>
La Jolla, CA, 92037-1508, USA<br>
<br>
Tel: (858) 546-7027<br>
Fax: (858) 546-7133<br>
<a href="mailto:mmaunder@iattc.org" target="_blank">mmaunder@iattc.org</a><br>
<a href="http://www.fisheriesstockassessment.com/TikiWiki/tiki-index.php?page=Mark+Maunder" target="_blank">http://www.fisheriesstockassessment.com/TikiWiki/tiki-index.php?page=Mark+Maunder</a><br>
<br>
Visit the AD Model Builder project at<br>
<a href="http://admb-project.org/" target="_blank">http://admb-project.org/</a><br>
<br>
See the following website for information on fisheries stock assessment<br>
<a href="http://www.fisheriesstockassessment.com/" target="_blank">http://www.fisheriesstockassessment.com/</a></p>
<div>
<div>
<p class="MsoNormal"> <br>
<br>
-----Original Message-----<br>
From: <a href="mailto:users-bounces@admb-project.org" target="_blank">users-bounces@admb-project.org</a> [mailto:<a href="mailto:users-bounces@admb-project.org" target="_blank">users-bounces@admb-project.org</a>] On Behalf Of H. Skaug<br>
Sent: Tuesday, February 22, 2011 8:19 AM<br>
To: Chris Gast<br>
Cc: <a href="mailto:users@admb-project.org" target="_blank">users@admb-project.org</a><br>
Subject: Re: [ADMB Users] NLMM Model Selection<br>
<br>
On Tue, Feb 22, 2011 at 5:10 PM, Chris Gast <<a href="mailto:cmgast@gmail.com" target="_blank">cmgast@gmail.com</a>> wrote:<br>
> I'm sorry, I must be getting confused. My message from yesterday (question<br>
> 1) asked if the objective function value in the .par file was the<br>
> post-integration marginal likelihood, to which you responded in the<br>
> negative, but now it seems as if you're saying what I had originally<br>
> postulated is true. Maybe it's just a terminology difference. It is my<br>
> understanding that the .par file contains the optimum loglikelihood value<br>
> obtained, where the loglikelihood value is marginalized over the random<br>
> effects. Perhaps there is some miscommunication? I think we're talking<br>
> about the same thing.<br>
<br>
Yes, we are. I misunderstood your original question.<br>
<br>
> Since the marginal likelihood is available in the .par file, is there a way<br>
> to output the conditional likelihood following optimization?<br>
<br>
No, not any direct way that I am aware of. It is possible that this<br>
is what you get before the random effects kick in (assuming those<br>
active in phase 2 or later).<br>
<br>
hans</p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@admb-project.org" target="_blank">Users@admb-project.org</a><br>
<a href="http://lists.admb-project.org/mailman/listinfo/users" target="_blank">http://lists.admb-project.org/mailman/listinfo/users</a></p>
</div>
</div>
</div>
<p class="MsoNormal"> </p>
</div>
</div></div></div>
</div>
</blockquote></div><br></div>