<div class="gmail_extra">Hi Developers,</div><div class="gmail_extra">I committed some increases in default memory settings, including 100x increase in GRADSTACK_BUFFER_SIZE and CMPDIF_BUFFER_SIZE and 10x increases in MAX_<u></u>NVAR_OFFSET and NUM_<u></u>DEPENDENT_VARIABLES. A full list of differences is here: <a href="http://www.admb-project.org/redmine/projects/issues/repository/revisions/431/diff/trunk/src/linad99/gradstrc.cpp" target="_blank">http://www.admb-project.org/redmine/projects/issues/repository/revisions/431/diff/trunk/src/linad99/gradstrc.cpp</a>. </div>
<div class="gmail_extra"><br></div><div class="gmail_extra">This certainly won't work for all models, but most model should be able to have their settings in TOP_OF_MAIN eliminated and still run fine, with the exception of arrmblsize settings.</div>
<div class="gmail_extra"><br></div><div class="gmail_extra">Currently the default arrmblsize is set by having tpl2cpp insert the following lines:</div><div class="gmail_extra"><br></div><div class="gmail_extra"><div class="gmail_extra">
<font face="'courier new', monospace"> #if defined(__GNUDOS__) || defined(DOS386) || defined(__DPMI32__) || \</font></div><div class="gmail_extra"><font face="'courier new', monospace"> defined(__MSVC32__)</font></div>
<div class="gmail_extra"><font face="'courier new', monospace"> if (!arrmblsize) arrmblsize=150000;</font></div><div class="gmail_extra"><font face="'courier new', monospace"> #else</font></div><div class="gmail_extra">
<font face="'courier new', monospace"> if (!arrmblsize) arrmblsize=25000;</font></div><div class="gmail_extra"><font face="'courier new', monospace"> #endif</font></div><div><br></div></div><div class="gmail_extra">
Could someone with more knowledge of these matters than me explain what's going on here? </div><div class="gmail_extra">Is the first "if" just identifying windows systems, and if so, is it still necessary?</div>
<div class="gmail_extra">-Ian</div><div class="gmail_extra"><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Mar 23, 2012 at 2:12 PM, Ian Taylor <span dir="ltr"><<a href="mailto:ian.taylor@noaa.gov" target="_blank">ian.taylor@noaa.gov</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Allan and Arni make good points.<div>I don't mean for this effort to get out of hand, nor that we will come up with the silver bullet of memory settings. The current default for CMPDIF and GRADSTACK buffers are about 1.3MB which is similar to the limits of my first cell phone (a Nokia from 12 years ago). My current phone has about 500 times more memory.</div>
<div><br></div><div>Bumping up the default limits by a factor of 100 would still keep things small while significantly reducing the number of errors that new users have to wade through.</div><div><br></div><div>In the future, we could presumably add a switch that would use higher defaults for ADMB-RE models.</div>
<span><font color="#888888">
</font></span><div><span><font color="#888888">-Ian</font></span><div><div><br><div>
<br><div class="gmail_quote">On Fri, Mar 23, 2012 at 1:50 PM, Allan Hicks <span dir="ltr"><<a href="mailto:allan.hicks@noaa.gov" target="_blank">allan.hicks@noaa.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
And here are the settings for Awatea (a branch of Coleraine):<div><br></div><div><div> gradient_structure::set_GRADSTACK_BUFFER_SIZE(1000000);</div><div><div> gradient_structure::set_CMPDIF_BUFFER_SIZE(15000000);</div>
<div>
gradient_structure::set_MAX_NVAR_OFFSET(500);</div><div> gradient_structure::set_NUM_DEPENDENT_VARIABLES(500);</div><div><br></div><div><br></div></div><div>To answer Arni's question about memory allocation, I believe than Ian Taylor and I found that setting high values of GRADSTACK_BUFFER_SIZE did not allocate more memory than necessary, but setting high values of CMPDIFF_BUFFER_SIZE did increase the memory usage even when additional memory was not needed. Also, I think that one needs to be careful and think of the minimum setup a user will have that will be acceptable. For example, Ian and I set some defaults for Stock Synthesis that seemed fine on our machines, but a user with less memory received a memory allocation error that made the model fail. So, when setting default values for ADMB, I think that we may need to consider the minimum specifications of all platforms under consideration. (Didn't Dave once compile a model for Android or some tablet OS? Not that we all want to do stock assessments on our phones.)</div>
<span><font color="#888888">
<div><br></div><div>Allan</div><br></font></span><div class="gmail_quote"><div><div>On Fri, Mar 23, 2012 at 1:39 PM, Arni Magnusson <span dir="ltr"><<a href="mailto:arnima@hafro.is" target="_blank">arnima@hafro.is</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>
Sorry about the triple mail, but here are some more examples:<br>
<br>
<br>
Salmon survival (GLM by Magnusson)<br>
<br>
TOP_OF_MAIN_SECTION<br>
arrmblsize = 1000000;<br>
<br>
The salmon data had 8000 observations in 4 columns. I found 1000000 after increasing incrementally by a factor 10, if I remember correctly - this was a decade ago...<br>
<br>
<br>
SAM (stock assmt model in ADMB-RE by Nielsen)<br>
<br>
TOP_OF_MAIN_SECTION<br>
arrmblsize=2000000;<br>
gradient_structure::set_<u></u>GRADSTACK_BUFFER_SIZE(150000);<br>
gradient_structure::set_<u></u>CMPDIF_BUFFER_SIZE(800000);<br>
gradient_structure::set_MAX_<u></u>NVAR_OFFSET(100000);<br>
gradient_structure::set_NUM_<u></u>DEPENDENT_VARIABLES(5000);<br>
<br>
<br>
ADMB Examples (Fournier, only 3 out of 10 models change memory limits)<br>
arrmblsize from 500000 to 20000000<br>
GRADSTACK_BUFFER_SIZE from 200000 to 1000000<br>
CMPDIF_BUFFER_SIZE from 2100000 to 25000000<br>
MAX_NVAR_OFFSET 500 to 1200<br>
<br>
<br>
ADMB-RE Examples (Fournier and Skaug, over 20 models)<br>
<br>
TOP_OF_MAIN_SECTION<br>
arrmblsize from 400000 to 40000000<br>
GRADSTACK_BUFFER_SIZE from 300000 to 30000000<br>
CMPDIF_BUFFER_SIZE from 20000 to 100000000<br>
MAX_NVAR_OFFSET from 2000 to 2000000<br>
<br>
<br>
Maybe ADMB-RE users can report how their settings compare with the ADMB-RE examples listed above. My guess is that many ADMB-RE applications may need more memory than the examples above - as Fournier and Skaug didn't need very large datasets to demonstrate their models (and 'make verify' should finish reasonably fast).<br>
<br>
<br>
By the way, will the increased memory defaults mean that tiny ADMB models will take substantial memory when run? Or are these just limits on how much memory a model can grab?<div><br>
<br>
<br>
Arni<br>
<br>
<br>
<br>
On Thu, 22 Mar 2012, Ian Taylor wrote:<br>
<br>
</div><div><div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi ADMB Users,<br>
<br>
The default memory settings in ADMB are out of date and I've been tasked with coming up with replacements that will work for reasonable sized models (big models will still require manual settings). It occurs to me that it makes sense to survey what common requirements of ADMB users might be.<br>
<br>
Have you been required to change the memory settings for your model using either commands like the following in the TOP_OF_MAIN_SECTION,<br>
<br>
arrmblsize=500000;<br>
gradient_structure::set_<u></u>GRADSTACK_BUFFER_SIZE(200000);<br>
gradient_structure::set_<u></u>CMPDIF_BUFFER_SIZE(2100000);<br>
gradient_structure::set_MAX_<u></u>NVAR_OFFSET(500);<br>
<br>
or through command line arguments like -ams, -gbs, -cbs, or -mno?<br>
<br>
If so, and if you've put thought into the choices (rather than blindly picking values), could you send me your settings in a reply to this email?<br>
<br>
Thanks,<br>
<br>
-Ian<br>
<br>
</blockquote></div></div></div></div><div><div><div>
______________________________<u></u>_________________<br>
Users mailing list<br>
<a href="mailto:Users@admb-project.org" target="_blank">Users@admb-project.org</a><br>
<a href="http://lists.admb-project.org/mailman/listinfo/users" target="_blank">http://lists.admb-project.org/<u></u>mailman/listinfo/users</a><br>
</div></div></div></blockquote></div><br></div>
</blockquote></div><br></div>
</div></div></div>
</blockquote></div><br></div>