[Developers] memory defaults

Wed Apr 25 10:37:10 PDT 2012

Hi Developers,
I committed some increases in default memory settings, including 100x
increase in GRADSTACK_BUFFER_SIZE and CMPDIF_BUFFER_SIZE and 10x increases
in MAX_**NVAR_OFFSET and NUM_**DEPENDENT_VARIABLES. A full list of
differences is here:
http://www.admb-project.org/redmine/projects/issues/repository/revisions/431/diff/trunk/src/linad99/gradstrc.cpp
.

This certainly won't work for all models, but most model should be able to
have their settings in TOP_OF_MAIN eliminated and still run fine, with the
exception of arrmblsize settings.

Currently the default arrmblsize is set by having tpl2cpp insert the
following lines:

  #if defined(__GNUDOS__) || defined(DOS386) || defined(__DPMI32__)  || \
     defined(__MSVC32__)
      if (!arrmblsize) arrmblsize=150000;
  #else
      if (!arrmblsize) arrmblsize=25000;
  #endif

Could someone with more knowledge of these matters than me explain what's
going on here?
Is the first "if" just identifying windows systems, and if so, is it still
necessary?
-Ian

On Fri, Mar 23, 2012 at 2:12 PM, Ian Taylor <ian.taylor at noaa.gov> wrote:

> Allan and Arni make good points.
> I don't mean for this effort to get out of hand, nor that we will come up
> with the silver bullet of memory settings. The current default for CMPDIF
> and GRADSTACK buffers are about 1.3MB which is similar to the limits of my
> first cell phone (a Nokia from 12 years ago). My current phone has about
> 500 times more memory.
>
> Bumping up the default limits by a factor of 100 would still keep things
> small while significantly reducing the number of errors that new users have
> to wade through.
>
> In the future, we could presumably add a switch that would use higher
> defaults for ADMB-RE models.
>  -Ian
>
>
> On Fri, Mar 23, 2012 at 1:50 PM, Allan Hicks <allan.hicks at noaa.gov> wrote:
>
>> And here are the settings for Awatea (a branch of Coleraine):
>>
>>     gradient_structure::set_GRADSTACK_BUFFER_SIZE(1000000);
>>     gradient_structure::set_CMPDIF_BUFFER_SIZE(15000000);
>>      gradient_structure::set_MAX_NVAR_OFFSET(500);
>>     gradient_structure::set_NUM_DEPENDENT_VARIABLES(500);
>>
>>
>> To answer Arni's question about memory allocation, I believe than Ian
>> Taylor and I found that setting high values of GRADSTACK_BUFFER_SIZE did
>> not allocate more memory than necessary, but setting high values of
>> CMPDIFF_BUFFER_SIZE did increase the memory usage even when additional
>> memory was not needed.  Also, I think that one needs to be careful and
>> think of the minimum setup a user will have that will be acceptable.  For
>> example, Ian and I set some defaults for Stock Synthesis that seemed fine
>> on our machines, but a user with less memory received a memory allocation
>> error that made the model fail.  So, when setting default values for ADMB,
>> I think that we may need to consider the minimum specifications of all
>> platforms under consideration. (Didn't Dave once compile a model for
>> Android or some tablet OS? Not that we all want to do stock assessments on
>> our phones.)
>>
>> Allan
>>
>> On Fri, Mar 23, 2012 at 1:39 PM, Arni Magnusson <arnima at hafro.is> wrote:
>>
>>> Sorry about the triple mail, but here are some more examples:
>>>
>>>
>>> Salmon survival (GLM by Magnusson)
>>>
>>> TOP_OF_MAIN_SECTION
>>>  arrmblsize = 1000000;
>>>
>>> The salmon data had 8000 observations in 4 columns. I found 1000000
>>> after increasing incrementally by a factor 10, if I remember correctly -
>>> this was a decade ago...
>>>
>>>
>>> SAM (stock assmt model in ADMB-RE by Nielsen)
>>>
>>> TOP_OF_MAIN_SECTION
>>>  arrmblsize=2000000;
>>>  gradient_structure::set_**GRADSTACK_BUFFER_SIZE(150000);
>>>  gradient_structure::set_**CMPDIF_BUFFER_SIZE(800000);
>>>  gradient_structure::set_MAX_**NVAR_OFFSET(100000);
>>>  gradient_structure::set_NUM_**DEPENDENT_VARIABLES(5000);
>>>
>>>
>>> ADMB Examples (Fournier, only 3 out of 10 models change memory limits)
>>>  arrmblsize from 500000 to 20000000
>>>  GRADSTACK_BUFFER_SIZE from 200000 to 1000000
>>>  CMPDIF_BUFFER_SIZE from 2100000 to 25000000
>>>  MAX_NVAR_OFFSET 500 to 1200
>>>
>>>
>>> ADMB-RE Examples (Fournier and Skaug, over 20 models)
>>>
>>> TOP_OF_MAIN_SECTION
>>>  arrmblsize from 400000 to 40000000
>>>  GRADSTACK_BUFFER_SIZE from 300000 to 30000000
>>>  CMPDIF_BUFFER_SIZE from 20000 to 100000000
>>>  MAX_NVAR_OFFSET from 2000 to 2000000
>>>
>>>
>>> Maybe ADMB-RE users can report how their settings compare with the
>>> ADMB-RE examples listed above. My guess is that many ADMB-RE applications
>>> may need more memory than the examples above - as Fournier and Skaug didn't
>>> need very large datasets to demonstrate their models (and 'make verify'
>>> should finish reasonably fast).
>>>
>>>
>>> By the way, will the increased memory defaults mean that tiny ADMB
>>> models will take substantial memory when run? Or are these just limits on
>>> how much memory a model can grab?
>>>
>>>
>>>
>>> Arni
>>>
>>>
>>>
>>> On Thu, 22 Mar 2012, Ian Taylor wrote:
>>>
>>>  Hi ADMB Users,
>>>>
>>>> The default memory settings in ADMB are out of date and I've been
>>>> tasked with coming up with replacements that will work for reasonable sized
>>>> models (big models will still require manual settings). It occurs to me
>>>> that it makes sense to survey what common requirements of ADMB users might
>>>> be.
>>>>
>>>> Have you been required to change the memory settings for your model
>>>> using either commands like the following in the TOP_OF_MAIN_SECTION,
>>>>
>>>>  arrmblsize=500000;
>>>>  gradient_structure::set_**GRADSTACK_BUFFER_SIZE(200000);
>>>>  gradient_structure::set_**CMPDIF_BUFFER_SIZE(2100000);
>>>>  gradient_structure::set_MAX_**NVAR_OFFSET(500);
>>>>
>>>> or through command line arguments like -ams, -gbs, -cbs, or -mno?
>>>>
>>>> If so, and if you've put thought into the choices (rather than blindly
>>>> picking values), could you send me your settings in a reply to this email?
>>>>
>>>> Thanks,
>>>>
>>>> -Ian
>>>>
>>>>  ______________________________**_________________
>>> Users mailing list
>>> Users at admb-project.org
>>> http://lists.admb-project.org/**mailman/listinfo/users<http://lists.admb-project.org/mailman/listinfo/users>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.admb-project.org/pipermail/developers/attachments/20120425/6286e120/attachment.html>