[Developers] memory defaults
dave fournier
davef at otter-rsch.com
Wed Apr 25 15:47:17 PDT 2012
On 12-04-25 10:37 AM, Ian Taylor wrote:
> Hi Developers,
> I committed some increases in default memory settings, including 100x
> increase in GRADSTACK_BUFFER_SIZE and CMPDIF_BUFFER_SIZE and 10x
> increases in MAX_NVAR_OFFSET and NUM_DEPENDENT_VARIABLES. A full list
> of differences is here:
> http://www.admb-project.org/redmine/projects/issues/repository/revisions/431/diff/trunk/src/linad99/gradstrc.cpp.
>
>
> This certainly won't work for all models, but most model should be
> able to have their settings in TOP_OF_MAIN eliminated and still run
> fine, with the exception of arrmblsize settings.
>
> Currently the default arrmblsize is set by having tpl2cpp insert the
> following lines:
>
> #if defined(__GNUDOS__) || defined(DOS386) || defined(__DPMI32__) || \
> defined(__MSVC32__)
> if (!arrmblsize) arrmblsize=150000;
> #else
> if (!arrmblsize) arrmblsize=25000;
> #endif
>
Its not really relevant any more. I think it was mostly to distiguish
between 16 and 32 bit
version of MS operating systems. The frist version of autodif was
running on 640K limit
16 bit real mode MSDOS. DPMI32 was a 32 bit dos extender on top of that
etc.
This is all around 1990 when men were men.
> Could someone with more knowledge of these matters than me explain
> what's going on here?
> Is the first "if" just identifying windows systems, and if so, is it
> still necessary?
> -Ian
>
>
>
> On Fri, Mar 23, 2012 at 2:12 PM, Ian Taylor <ian.taylor at noaa.gov
> <mailto:ian.taylor at noaa.gov>> wrote:
>
> Allan and Arni make good points.
> I don't mean for this effort to get out of hand, nor that we will
> come up with the silver bullet of memory settings. The current
> default for CMPDIF and GRADSTACK buffers are about 1.3MB which is
> similar to the limits of my first cell phone (a Nokia from 12
> years ago). My current phone has about 500 times more memory.
>
> Bumping up the default limits by a factor of 100 would still keep
> things small while significantly reducing the number of errors
> that new users have to wade through.
>
> In the future, we could presumably add a switch that would use
> higher defaults for ADMB-RE models.
> -Ian
>
>
> On Fri, Mar 23, 2012 at 1:50 PM, Allan Hicks <allan.hicks at noaa.gov
> <mailto:allan.hicks at noaa.gov>> wrote:
>
> And here are the settings for Awatea (a branch of Coleraine):
>
> gradient_structure::set_GRADSTACK_BUFFER_SIZE(1000000);
> gradient_structure::set_CMPDIF_BUFFER_SIZE(15000000);
> gradient_structure::set_MAX_NVAR_OFFSET(500);
> gradient_structure::set_NUM_DEPENDENT_VARIABLES(500);
>
>
> To answer Arni's question about memory allocation, I believe
> than Ian Taylor and I found that setting high values of
> GRADSTACK_BUFFER_SIZE did not allocate more memory than
> necessary, but setting high values of CMPDIFF_BUFFER_SIZE did
> increase the memory usage even when additional memory was not
> needed. Also, I think that one needs to be careful and think
> of the minimum setup a user will have that will be acceptable.
> For example, Ian and I set some defaults for Stock Synthesis
> that seemed fine on our machines, but a user with less memory
> received a memory allocation error that made the model fail.
> So, when setting default values for ADMB, I think that we may
> need to consider the minimum specifications of all platforms
> under consideration. (Didn't Dave once compile a model for
> Android or some tablet OS? Not that we all want to do stock
> assessments on our phones.)
>
> Allan
>
> On Fri, Mar 23, 2012 at 1:39 PM, Arni Magnusson
> <arnima at hafro.is <mailto:arnima at hafro.is>> wrote:
>
> Sorry about the triple mail, but here are some more examples:
>
>
> Salmon survival (GLM by Magnusson)
>
> TOP_OF_MAIN_SECTION
> arrmblsize = 1000000;
>
> The salmon data had 8000 observations in 4 columns. I
> found 1000000 after increasing incrementally by a factor
> 10, if I remember correctly - this was a decade ago...
>
>
> SAM (stock assmt model in ADMB-RE by Nielsen)
>
> TOP_OF_MAIN_SECTION
> arrmblsize=2000000;
> gradient_structure::set_GRADSTACK_BUFFER_SIZE(150000);
> gradient_structure::set_CMPDIF_BUFFER_SIZE(800000);
> gradient_structure::set_MAX_NVAR_OFFSET(100000);
> gradient_structure::set_NUM_DEPENDENT_VARIABLES(5000);
>
>
> ADMB Examples (Fournier, only 3 out of 10 models change
> memory limits)
> arrmblsize from 500000 to 20000000
> GRADSTACK_BUFFER_SIZE from 200000 to 1000000
> CMPDIF_BUFFER_SIZE from 2100000 to 25000000
> MAX_NVAR_OFFSET 500 to 1200
>
>
> ADMB-RE Examples (Fournier and Skaug, over 20 models)
>
> TOP_OF_MAIN_SECTION
> arrmblsize from 400000 to 40000000
> GRADSTACK_BUFFER_SIZE from 300000 to 30000000
> CMPDIF_BUFFER_SIZE from 20000 to 100000000
> MAX_NVAR_OFFSET from 2000 to 2000000
>
>
> Maybe ADMB-RE users can report how their settings compare
> with the ADMB-RE examples listed above. My guess is that
> many ADMB-RE applications may need more memory than the
> examples above - as Fournier and Skaug didn't need very
> large datasets to demonstrate their models (and 'make
> verify' should finish reasonably fast).
>
>
> By the way, will the increased memory defaults mean that
> tiny ADMB models will take substantial memory when run? Or
> are these just limits on how much memory a model can grab?
>
>
>
> Arni
>
>
>
> On Thu, 22 Mar 2012, Ian Taylor wrote:
>
> Hi ADMB Users,
>
> The default memory settings in ADMB are out of date
> and I've been tasked with coming up with replacements
> that will work for reasonable sized models (big models
> will still require manual settings). It occurs to me
> that it makes sense to survey what common requirements
> of ADMB users might be.
>
> Have you been required to change the memory settings
> for your model using either commands like the
> following in the TOP_OF_MAIN_SECTION,
>
> arrmblsize=500000;
> gradient_structure::set_GRADSTACK_BUFFER_SIZE(200000);
> gradient_structure::set_CMPDIF_BUFFER_SIZE(2100000);
> gradient_structure::set_MAX_NVAR_OFFSET(500);
>
> or through command line arguments like -ams, -gbs,
> -cbs, or -mno?
>
> If so, and if you've put thought into the choices
> (rather than blindly picking values), could you send
> me your settings in a reply to this email?
>
> Thanks,
>
> -Ian
>
> _______________________________________________
> Users mailing list
> Users at admb-project.org <mailto:Users at admb-project.org>
> http://lists.admb-project.org/mailman/listinfo/users
>
>
>
>
>
> _______________________________________________
> Developers mailing list
> Developers at admb-project.org
> http://lists.admb-project.org/mailman/listinfo/developers
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.admb-project.org/pipermail/developers/attachments/20120425/6a694bd1/attachment.html>
More information about the Developers
mailing list