[ADMB Users] ADMB and memory issues

[Mark Payne]

Dear ADMBers,

I am just about to embark on an optimisation problem that involves fitting the parameters of a PDE to a fairly large data set - in the absence of an analytical solution, I need to integrate the PDE numerically to predict each data point. Unfortunately, due to the structure of the problem, I have to integrate the same PDE again slightly differently for each of the 1000 data points I have.

As you can probably see, this gets to be "operation" intensive, requiring many calculations to get to the final likelihood that I will be optimising. I would dearly like to do this in ADMB, but I'm very concerned that the memory issues of keeping track of so many operations and their derivatives may simply blow up in my face and it become impossible to deal with on any mortal machine.

I was therefore wondering if I could get some advice about how I should start to plan this? This is there any way to try and make an a priori guess as to how much memory I'm going to need? How can I keep track of the memory from the software as I add components? Can I do a "memory profiling" exercise or similar, so that I can see which parts of system are the most expensive? Are there any tricks that I can do to reduce the memory requirements? Are there any good resources that deal with these issues?

I'm looking forward to hearing your opinions and ideas.

Cheers,

Mark