I too would happily have my models do minimization in 1/32 the time. Reading about OpenCL makes it sound quite flexible across different GPU models.<div>Perhaps for starters, Dave, could you commit or email the modified newfmin.cpp file, perhaps with a brief note about any other changes needed to make it run.</div>
<div><br></div><div>Arni, if HAFRO has an extra US$2.10, you can rent an hour of GPU cluster from Amazon:</div><div><a href="http://aws.typepad.com/aws/2010/11/new-ec2-instance-type-the-cluster-gpu-instance.html">http://aws.typepad.com/aws/2010/11/new-ec2-instance-type-the-cluster-gpu-instance.html</a>
</div><br><div class="gmail_quote">On Tue, May 8, 2012 at 10:44 AM, Arni Magnusson <span dir="ltr"><<a href="mailto:arnima@hafro.is" target="_blank">arnima@hafro.is</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Dave,<br>
<br>
Speeding up ADMB by a factor of 32 is a staggering improvement indeed! It's great to see the example written in user-end TPL (sections etc.) and not some lowest-level hardware-specific system calls in C. Obviously, those calls are being made somewhere, but the level of abstraction that you demonstrate looks very promising.<br>
<br>
Like you have suggested, it is time to get as many ADMB developers as possible to try this out, but how do I start? Specifically:<br>
<br>
(1) Does this approach assume a GPU from a certain vendor?<br>
<br>
(2) Do I need to install some GPU developer library? Hopefully no license conflict with BSD.<br>
<br>
(3) Can we expect this to work on all operating systems?<br>
<br>
(4) Can we do this speed comparison for other ADMB examples, like simple.tpl, catage.tpl, or even random effects? If not, what would need to be done in order to run those models on GPU?<br>
<br>
I ask these questions in an enthusiastic, not pessimistic, tone. This is revolutionary stuff. After all, the shiny 48-core Linux server downstairs probably has no GPU, so maybe the next IT purchase should be a GPU cluster?<br>
<br>
It would be great if ADMB will provide a -gpu option in the near future. I imagine the user would pass that option at an early stage (adcomp and adlink) and not the end stage (as in mymodel -gpu)?<span class="HOEnZb"><font color="#888888"><br>
<br>
Arni</font></span><div class="HOEnZb"><div class="h5"><br>
<br>
<br>
<br>
On Tue, 8 May 2012, dave fournier wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
To get a proof of concept for any programming technique it is nice to get a big result fairly easily. almost all ADMB users rely on the function minimizer fmin in the file newfmin.cpp. So to improve the performance of this function in a more or less transparent would immediately help a lot of users.<br>
<br>
<br>
I hacked the newfmin.cpp file to add the BFGS quasi Newton update with the (sort of) hess inverse kept on the GPU and main calcs done on the GPU.<br>
<br>
I tested this with a modified Rosenbrock function with 6144 parameters. The new setup is both much faster and more stable than the old one on newfmin. It appears that newfmin uses a different quasi-Newton update which is not as efficient for a large number of parameters.<br>
<br>
This is the tpl file for the example.<br>
<br>
DATA_SECTION<br>
int n<br>
!! n=4096+2048;<br>
PARAMETER_SECTION<br>
init_vector x(1,n);<br>
objective_function_value f<br>
PROCEDURE_SECTION<br>
for (int i=1;i<=n/2;i++)<br>
{<br>
f+=100.*square(square(x(2*i-1)<u></u>)-x(2*i))+square(x(2*i-1)-1.0)<u></u>;<br>
}<br>
<br>
The new GPU version took 36 seconds and 477 function evals to converge<br>
- final statistics:<br>
6144 variables; iteration 277; function evaluation 477<br>
Function value 3.2531e-21; maximum gradient component mag 9.7979e-11<br>
Exit code = 1; converg criter 1.0000e-10<br>
<br>
real 0m35.414s<br>
user 0m4.417s <--- most time waiting for the GPU calcs<br>
sys 0m0.616s<br>
<br>
Old version took 288 seconds to do 477 function evaluations but is not nearly as good at this point.<br>
<br>
6144 variables; iteration 300; function evaluation 485; phase 1<br>
Function value 6.6252316e+00; maximum gradient component mag -8.4966e+00<br>
<br>
Old version converged in about 19 min 36 seconds so the new version with BFGS update on the GPU is about 32 times faster than the old version and probably more stable.<br>
<br>
Here is the old version final output<br>
- final statistics:<br>
6144 variables; iteration 1212; function evaluation 2119<br>
Function value 1.7758e-21; maximum gradient component mag 9.7086e-11<br>
Exit code = 1; converg criter 1.0000e-10<br>
<br>
real 19m36.357s<br>
user 19m35.848s<br>
sys 0m0.093s<br>
<br>
Yawn.<br>
<br>
</blockquote>
______________________________<u></u>_________________<br>
Developers mailing list<br>
<a href="mailto:Developers@admb-project.org" target="_blank">Developers@admb-project.org</a><br>
<a href="http://lists.admb-project.org/mailman/listinfo/developers" target="_blank">http://lists.admb-project.org/<u></u>mailman/listinfo/developers</a><br>
</div></div></blockquote></div><br>