[Developers] profiling ADMB model with gprof
dave fournier
davef at otter-rsch.com
Mon Nov 15 10:52:17 PST 2010
Ian Taylor wrote:
I suspect the difference could be in writing to temporary files.
> The suggestion of trying gprof is something that I'd been meaning to
> do in the past, and now there was an incentive.
> I changed the subject line of this email to better reflect that this
> is now the topic.
>
> I haven't seen any instructions on using gprof for ADMB, so I posted
> the steps that eventually succeeded at
> http://admb-project.org/community/tutorials-and-examples/how-to-use-gprof-the-gnu-profiler.
> If it turns out that these options were already built into the
> existing scripts or if there's an easier way, let me know so I can
> update the instructions on the web. (If the instructions are any good,
> then maybe they could be built upon by Hans, Johnoel, Jim I. and Ray
> Conser, who were assigned the lower priority topic of "code tuning"
> <http://admb-project.org/community/admb-meeting-march-29-31/ADMBMeetingReportv3.pdf#page=4>
> at the past ADMB meeting).
>
> I haven't really digested the results yet, because there are a lot of
> differences between the profiles I got for ADMB versions 9 and 10.
> However, here's a sample of the function calls that took up the most
> time, matched up with the times (in seconds) from version 9 (compiled
> from the identical TPL file and identical inputs):
>
> v9.secs v10.secs function.name
> 2.62 14.8
> dvector::save_dvector_derivatives(dvar_vector_position const&) const
> 5.72 9.43 DF_FILE::fread(double const&)
> NA 9.1 dvector::operator()(int)
> 0.61 6.55 gradient_structure::jacobcalc(int,
> std::basic_ofstream<char, std::char_traits<char> > const&)
> 1.05 6.21 restore_dvar_vector_derivatives(dvar_vector_position
> const&)
> 0.52 5.59 gradcalc(int, dvector const&)
> NA 5.08 dvector::elem(int)
> 1.66 4.33 DF_FILE::fwrite(double)
> 1.96 4.21 restore_dvar_vector_value(dvar_vector_position const&)
> NA 3.86 dvar_vector_position::indexmin() const
>
> These functions that are taking the most time seem to be deep in ADMB
> (or AUTODIF) and not stuff that's easy for a user to work around. If
> I'm not the only one finding version 10 to be slower than version 9,
> then I'm guessing that somebody else is better equipped to suggest
> what to do about it.
> -Ian
>
> On 11/12/2010 6:53 AM, dave fournier wrote:
>> Ian Taylor wrote:
>>
>> You could compile it for profiling and find out where the time is spent
>> with gprof.
>>> After sending the yesterday, I caught the missing -O3. I tried
>>> adding it back and also just doing the compiling with the old
>>> script. But as long as I used the new libraries at the adlink stage,
>>> it went slow. And by slow, I mean more than 8 minutes for a model
>>> that previously took 2 1/2 minutes in ADMB version 9 if optimized
>>> and just over 3 minutes in safe mode (which also suggests that it's
>>> not just a matter of optimizing).
>>>
>>> I can try other models to see if the pattern I saw was consistent,
>>> but I'm also curious if anyone else has compared the speeds between
>>> versions. I don't know if this is in the documentation, but run
>>> times can be quantified with code like the following.
>>> -Ian
>>>
>>> TOP_OF_MAIN_SECTION
>>> time(&start);
>>>
>>> GLOBALS_SECTION
>>> #include <time.h>
>>> time_t start,finish;
>>> double elapsed_time;
>>>
>>> FINAL_SECTION
>>> time(&finish);
>>> elapsed_time = difftime(finish,start);
>>>
>>> On 11/12/2010 2:09 PM, Johnoel Ancheta wrote:
>>>> Opps... Sorry...
>>>>
>>>> I accidentally committed the Makefile without the -O3 option.
>>>>
>>>> I'll correct and commit the changes.
>>>>
>>>> Thanks Dave for catching that.
>>>>
>>>> Johnoel
>>>>
>>>> On 11/12/10 4:09 AM, dave fournier wrote:
>>>>> Ian Taylor wrote:
>>>>>
>>>>> Just a thought Re speed. I don't see a -O3 flag for gcc for
>>>>> optimization.
>>>>>> Hi Johnoel and others,
>>>>>>
>>>>>> Compiling ADMB 10 Beta on with 64-bit linux worked perfectly for
>>>>>> me with the files in the download.
>>>>>>
>>>>>> However, the new script you sent by email failed. From the
>>>>>> commands below (fuller notes attached), it looks like for Ubuntu
>>>>>> at least, it would be better to test for 64 vs 32 using uname -m
>>>>>> rather than uname -p.
>>>>>>
>>>>>> ~/admb/admb-10.0Beta $ uname -a
>>>>>> Linux ian-laptop 2.6.32-25-generic #45-Ubuntu SMP Sat Oct 16
>>>>>> 19:52:42 UTC 2010 x86_64 GNU/Linux
>>>>>> ~/admb/admb-10.0Beta $ uname -s
>>>>>> Linux
>>>>>> ~/admb/admb-10.0Beta $ uname -p
>>>>>> unknown
>>>>>> ~/admb/admb-10.0Beta $ uname -m
>>>>>> x86_64
>>>>>>
>>>>>> Aside from the good news of the successful install, this version
>>>>>> 10 Beta also produced identical results, which is also good news.
>>>>>> However, it seems to be running slower than version 9, by a large
>>>>>> margin, at least for a single model that I tried. Perhaps related
>>>>>> to this, when I compiled using GCC in and ADMB version 9, the
>>>>>> safe executable was about double the size of the optimized.
>>>>>> Compiling with ADMB version 10, the file sizes are similar, and
>>>>>> in between the old sizes. Perhaps the speed and file size
>>>>>> differences are related to changes in the compile scripts? My old
>>>>>> script was dated "23 May 2009".
>>>>>>
>>>>>> Finally, thank you, Johnoel, and all the others who've
>>>>>> contributed to the developments that have gone into this new beta
>>>>>> release.
>>>>>> -Ian
>>>>>>
>>>>>> On 11/10/2010 02:44 PM, Johnoel Ancheta wrote:
>>>>>>> Updated configure script is available.
>>>>>>>
>>>>>>> http://admb-project.googlecode.com/files/configure
>>>>>>>
>>>>>>> Download and replace the configure script in the admb-10.0Beta
>>>>>>> distribution.
>>>>>>>
>>>>>>>
>>>>>>> On 11/10/10 11:09 AM, Ben Bolker wrote:
>>>>>>>> (running under VMWare Fusion on Mac OS X.6, but I don't
>>>>>>>> think that
>>>>>>>> matters)
>>>>>>>>
>>>>>>>> It looks like configure is getting confused and thinking my
>>>>>>>> system is
>>>>>>>> 64-bit ... ?
>>>>>>>>
>>>>>>>>> uname -a
>>>>>>>> Linux ubuntu-10 2.6.32-25-generic #45-Ubuntu SMP Sat Oct 16
>>>>>>>> 19:48:22 UTC
>>>>>>>> 2010 i686 GNU/Linux
>>>>>>>>
>>>>>>>> uname -m
>>>>>>>> i686 ## indicates 32-bit kernel
>>>>>>>>
>>>>>>>> $ gcc -v
>>>>>>>> Using built-in specs.
>>>>>>>> Target: i486-linux-gnu
>>>>>>>> Configured with: ../src/configure -v --with-pkgversion='Ubuntu
>>>>>>>> 4.4.3-4ubuntu5'
>>>>>>>> --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs
>>>>>>>> --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr
>>>>>>>> --enable-shared --enable-multiarch --enable-linker-build-id
>>>>>>>> --with-system-zlib --libexecdir=/usr/lib
>>>>>>>> --without-included-gettext
>>>>>>>> --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4
>>>>>>>> --program-suffix=-4.4 --enable-nls --enable-clocale=gnu
>>>>>>>> --enable-libstdcxx-debug --enable-plugin --enable-objc-gc
>>>>>>>> --enable-targets=all --disable-werror --with-arch-32=i486
>>>>>>>> --with-tune=generic --enable-checking=release
>>>>>>>> --build=i486-linux-gnu
>>>>>>>> --host=i486-linux-gnu --target=i486-linux-gnu
>>>>>>>> Thread model: posix
>>>>>>>> gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> bolker at ubuntu-10:/usr/local/src/admb-10.0Beta$
>>>>>>>> <mailto:bolker at ubuntu-10:/usr/local/src/admb-10.0Beta$>
>>>>>>>> <mailto:bolker at ubuntu-10:/usr/local/src/admb-10.0Beta$>
>>>>>>>> <mailto:bolker at ubuntu-10:/usr/local/src/admb-10.0Beta$> sudo
>>>>>>>> ./configure
>>>>>>>> checking for gcc... gcc
>>>>>>>> checking whether the C compiler works... yes
>>>>>>>> checking for C compiler default output file name... a.out
>>>>>>>> checking for suffix of executables...
>>>>>>>> checking whether we are cross compiling... no
>>>>>>>> checking for suffix of object files... o
>>>>>>>> checking whether we are using the GNU C compiler... yes
>>>>>>>> checking whether gcc accepts -g... yes
>>>>>>>> checking for gcc option to accept ISO C89... none needed
>>>>>>>> checking for g++... g++
>>>>>>>> checking whether we are using the GNU C++ compiler... yes
>>>>>>>> checking whether g++ accepts -g... yes
>>>>>>>> checking how to run the C++ preprocessor... g++ -E
>>>>>>>> checking for grep that handles long lines and -e... /bin/grep
>>>>>>>> checking for egrep... /bin/grep -E
>>>>>>>> checking for ANSI C header files... yes
>>>>>>>> checking for sys/types.h... yes
>>>>>>>> checking for sys/stat.h... yes
>>>>>>>> checking for stdlib.h... yes
>>>>>>>> checking for string.h... yes
>>>>>>>> checking for memory.h... yes
>>>>>>>> checking for strings.h... yes
>>>>>>>> checking for inttypes.h... yes
>>>>>>>> checking for stdint.h... yes
>>>>>>>> checking for unistd.h... yes
>>>>>>>> checking iostream usability... yes
>>>>>>>> checking iostream presence... yes
>>>>>>>> checking for iostream... yes
>>>>>>>> checking for flex... flex
>>>>>>>> checking lex output file root... lex.yy
>>>>>>>> checking lex library... none needed
>>>>>>>> checking whether yytext is a pointer... no
>>>>>>>> checking for sed... yes
>>>>>>>> checking whether C++ compiler is 64-bit enabled... yes
>>>>>>>> checking for svnversion... yes
>>>>>>>> configure: creating ./config.status
>>>>>>>> config.status: creating GNUmakefile
>>>>>>>> config.status: creating admb_configure.h
>>>>>>>> config.status: creating scripts/bash/admb-bin
>>>>>>>>
>>>>>>>> Configure completed for admb10.0Beta-linux-gcc4.4.3-64bit
>>>>>>>> To build ADMB, type 'make'
>>>>>>>> bolker at ubuntu-10:/usr/local/src/admb-10.0Beta$
>>>>>>>> <mailto:bolker at ubuntu-10:/usr/local/src/admb-10.0Beta$>
>>>>>>>> <mailto:bolker at ubuntu-10:/usr/local/src/admb-10.0Beta$>
>>>>>>>> <mailto:bolker at ubuntu-10:/usr/local/src/admb-10.0Beta$> sudo make
>>>>>>>> CXX=g++ CXXFLAGS="-m64"
>>>>>>>> LIBPATH=../build/admb10.0Beta-linux-gcc4.4.3-64bit/objects
>>>>>>>> DISK=../build/admb10.0Beta-linux-gcc4.4.3-64bit/dist
>>>>>>>> ADMB_CONFIGURE=1
>>>>>>>> make --directory=src --file=linux.mak
>>>>>>>> make[1]: Entering directory `/usr/local/src/admb-10.0Beta/src'
>>>>>>>> rm -rf ../build/admb10.0Beta-linux-gcc4.4.3-64bit/dist
>>>>>>>> mkdir -p
>>>>>>>> ../build/admb10.0Beta-linux-gcc4.4.3-64bit/dist/{bin,lib,include,docs,docs/manuals,examples}
>>>>>>>>
>>>>>>>> mkdir -p
>>>>>>>> ../build/admb10.0Beta-linux-gcc4.4.3-64bit/objects/linad99-olp
>>>>>>>> mkdir -p
>>>>>>>> ../build/admb10.0Beta-linux-gcc4.4.3-64bit/objects/linad99-slp
>>>>>>>> mkdir -p
>>>>>>>> ../build/admb10.0Beta-linux-gcc4.4.3-64bit/objects/nh99-olp
>>>>>>>> mkdir -p
>>>>>>>> ../build/admb10.0Beta-linux-gcc4.4.3-64bit/objects/nh99-olp-stub
>>>>>>>> mkdir -p
>>>>>>>> ../build/admb10.0Beta-linux-gcc4.4.3-64bit/objects/tools99-olp
>>>>>>>> mkdir -p
>>>>>>>> ../build/admb10.0Beta-linux-gcc4.4.3-64bit/objects/df1b2-separable-slp
>>>>>>>>
>>>>>>>> mkdir -p
>>>>>>>> ../build/admb10.0Beta-linux-gcc4.4.3-64bit/objects/df1b2-separable-olp
>>>>>>>>
>>>>>>>> make --directory=df1b2-separable CC=gcc CXXFLAGS="-c -m64 -Wall
>>>>>>>> -Wno-deprecated -DUSE_LAPLACE -fpermissive -I../df1b2-separable
>>>>>>>> -I../nh99 -I../linad99 -I../tools99 -D__SPDLL__ -D__GNUDOS__
>>>>>>>> -Dlinux
>>>>>>>> -DOPT_LIB"
>>>>>>>> LIBPATH=../../build/admb10.0Beta-linux-gcc4.4.3-64bit/objects/df1b2-separable-olp
>>>>>>>>
>>>>>>>> DISKDIR=../../build/admb10.0Beta-linux-gcc4.4.3-64bit/dist -f
>>>>>>>> optg32-rh8-laplace.mak disk
>>>>>>>> make[2]: Entering directory
>>>>>>>> `/usr/local/src/admb-10.0Beta/src/df1b2-separable'
>>>>>>>> g++ -c -m64 -Wall -Wno-deprecated -DUSE_LAPLACE -fpermissive
>>>>>>>> -I../df1b2-separable -I../nh99 -I../linad99 -I../tools99
>>>>>>>> -D__SPDLL__
>>>>>>>> -D__GNUDOS__ -Dlinux -DOPT_LIB df1b2impspf.cpp -o
>>>>>>>> ../../build/admb10.0Beta-linux-gcc4.4.3-64bit/objects/df1b2-separable-olp/df1b2impspf.obj
>>>>>>>>
>>>>>>>> In file included from /usr/include/features.h:378,
>>>>>>>> from /usr/include/unistd.h:26,
>>>>>>>> from ../linad99/fvar.hpp:173,
>>>>>>>> from ../nh99/admodel.h:69,
>>>>>>>> from df1b2impspf.cpp:8:
>>>>>>>> /usr/include/gnu/stubs.h:9:27: error: gnu/stubs-64.h: No such
>>>>>>>> file or
>>>>>>>> directory
>>>>>>>> df1b2impspf.cpp: In function ‘double
>>>>>>>> calculate_importance_sample_shess(const dvector&, const
>>>>>>>> dvector&, const
>>>>>>>> dmatrix&, const dvector&, const dvector&, const dmatrix&,
>>>>>>>> function_minimizer*)’:
>>>>>>>> df1b2impspf.cpp:20: warning: unused variable ‘Hessadjoint’
>>>>>>>> df1b2impspf.cpp:44: warning: unused variable ‘onvar’
>>>>>>>> df1b2impspf.cpp:49: warning: unused variable ‘j’
>>>>>>>> make[2]: *** [df1b2impspf.obj] Error 1
>>>>>>>> make[2]: Leaving directory
>>>>>>>> `/usr/local/src/admb-10.0Beta/src/df1b2-separable'
>>>>>>>> make[1]: *** [dist] Error 2
>>>>>>>> make[1]: Leaving directory `/usr/local/src/admb-10.0Beta/src'
>>>>>>>> make: *** [all] Error 2
>>>>>>>> _______________________________________________
>>>>>>>> Developers mailing list
>>>>>>>> Developers at admb-project.org
>>>>>>>> <mailto:Developers at admb-project.org>
>>>>>>>> <mailto:Developers at admb-project.org>
>>>>>>>> <mailto:Developers at admb-project.org>
>>>>>>>> http://lists.admb-project.org/mailman/listinfo/developers
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Developers mailing list
>>>>>>> Developers at admb-project.org <mailto:Developers at admb-project.org>
>>>>>>> <mailto:Developers at admb-project.org>
>>>>>>> <mailto:Developers at admb-project.org>
>>>>>>> http://lists.admb-project.org/mailman/listinfo/developers
>>>>>> ------------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Developers mailing list
>>>>>> Developers at admb-project.org <mailto:Developers at admb-project.org>
>>>>>> <mailto:Developers at admb-project.org>
>>>>>> http://lists.admb-project.org/mailman/listinfo/developers
>>>>>
>>>>
>>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Developers mailing list
> Developers at admb-project.org
> http://lists.admb-project.org/mailman/listinfo/developers
>
More information about the Developers
mailing list