[Developers] admb threading
Johnoel Ancheta
johnoel at hawaii.edu
Tue Dec 4 10:46:44 PST 2012
A branch for ADMB Thread development is available
$ svn checkout http://www.admb-project.org/svn/branches/threaded2
$ make linux
$ make tests
If you would like to work together, please let me know.
Johnoel
On Tue, Dec 4, 2012 at 8:39 AM, Matthew Supernaw
<matthew.supernaw at noaa.gov>wrote:
>
> So I just tried the threading example and it worked.
>
> A couple of questions:
>
> 1. What are the consequences for modifying the copy constructor to make a
> deep copy(i'm sure there are many)?
>
> 2. Can "send_xxx_to_slave" be replaced with a deep copy function?
>
> For instance:
>
> df1b2variable DeepCopy(const df1b2variable &x){
> return deep_copy_of_x;
> }
>
>
> class ADMBThread{
> public:
>
> df1b2variable x;
>
>
> virtual void Run(){
> do something to x….
> }
>
> };
>
>
> int main(){
> df1b2variable x = rand();
> ADMBThread thread;
>
> thread.x = DeepCopy(x);
>
> thread.Start();
>
> }
>
>
> Thanks.
>
> Matthew
>
>
>
>
> On Dec 4, 2012, at 9:19 AM, Matthew Supernaw wrote:
>
>
> Dave,
>
> You are correct, I didn't review your example code. I'll take a look.
>
> Thanks for the architectural overview. It would be convenient if this
> information was recorded in software design document.
>
>
>
> "I guess you didn't bother wasting your time looking at my example for
> dvariables.
>
> The big problem with these objects is not some trivial locking in the
> constructor etc.
> It comes from the fact that the data structure which store all the
> information for
> multiple levels of reverse AD are not thread safe. If you are doing
> df1b2variable
> arithmetic each thread needs its own copy of these."
>
>
> On Dec 3, 2012, at 2:39 PM, developers-request at admb-project.org wrote:
>
> Send Developers mailing list submissions to
> developers at admb-project.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.admb-project.org/mailman/listinfo/developers
> or, via email, send a message with subject or body 'help' to
> developers-request at admb-project.org
>
> You can reach the person managing the list at
> developers-owner at admb-project.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Developers digest..."
>
>
> Today's Topics:
>
> 1. Re: admb threading (Matthew Supernaw)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 3 Dec 2012 14:29:01 -0500
> From: Matthew Supernaw <matthew.supernaw at noaa.gov>
> To: developers at admb-project.org
> Subject: Re: [Developers] admb threading
> Message-ID: <C8E76651-698E-4D98-A995-40A2F259B069 at noaa.gov>
> Content-Type: text/plain; charset="us-ascii"
>
>
> Perhaps making admb type containers thread safe is a good place to start.
> This would make it easier to modify admb internally as well as allow end
> users to make their own applications concurrent.
>
> I've started working on this issue for one of our staff. After profiling
> his model, I found nearly 70-80% of the runtime was in his user_function
> looping through a df1b2matrix and doing operations on its elements. In this
> particular case,
> admb is quite fast and is really just waiting for user_function
> evaluations(nested for loops). This is the perfect location for
> multi-threading.
>
>
> At the moment I'm working on a thread safe wrapper for df1b2matrix in
> order to speed up this particular model. This should be all that is needed
> for this particular case, but I suspect this cause may be a common.
>
>
> Attached is a simple platform independent threading library.
>
>
> My strategy for making df1b2matrix thread safe is to wrap it as follows:
>
>
> 11 #include <admodel.h>
> 12 #include "Threads.hpp"
> 13
> 14
> 15 namespace admb {
> 16
> 17 class ts_df1b2matrix {
> 18
> 19 int allocated(void) {
> 20 noaa::threads::Lock(mutex_);
> 21 df1b2matrix_.allocated();
> 22 }
> 23
> 24 void initialize(void) {
> 25 noaa::threads::Lock(mutex_);
> 26 df1b2matrix_.initialize();
> 27
> 28 }
> 29
> 30 ~ts_df1b2matrix() {
> 31 noaa::threads::Lock(mutex_);
> 32 ~df1b2matrix_;
> 33 }
> 34
> 35 int rowmin(void) const {
> 36 noaa::threads::Lock(mutex_);
> 37 df1b2matrix_.rowmin();
> 38 }
> 39
> 40 int indexmin(void) const {
> 41 noaa::threads::Lock(mutex_);
> 42 df1b2matrix_.indexmin();
> 43 }
> 44
> 45 int indexmax(void) const {
> 46 noaa::threads::Lock(mutex_);
> 47 df1b2matrix_.indexmax();
> 48 }
> 49
> 50 int rowmax(void) const {
> 51 noaa::threads::Lock(mutex_);
> 52 df1b2matrix_.rowmax();
> 53 }
> 54
> 55 int size(void) const {
> 56 noaa::threads::Lock(mutex_);
> 57 df1b2matrix_.size();
> 58 }
> 59
> 60 ts_df1b2matrix(int nrl, int nrh) {
> 61 noaa::threads::Lock(mutex_);
> 62 df1b2matrix_(nrl, nrh);
> 63 }
> 64
> 65 ts_df1b2matrix(const df1b2matrix &other) {
> 66 noaa::threads::Lock(mutex_);
> 67 df1b2matrix_(other);
> 68 }
> 69
> 70 ts_df1b2matrix(const ts_df1b2matrix &other) {
> 71 noaa::threads::Lock(mutex_);
> 72 noaa::threads::Lock2(other.mutex_);
> 73 df1b2matrix_(other.df1b2matrix_);
> 74 }
> 75
> 76 ts_df1b2matrix(int nrl, int nrh, const index_type &ncl, const
> index_type &nch) {
> 77 noaa::threads::Lock(mutex_);
> 78 df1b2matrix(nrl, nrh, ncl, nch);
> 79 }
> 80
> 81 ts_df1b2matrix& operator=(const df3_one_matrix &other) {
> 82 noaa::threads::Lock(mutex_);
> 83 this->df1b2matrix_ = other;
> 84 }
> 85
> .
> .
> .
> .
>
> 162
> 163 private:
> 164 mutable noaa::threads::Mutex mutex_;
> 165 df1b2matrix df1b2matrix_;
> 166
> 167
> 168 };
>
>
> The call "noaa::threads::Lock(mutex_); " will lock the function and then
> unlock it as it goes out of scope.
> I'll be happy to pass the finished product for review.
>
>
>
>
>
>
>
>
>
>
> Matthew Supernaw
> Scientific Programmer
> National Oceanic and Atmospheric Administration
> National Marine Fisheries Service
> Sustainable Fisheries Division
> St. Petersburg, FL, 33701
> Office 727-551-5606
> Fax 727-824-5300
>
> On Nov 29, 2012, at 3:00 PM, developers-request at admb-project.org wrote:
>
> Send Developers mailing list submissions to
>
> developers at admb-project.org
>
>
> To subscribe or unsubscribe via the World Wide Web, visit
>
> http://lists.admb-project.org/mailman/listinfo/developers
>
> or, via email, send a message with subject or body 'help' to
>
> developers-request at admb-project.org
>
>
> You can reach the person managing the list at
>
> developers-owner at admb-project.org
>
>
> When replying, please edit your Subject line so it is more specific
>
> than "Re: Contents of Developers digest..."
>
>
>
> Today's Topics:
>
>
> 1. Re: admb threading (Mark Maunder)
>
> 2. Re: admb threading (dave fournier)
>
>
>
> ----------------------------------------------------------------------
>
>
> Message: 1
>
> Date: Thu, 29 Nov 2012 17:50:41 +0000
>
> From: Mark Maunder <mmaunder at iattc.org>
>
> To: dave fournier <davef at otter-rsch.com>,
>
> "developers at admb-project.org" <developers at admb-project.org>
>
> Subject: Re: [Developers] admb threading
>
> Message-ID:
>
> <339913E1960AE142A9373DFCD849F3DA325FA148 at mail1.lajolla.iattc.org>
>
> Content-Type: text/plain; charset="us-ascii"
>
>
> What about the calculation of the hessian, which can be quite long on
> parameter rich models.
>
> Profile likelihoods would also be another easy one
>
>
>
> -----Original Message-----
>
> From: developers-bounces at admb-project.org [mailto:
> developers-bounces at admb-project.org] On Behalf Of dave fournier
>
> Sent: Thursday, November 29, 2012 9:26 AM
>
> To: developers at admb-project.org
>
> Subject: Re: [Developers] admb threading
>
>
> On 12-11-29 09:11 AM, Hans J. Skaug wrote:
>
>
> The obvious transparent one is the -ndb (num der blocks) which was already
> set up for mult-threading, and I recall Derek was doing something with
> that, but I never heard about it again, and it is not for separable models.
> For separable models one could split up the separable function calls by
> different threads in a transparent manner. Both of these involve using the
> __thread declaration to deal with some global data structures. The real
> point of my proof of concept example was to demonstrate that this can be
> done quite easily.
>
>
>
>
> Both are useful, but currently "transparent to the user" is the most
> important.
>
>
> hans
>
>
> -----Original Message-----
>
> From: developers-bounces at admb-project.org [mailto:developers-
>
> bounces at admb-project.org] On Behalf Of Mark Maunder
>
> Sent: Thursday, November 29, 2012 8:23 AM
>
> To: John Sibert; ADMB Developers
>
> Subject: Re: [Developers] admb threading
>
>
> parallel code that is "transparent" to the user
>
>
> -----Original Message-----
>
> From: developers-bounces at admb-project.org [mailto:developers-
>
> bounces at admb-project.org] On Behalf Of John Sibert
>
> Sent: Wednesday, November 28, 2012 4:30 PM
>
> To: ADMB Developers
>
> Subject: [Developers] admb threading
>
>
> Johnoel and I need some feedback about how to approach threading.
>
> Dave has provided a nice proof of concept using pthreads to implement
>
> parallel processing on larger chunks of code. This approach is likely
>
> to have the biggest performance improvement, but seems application
>
> specific and would require more expertize on the part of users.
>
>
> Alternatively it is possible to implement threading internally in the
>
> ADMB libraries, concentrating on smaller chunks of code, for instance
>
> the solve(...) function. This approach would probably have smaller
>
> performance payoff in most applications, but would be more transparent to
> users.
>
>
> In principle, the two approaches are not mutually exclusive.
>
>
> So my question to the ADMB Developer group is what did we mean when
>
> we assigned a high priority to parallelization? Do we want parallel
>
> code that is "transparent" to the user (if so what parts of the would
>
> have the highest priority)? Or do we want to develop tools that allow
>
> users to create their on threaded code for specific applications?
>
> (Don't tell me both.)
>
>
> Cheers,
>
> John
>
> PS enjoy the attached.
>
>
> --
>
> John Sibert
>
> Emeritus Researcher, SOEST
>
> University of Hawaii at Manoa
>
>
> Visit the ADMB project http://admb-project.org/
>
>
> _______________________________________________
>
> Developers mailing list
>
> Developers at admb-project.org
>
> http://lists.admb-project.org/mailman/listinfo/developers
>
> _______________________________________________
>
> Developers mailing list
>
> Developers at admb-project.org
>
> http://lists.admb-project.org/mailman/listinfo/developers
>
>
> _______________________________________________
>
> Developers mailing list
>
> Developers at admb-project.org
>
> http://lists.admb-project.org/mailman/listinfo/developers
>
>
>
> ------------------------------
>
>
> Message: 2
>
> Date: Thu, 29 Nov 2012 09:53:45 -0800
>
> From: dave fournier <davef at otter-rsch.com>
>
> To: Mark Maunder <mmaunder at iattc.org>
>
> Cc: "developers at admb-project.org" <developers at admb-project.org>
>
> Subject: Re: [Developers] admb threading
>
> Message-ID: <50B7A129.5030200 at otter-rsch.com>
>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>
> On 12-11-29 09:50 AM, Mark Maunder wrote:
>
>
> The biggest improvement to the profile likelihood would be to replace
>
> the current
>
> penalty function method with the augmented Lagrangian.
>
>
> What about the calculation of the hessian, which can be quite long on
> parameter rich models.
>
> Profile likelihoods would also be another easy one
>
>
>
> -----Original Message-----
>
> From: developers-bounces at admb-project.org [mailto:
> developers-bounces at admb-project.org] On Behalf Of dave fournier
>
> Sent: Thursday, November 29, 2012 9:26 AM
>
> To: developers at admb-project.org
>
> Subject: Re: [Developers] admb threading
>
>
> On 12-11-29 09:11 AM, Hans J. Skaug wrote:
>
>
> The obvious transparent one is the -ndb (num der blocks) which was already
> set up for mult-threading, and I recall Derek was doing something with
> that, but I never heard about it again, and it is not for separable models.
> For separable models one could split up the separable function calls by
> different threads in a transparent manner. Both of these involve using the
> __thread declaration to deal with some global data structures. The real
> point of my proof of concept example was to demonstrate that this can be
> done quite easily.
>
>
>
>
> Both are useful, but currently "transparent to the user" is the most
> important.
>
>
> hans
>
>
> -----Original Message-----
>
> From: developers-bounces at admb-project.org [mailto:developers-
>
> bounces at admb-project.org] On Behalf Of Mark Maunder
>
> Sent: Thursday, November 29, 2012 8:23 AM
>
> To: John Sibert; ADMB Developers
>
> Subject: Re: [Developers] admb threading
>
>
> parallel code that is "transparent" to the user
>
>
> -----Original Message-----
>
> From: developers-bounces at admb-project.org [mailto:developers-
>
> bounces at admb-project.org] On Behalf Of John Sibert
>
> Sent: Wednesday, November 28, 2012 4:30 PM
>
> To: ADMB Developers
>
> Subject: [Developers] admb threading
>
>
> Johnoel and I need some feedback about how to approach threading.
>
> Dave has provided a nice proof of concept using pthreads to implement
>
> parallel processing on larger chunks of code. This approach is likely
>
> to have the biggest performance improvement, but seems application
>
> specific and would require more expertize on the part of users.
>
>
> Alternatively it is possible to implement threading internally in the
>
> ADMB libraries, concentrating on smaller chunks of code, for instance
>
> the solve(...) function. This approach would probably have smaller
>
> performance payoff in most applications, but would be more transparent to
> users.
>
>
> In principle, the two approaches are not mutually exclusive.
>
>
> So my question to the ADMB Developer group is what did we mean when
>
> we assigned a high priority to parallelization? Do we want parallel
>
> code that is "transparent" to the user (if so what parts of the would
>
> have the highest priority)? Or do we want to develop tools that allow
>
> users to create their on threaded code for specific applications?
>
> (Don't tell me both.)
>
>
> Cheers,
>
> John
>
> PS enjoy the attached.
>
>
> --
>
> John Sibert
>
> Emeritus Researcher, SOEST
>
> University of Hawaii at Manoa
>
>
> Visit the ADMB project http://admb-project.org/
>
>
> _______________________________________________
>
> Developers mailing list
>
> Developers at admb-project.org
>
> http://lists.admb-project.org/mailman/listinfo/developers
>
> _______________________________________________
>
> Developers mailing list
>
> Developers at admb-project.org
>
> http://lists.admb-project.org/mailman/listinfo/developers
>
> _______________________________________________
>
> Developers mailing list
>
> Developers at admb-project.org
>
> http://lists.admb-project.org/mailman/listinfo/developers
>
>
>
>
> ------------------------------
>
>
> _______________________________________________
>
> Developers mailing list
>
> Developers at admb-project.org
>
> http://lists.admb-project.org/mailman/listinfo/developers
>
>
>
> End of Developers Digest, Vol 45, Issue 5
>
> *****************************************
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.admb-project.org/pipermail/developers/attachments/20121203/086c40d2/attachment.html
> >
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: Threads.hpp
> Type: application/octet-stream
> Size: 27706 bytes
> Desc: not available
> URL: <
> http://lists.admb-project.org/pipermail/developers/attachments/20121203/086c40d2/attachment.obj
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://lists.admb-project.org/pipermail/developers/attachments/20121203/086c40d2/attachment-0001.html
> >
>
> ------------------------------
>
> _______________________________________________
> Developers mailing list
> Developers at admb-project.org
> http://lists.admb-project.org/mailman/listinfo/developers
>
>
> End of Developers Digest, Vol 46, Issue 3
> *****************************************
>
>
>
>
> _______________________________________________
> Developers mailing list
> Developers at admb-project.org
> http://lists.admb-project.org/mailman/listinfo/developers
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.admb-project.org/pipermail/developers/attachments/20121204/fa01c1f3/attachment-0001.html>
More information about the Developers
mailing list