[Developers] admb threading
Matthew Supernaw
matthew.supernaw at noaa.gov
Tue Dec 4 06:19:18 PST 2012
Dave,
You are correct, I didn't review your example code. I'll take a look.
Thanks for the architectural overview. It would be convenient if this information was recorded in software design document.
"I guess you didn't bother wasting your time looking at my example for
dvariables.
The big problem with these objects is not some trivial locking in the
constructor etc.
It comes from the fact that the data structure which store all the
information for
multiple levels of reverse AD are not thread safe. If you are doing
df1b2variable
arithmetic each thread needs its own copy of these."
On Dec 3, 2012, at 2:39 PM, developers-request at admb-project.org wrote:
> Send Developers mailing list submissions to
> developers at admb-project.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.admb-project.org/mailman/listinfo/developers
> or, via email, send a message with subject or body 'help' to
> developers-request at admb-project.org
>
> You can reach the person managing the list at
> developers-owner at admb-project.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Developers digest..."
>
>
> Today's Topics:
>
> 1. Re: admb threading (Matthew Supernaw)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 3 Dec 2012 14:29:01 -0500
> From: Matthew Supernaw <matthew.supernaw at noaa.gov>
> To: developers at admb-project.org
> Subject: Re: [Developers] admb threading
> Message-ID: <C8E76651-698E-4D98-A995-40A2F259B069 at noaa.gov>
> Content-Type: text/plain; charset="us-ascii"
>
>
> Perhaps making admb type containers thread safe is a good place to start. This would make it easier to modify admb internally as well as allow end users to make their own applications concurrent.
>
> I've started working on this issue for one of our staff. After profiling his model, I found nearly 70-80% of the runtime was in his user_function looping through a df1b2matrix and doing operations on its elements. In this particular case,
> admb is quite fast and is really just waiting for user_function evaluations(nested for loops). This is the perfect location for multi-threading.
>
>
> At the moment I'm working on a thread safe wrapper for df1b2matrix in order to speed up this particular model. This should be all that is needed for this particular case, but I suspect this cause may be a common.
>
>
> Attached is a simple platform independent threading library.
>
>
> My strategy for making df1b2matrix thread safe is to wrap it as follows:
>
>
> 11 #include <admodel.h>
> 12 #include "Threads.hpp"
> 13
> 14
> 15 namespace admb {
> 16
> 17 class ts_df1b2matrix {
> 18
> 19 int allocated(void) {
> 20 noaa::threads::Lock(mutex_);
> 21 df1b2matrix_.allocated();
> 22 }
> 23
> 24 void initialize(void) {
> 25 noaa::threads::Lock(mutex_);
> 26 df1b2matrix_.initialize();
> 27
> 28 }
> 29
> 30 ~ts_df1b2matrix() {
> 31 noaa::threads::Lock(mutex_);
> 32 ~df1b2matrix_;
> 33 }
> 34
> 35 int rowmin(void) const {
> 36 noaa::threads::Lock(mutex_);
> 37 df1b2matrix_.rowmin();
> 38 }
> 39
> 40 int indexmin(void) const {
> 41 noaa::threads::Lock(mutex_);
> 42 df1b2matrix_.indexmin();
> 43 }
> 44
> 45 int indexmax(void) const {
> 46 noaa::threads::Lock(mutex_);
> 47 df1b2matrix_.indexmax();
> 48 }
> 49
> 50 int rowmax(void) const {
> 51 noaa::threads::Lock(mutex_);
> 52 df1b2matrix_.rowmax();
> 53 }
> 54
> 55 int size(void) const {
> 56 noaa::threads::Lock(mutex_);
> 57 df1b2matrix_.size();
> 58 }
> 59
> 60 ts_df1b2matrix(int nrl, int nrh) {
> 61 noaa::threads::Lock(mutex_);
> 62 df1b2matrix_(nrl, nrh);
> 63 }
> 64
> 65 ts_df1b2matrix(const df1b2matrix &other) {
> 66 noaa::threads::Lock(mutex_);
> 67 df1b2matrix_(other);
> 68 }
> 69
> 70 ts_df1b2matrix(const ts_df1b2matrix &other) {
> 71 noaa::threads::Lock(mutex_);
> 72 noaa::threads::Lock2(other.mutex_);
> 73 df1b2matrix_(other.df1b2matrix_);
> 74 }
> 75
> 76 ts_df1b2matrix(int nrl, int nrh, const index_type &ncl, const index_type &nch) {
> 77 noaa::threads::Lock(mutex_);
> 78 df1b2matrix(nrl, nrh, ncl, nch);
> 79 }
> 80
> 81 ts_df1b2matrix& operator=(const df3_one_matrix &other) {
> 82 noaa::threads::Lock(mutex_);
> 83 this->df1b2matrix_ = other;
> 84 }
> 85
> .
> .
> .
> .
>
> 162
> 163 private:
> 164 mutable noaa::threads::Mutex mutex_;
> 165 df1b2matrix df1b2matrix_;
> 166
> 167
> 168 };
>
>
> The call "noaa::threads::Lock(mutex_); " will lock the function and then unlock it as it goes out of scope.
> I'll be happy to pass the finished product for review.
>
>
>
>
>
>
>
>
>
>
> Matthew Supernaw
> Scientific Programmer
> National Oceanic and Atmospheric Administration
> National Marine Fisheries Service
> Sustainable Fisheries Division
> St. Petersburg, FL, 33701
> Office 727-551-5606
> Fax 727-824-5300
>
> On Nov 29, 2012, at 3:00 PM, developers-request at admb-project.org wrote:
>
>> Send Developers mailing list submissions to
>> developers at admb-project.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://lists.admb-project.org/mailman/listinfo/developers
>> or, via email, send a message with subject or body 'help' to
>> developers-request at admb-project.org
>>
>> You can reach the person managing the list at
>> developers-owner at admb-project.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of Developers digest..."
>>
>>
>> Today's Topics:
>>
>> 1. Re: admb threading (Mark Maunder)
>> 2. Re: admb threading (dave fournier)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Thu, 29 Nov 2012 17:50:41 +0000
>> From: Mark Maunder <mmaunder at iattc.org>
>> To: dave fournier <davef at otter-rsch.com>,
>> "developers at admb-project.org" <developers at admb-project.org>
>> Subject: Re: [Developers] admb threading
>> Message-ID:
>> <339913E1960AE142A9373DFCD849F3DA325FA148 at mail1.lajolla.iattc.org>
>> Content-Type: text/plain; charset="us-ascii"
>>
>> What about the calculation of the hessian, which can be quite long on parameter rich models.
>> Profile likelihoods would also be another easy one
>>
>>
>> -----Original Message-----
>> From: developers-bounces at admb-project.org [mailto:developers-bounces at admb-project.org] On Behalf Of dave fournier
>> Sent: Thursday, November 29, 2012 9:26 AM
>> To: developers at admb-project.org
>> Subject: Re: [Developers] admb threading
>>
>> On 12-11-29 09:11 AM, Hans J. Skaug wrote:
>>
>> The obvious transparent one is the -ndb (num der blocks) which was already set up for mult-threading, and I recall Derek was doing something with that, but I never heard about it again, and it is not for separable models. For separable models one could split up the separable function calls by different threads in a transparent manner. Both of these involve using the __thread declaration to deal with some global data structures. The real point of my proof of concept example was to demonstrate that this can be done quite easily.
>>
>>
>>
>>> Both are useful, but currently "transparent to the user" is the most important.
>>>
>>> hans
>>>
>>>> -----Original Message-----
>>>> From: developers-bounces at admb-project.org [mailto:developers-
>>>> bounces at admb-project.org] On Behalf Of Mark Maunder
>>>> Sent: Thursday, November 29, 2012 8:23 AM
>>>> To: John Sibert; ADMB Developers
>>>> Subject: Re: [Developers] admb threading
>>>>
>>>> parallel code that is "transparent" to the user
>>>>
>>>> -----Original Message-----
>>>> From: developers-bounces at admb-project.org [mailto:developers-
>>>> bounces at admb-project.org] On Behalf Of John Sibert
>>>> Sent: Wednesday, November 28, 2012 4:30 PM
>>>> To: ADMB Developers
>>>> Subject: [Developers] admb threading
>>>>
>>>> Johnoel and I need some feedback about how to approach threading.
>>>> Dave has provided a nice proof of concept using pthreads to implement
>>>> parallel processing on larger chunks of code. This approach is likely
>>>> to have the biggest performance improvement, but seems application
>>>> specific and would require more expertize on the part of users.
>>>>
>>>> Alternatively it is possible to implement threading internally in the
>>>> ADMB libraries, concentrating on smaller chunks of code, for instance
>>>> the solve(...) function. This approach would probably have smaller
>>>> performance payoff in most applications, but would be more transparent to users.
>>>>
>>>> In principle, the two approaches are not mutually exclusive.
>>>>
>>>> So my question to the ADMB Developer group is what did we mean when
>>>> we assigned a high priority to parallelization? Do we want parallel
>>>> code that is "transparent" to the user (if so what parts of the would
>>>> have the highest priority)? Or do we want to develop tools that allow
>>>> users to create their on threaded code for specific applications?
>>>> (Don't tell me both.)
>>>>
>>>> Cheers,
>>>> John
>>>> PS enjoy the attached.
>>>>
>>>> --
>>>> John Sibert
>>>> Emeritus Researcher, SOEST
>>>> University of Hawaii at Manoa
>>>>
>>>> Visit the ADMB project http://admb-project.org/
>>>>
>>>> _______________________________________________
>>>> Developers mailing list
>>>> Developers at admb-project.org
>>>> http://lists.admb-project.org/mailman/listinfo/developers
>>> _______________________________________________
>>> Developers mailing list
>>> Developers at admb-project.org
>>> http://lists.admb-project.org/mailman/listinfo/developers
>>
>> _______________________________________________
>> Developers mailing list
>> Developers at admb-project.org
>> http://lists.admb-project.org/mailman/listinfo/developers
>>
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Thu, 29 Nov 2012 09:53:45 -0800
>> From: dave fournier <davef at otter-rsch.com>
>> To: Mark Maunder <mmaunder at iattc.org>
>> Cc: "developers at admb-project.org" <developers at admb-project.org>
>> Subject: Re: [Developers] admb threading
>> Message-ID: <50B7A129.5030200 at otter-rsch.com>
>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>
>> On 12-11-29 09:50 AM, Mark Maunder wrote:
>>
>> The biggest improvement to the profile likelihood would be to replace
>> the current
>> penalty function method with the augmented Lagrangian.
>>
>>> What about the calculation of the hessian, which can be quite long on parameter rich models.
>>> Profile likelihoods would also be another easy one
>>>
>>>
>>> -----Original Message-----
>>> From: developers-bounces at admb-project.org [mailto:developers-bounces at admb-project.org] On Behalf Of dave fournier
>>> Sent: Thursday, November 29, 2012 9:26 AM
>>> To: developers at admb-project.org
>>> Subject: Re: [Developers] admb threading
>>>
>>> On 12-11-29 09:11 AM, Hans J. Skaug wrote:
>>>
>>> The obvious transparent one is the -ndb (num der blocks) which was already set up for mult-threading, and I recall Derek was doing something with that, but I never heard about it again, and it is not for separable models. For separable models one could split up the separable function calls by different threads in a transparent manner. Both of these involve using the __thread declaration to deal with some global data structures. The real point of my proof of concept example was to demonstrate that this can be done quite easily.
>>>
>>>
>>>
>>>> Both are useful, but currently "transparent to the user" is the most important.
>>>>
>>>> hans
>>>>
>>>>> -----Original Message-----
>>>>> From: developers-bounces at admb-project.org [mailto:developers-
>>>>> bounces at admb-project.org] On Behalf Of Mark Maunder
>>>>> Sent: Thursday, November 29, 2012 8:23 AM
>>>>> To: John Sibert; ADMB Developers
>>>>> Subject: Re: [Developers] admb threading
>>>>>
>>>>> parallel code that is "transparent" to the user
>>>>>
>>>>> -----Original Message-----
>>>>> From: developers-bounces at admb-project.org [mailto:developers-
>>>>> bounces at admb-project.org] On Behalf Of John Sibert
>>>>> Sent: Wednesday, November 28, 2012 4:30 PM
>>>>> To: ADMB Developers
>>>>> Subject: [Developers] admb threading
>>>>>
>>>>> Johnoel and I need some feedback about how to approach threading.
>>>>> Dave has provided a nice proof of concept using pthreads to implement
>>>>> parallel processing on larger chunks of code. This approach is likely
>>>>> to have the biggest performance improvement, but seems application
>>>>> specific and would require more expertize on the part of users.
>>>>>
>>>>> Alternatively it is possible to implement threading internally in the
>>>>> ADMB libraries, concentrating on smaller chunks of code, for instance
>>>>> the solve(...) function. This approach would probably have smaller
>>>>> performance payoff in most applications, but would be more transparent to users.
>>>>>
>>>>> In principle, the two approaches are not mutually exclusive.
>>>>>
>>>>> So my question to the ADMB Developer group is what did we mean when
>>>>> we assigned a high priority to parallelization? Do we want parallel
>>>>> code that is "transparent" to the user (if so what parts of the would
>>>>> have the highest priority)? Or do we want to develop tools that allow
>>>>> users to create their on threaded code for specific applications?
>>>>> (Don't tell me both.)
>>>>>
>>>>> Cheers,
>>>>> John
>>>>> PS enjoy the attached.
>>>>>
>>>>> --
>>>>> John Sibert
>>>>> Emeritus Researcher, SOEST
>>>>> University of Hawaii at Manoa
>>>>>
>>>>> Visit the ADMB project http://admb-project.org/
>>>>>
>>>>> _______________________________________________
>>>>> Developers mailing list
>>>>> Developers at admb-project.org
>>>>> http://lists.admb-project.org/mailman/listinfo/developers
>>>> _______________________________________________
>>>> Developers mailing list
>>>> Developers at admb-project.org
>>>> http://lists.admb-project.org/mailman/listinfo/developers
>>> _______________________________________________
>>> Developers mailing list
>>> Developers at admb-project.org
>>> http://lists.admb-project.org/mailman/listinfo/developers
>>
>>
>>
>> ------------------------------
>>
>> _______________________________________________
>> Developers mailing list
>> Developers at admb-project.org
>> http://lists.admb-project.org/mailman/listinfo/developers
>>
>>
>> End of Developers Digest, Vol 45, Issue 5
>> *****************************************
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.admb-project.org/pipermail/developers/attachments/20121203/086c40d2/attachment.html>
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: Threads.hpp
> Type: application/octet-stream
> Size: 27706 bytes
> Desc: not available
> URL: <http://lists.admb-project.org/pipermail/developers/attachments/20121203/086c40d2/attachment.obj>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.admb-project.org/pipermail/developers/attachments/20121203/086c40d2/attachment-0001.html>
>
> ------------------------------
>
> _______________________________________________
> Developers mailing list
> Developers at admb-project.org
> http://lists.admb-project.org/mailman/listinfo/developers
>
>
> End of Developers Digest, Vol 46, Issue 3
> *****************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.admb-project.org/pipermail/developers/attachments/20121204/ec476a4e/attachment-0001.html>
More information about the Developers
mailing list