[Developers] admb threading

[Matthew Supernaw]

John,

I actually do not have an approach. In fact, I don't have enough information to begin.  Since there is no design document, what I'm trying to do is elicit a more detailed description of the logic behind these operations. 

Another question.  In your mind, how difficult would it be to modernize and encapsulate the gradient information in the prevariable itself? Would putting effort in to encapsulation make the parallelism effort easier?

Thanks.

On Dec 5, 2012, at 12:41 PM, John Sibert wrote:

> Matthew,
> 
> Putting low-level locks on variable objects (ie dvariable, df1b2variable) is equivalent to blocking every floating point operation in the model, and is guaranteed to slow things down a bit. Blocking the gradient structure would have the same effect. In both cases, I can imagine problems with the order in which the reverse calculations in gradcalc() are executed. This approach leaves gradcalc() as a single thread. Since the reverse computation typically uses 2-5 times as many floating point operations as the forward computation, it would be a good idea to find a way to do these in parallel.
> 
> Duplicating the gradient structures and making deep copies of objects involved in derivative computations seems at first to be a bit inefficient. Perhaps, but memory is cheap and creating thread-specific gradient structures ensures that the derivatives can be correctly computed.
> 
> I'm not saying that your approach will absolutely not work. It might work (in the sense that you can make the derivatives correct), but performance benefits will be difficult to measure and will be application specific.
> 
> Here is a cautionary tale. About 15 years ago, I implemented pthread in an easily parallel section of model for which I had written my own derivative code. The only way it would work was to make separate data structures for each thread in which the derivative information was stored. The derivatives were correct, and it certainly decreased the amount of time spent in that particular section of the code. Alas, the model as a whole only performed slightly better because a different bottleneck came to the top. So you need thread-specific gradient information, and there is no guarantee that things will run faster. The performance benefit is application specific.
> 
> So for now, Johnoel and I will try to implement Dave's approach. We will be looking for operations that are inherently parallelizable (Hessian computations, separable functions, ... ), civilizing
> Dave's protoype code to make it a bit safer, extending the concept to higher dimensional objects, and exploring was to easily implement parallel sections of user models (such as those which use the funnel).
> 
> Thanks for sharing your thoughts and questions. It is one of the benefits of open source.
> 
> Cheers,
> John
> 
> John Sibert
> Emeritus Researcher, SOEST
> University of Hawaii at Manoa
> 
> Visit the ADMB project http://admb-project.org/
> 
> On 12/04/2012 03:46 PM, Matthew Supernaw wrote:
>> I misinterpreted your joke, I apologize.  How about we agree to be polite and continue?
>> 
>> Would a global mutex lock on the gradient_structure work?
>> 
>> If it would work, what is more costly copying data into a master/slave paradigm or having an operational bottle neck at the gradient_structure?
>> 
>> On Dec 4, 2012, at 4:46 PM, dave fournier wrote:
>> 
>>> On 12-12-04 01:15 PM, Matthew Supernaw wrote:
>>>> Which book,  "Concurrent Programming for Procedural Programmers"?
>>> Its usually called a joke.  I was referring to the code in my example.
>>> 
>>>> Anyway,  lets step through the logic of:
>>>> 
>>>> prevariable& operator+(const prevariable& v1, const prevariable& v2)
>>>> 
>>>> 
>>>> you have a call to:
>>>> 
>>>> gradient_structure::GRAD_STACK1->set_gradient_stack4(void(*),double*,double*double*)
>>> Go ahead. Do what you want.
>>>> and then:
>>>> 
>>>> default_evaluation4()
>>>> 
>>>> Why can't there be a mutex lock on gradient_structure?
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Dec 4, 2012, at 3:00 PM, developers-request at admb-project.org wrote:
>>>> 
>>>>> Send Developers mailing list submissions to
>>>>> 	developers at admb-project.org
>>>>> 
>>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>> 	http://lists.admb-project.org/mailman/listinfo/developers
>>>>> or, via email, send a message with subject or body 'help' to
>>>>> 	developers-request at admb-project.org
>>>>> 
>>>>> You can reach the person managing the list at
>>>>> 	developers-owner at admb-project.org
>>>>> 
>>>>> When replying, please edit your Subject line so it is more specific
>>>>> than "Re: Contents of Developers digest..."
>>>>> 
>>>>> 
>>>>> Today's Topics:
>>>>> 
>>>>>   1. Re: admb threading (dave fournier)
>>>>> 
>>>>> 
>>>>> ----------------------------------------------------------------------
>>>>> 
>>>>> Message: 1
>>>>> Date: Tue, 04 Dec 2012 11:56:50 -0800
>>>>> From: dave fournier <davef at otter-rsch.com>
>>>>> To: developers at admb-project.org
>>>>> Subject: Re: [Developers] admb threading
>>>>> Message-ID: <50BE5582.3000002 at otter-rsch.com>
>>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>> 
>>>>> Read the book!
>>>>> 
>>>>> A way to synchronize is in my example referred to earlier. Of course
>>>>> that is
>>>>> just for dvariables. For df1b2 variables there is a lot more structure,
>>>>> but the ideas
>>>>> would be the same.
>>>>> 
>>>>> 
>>>>> ------------------------------
>>>>> 
>>>>> _______________________________________________
>>>>> Developers mailing list
>>>>> Developers at admb-project.org
>>>>> http://lists.admb-project.org/mailman/listinfo/developers
>>>>> 
>>>>> 
>>>>> End of Developers Digest, Vol 46, Issue 10
>>>>> ******************************************
>>>> _______________________________________________
>>>> Developers mailing list
>>>> Developers at admb-project.org
>>>> http://lists.admb-project.org/mailman/listinfo/developers
>> _______________________________________________
>> Developers mailing list
>> Developers at admb-project.org
>> http://lists.admb-project.org/mailman/listinfo/developers
>> 
>