[ADMB Users] a difference in estimates between versions?
Derek Seiple
dseiple84 at gmail.com
Wed Oct 5 06:59:21 PDT 2011
I don't know much about the IDE part of things. Does anyone know if
ADMB-IDE gets updated as bug fixes are made to ADMB?
I ask because if I recall correctly, I think one of the functions used
in this tpl had a bug in it that was corrected after the release of
version 10. So the answer to my question could be important. Of course
I won't know until I test this example.
Saang-Yoon,
You can send me your dat file at dseiple84 at gmail.com, or send it to
the users list so all can see it.
Derek
On Wed, Oct 5, 2011 at 8:41 AM, Saang-Yoon <shyunuw at gmail.com> wrote:
> Allen, Derek, Ian and all.
>
> As requested, I am showing an actual example. Please note obvious
> differences between (1) and (2) below, esp. in SE of B1. The common
> TPL file is shown below.
> DAT file has long rows. I don't see here how to attach the file. If
> you need DAT file, let me your email so that I can send it.
>
> Not only difference in the results, but also I am surprised that these
> two versions differently work in response to a small change of
> parameter bounds. For example in the current TPL, setting the r
> range as 0-1 works in both versions, but setting it as 0-2 does NOT
> work in the 450-1. It is too soon to conclude, but my (limited)
> experience let me prefer older versions prior to 450-1: e.g., 440-2.
> Thank you very much for your response.
>
> Saang-Yoon
>
>
> (1) Results from previous version, ADMB IDE 450-1
>
> index name value std dev
> 1 r 4.9970e-001 7.2940e-002
> 2 B1 8.9504e+004 1.3599e+004
> 3 K 9.1202e+004 7.9562e+003
> 4 lnq -2.1695e+000 1.9095e-001
>
>
> (2) Results from the latest version, ADMB-IDE-452-1 (64bit)
> index name value std dev
> 1 r 5.0509e-001 1.4047e-002
> 2 B1 8.8638e+004 2.9544e+003
> 3 K 9.0710e+004 1.3304e+003
> 4 lnq -2.1587e+000 1.1781e-001
>
>
> ========== The following TPL was used.=============
>
> DATA_SECTION
> int i; //for loop;
>
> init_vector years(1,2);
> int nyrs;
> int ncol; //data column
> !!nyrs=years(2)-years(1)+1;
> !!ncol=3;
>
> init_matrix yieldsurvey(1,nyrs,1,ncol);
> vector yrs(1,nyrs);
> vector Yt(1,nyrs);
> vector survey(1,nyrs); //survey data
>
> !!yrs=trans(yieldsurvey)(1);
> !!Yt=trans(yieldsurvey)(2); //yield;
> !!survey=trans(yieldsurvey)(3);
>
> number minB1;
> number maxB1;
> !!minB1=Yt(1)-Yt(1)*15/100;
> !!maxB1=minB1+minB1*3000/100;
> //!!maxB1=sum(Yt);
>
> number minK;
> number maxK;
> !!minK=max(Yt)-max(Yt)*15/100;
> !!maxK=minK+minK*3000/100;
> //!!maxK=sum(Yt)*10;
>
> number minY;
> number meanY;
> !!minY=min(Yt);
> !!meanY=mean(Yt);
>
> !!cout<<"yrs: "<<yrs<<endl;
> !!cout<<"Yt: "<<Yt<<endl;
> !!cout<<"survey: "<<survey<<endl;
>
> PARAMETER_SECTION
> init_bounded_number r(0.0,1.0,4); //
> init_bounded_number B1(minB1,maxB1,3); //initial biomass
> init_bounded_number K(minK,maxK,2); //aka B_infinitive
>
> init_number lnq(1);
>
> vector Bt(1,nyrs+1);
> vector adjBt(1,nyrs);
>
> number q;
> vector muI(1,nyrs);
>
> number varlogI;
>
> number MSY;
> number Bmsy;
> number Fmsy;
>
> number Bpen
>
> vector Ft(1,nyrs);
>
> objective_function_value f;
>
> PROCEDURE_SECTION
> f=0.0;
> Bpen=0;
> q=mfexp(lnq);
>
> //Project B_t
> Bt(1)=B1;
> for(i=1;i<=nyrs;i++) {
> Bt(i+1)=Bt(i)+r*Bt(i)*(1-Bt(i)/K)-Yt(i); //logistic (Schaefer)
> model
> Bt(i+1)=posfun(Bt(i+1),meanY*2/3,Bpen); //keeps it above
> mean(Yt)
>
> adjBt(i)=Bt(i)-Yt(i)/2; //Schnute
> adjBt(i)=posfun(adjBt(i),10,Bpen); //keeps it above 10
> }
> f+=Bpen;
> //cout<<"f2: "<<f<<endl;
> //cout<<"Bt: "<<Bt<<endl;
>
> //survey data
> muI=q*adjBt;
> varlogI=norm2(log(survey)-log(muI))/nyrs; //note the analytical
> form
>
> f+=0.5*nyrs*log(varlogI)+norm2(log(survey)-log(muI))/(2.0*varlogI);
> //cout<<"f3: "<<f<<endl;
>
> //management references
>
> MSY=r*K/4;
> Bmsy=K/2;
> Fmsy=r/2;
>
> Ft=elem_div(Yt(1,nyrs),Bt(1,nyrs));
>
>
> REPORT_SECTION
> report<<"yr Yt It Bt fitIt Ft Ft/Fmsy (Bt-Yt) varlogI f"<<endl;
> for(i=1;i<=nyrs;i++) {
> report<<yrs(i)<<" "<<Yt(i)<<" "<<survey(i)<<" "<<Bt(i)<<"
> "<<muI(i)<<" ";
> report<<Ft(i)<<" "<<Ft(i)/Fmsy<<" "<<Bt(i)-Yt(i)<<" "<<varlogI<<"
> "<<f<<endl;
> };
> report<<"MSY, Bmsy, Fmsy: "<<MSY<<" "<<Bmsy<<" "<<Fmsy<<endl;
> report<<"K range: "<<minK<<" "<<maxK<<endl;
> report<<"B1 range: "<<minB1<<" "<<maxB1<<endl;
>
>
> GLOBALS_SECTION
> #include <admodel.h>
> #include <math.h>
> #include <stdio.h>
> #include <stddef.h>
> #include <stdlib.h>
>
>
>
>
> On Oct 5, 2:40 am, Allan.Hi... at noaa.gov wrote:
>> Hi Saang-Yoon,
>>
>> I agree with Ian Taylor here. It would be useful to determine if the
>> difference is due to 64-bit vs 32-bit, ADMB 9 vs ADMB 10, or a
>> combination of both.
>>
>> Also, if the large differences are seen in the standard errors, maybe a
>> different starting point would result in a better estimate of the
>> Hessian, but you may have already tried this. If you use the .bar file
>> (-binp) as the starting point, and do not estimate the parameters, is
>> there still a large difference (basically trying to determine if the
>> standard errors are largely different at the same converged values)?
>>
>> If you can send me a tpl and dat file, I will see if I can replicate
>> your differences. Ian Taylor and I have been testing SS3 compiled with
>> 32-bit and 64-bit versions, as well as ADMB 9 and ADMB 10 and we have
>> not found significant differences. But, we should probably make sure
>> that we are checking the standard errors.
>>
>> Thanks for pointing this out, and hopefully we can solve this without
>> too much difficulty.
>>
>> Allan
>>
>>
>>
>>
>>
>> ----- Original Message -----
>> From: Saang-Yoon <shyu... at gmail.com>
>> Date: Tuesday, October 4, 2011 7:07 pm
>> Subject: Re: [ADMB Users] a difference in estimates between versions?
>>
>> > For example, the difference in estimate of the standard error of a
>> > parameter estimate is about in order of 10 (ten). I can name more
>> > examples. The difference magnitude is NOT about decimal points. The
>> > "64bit" ADMB is somewhat suspicious.
>>
>> > On Oct 4, 9:15 pm, Derek Seiple <dseipl... at gmail.com> wrote:
>> > > What do you mean by "small but obvious" can you give an example?
>>
>> > > If by small you mean they are the same out to say 7 decimal places,
>> > > this is to be somewhat expected. In moving from version 9 to version
>> > > 10 several of the mathematical functions had to be changes for
>> > > licensing reasons. Having slightly different algorithms could result
>> > > in slightly different round off, and as Ian said a slightly
>> > different> path through the likelihood surface. If that is the case
>> > then I'm not
>> > > sure you could say one is more reliable than the other, per say.
>>
>> > > If they are big changes, then that might be worth looking into.
>>
>> > > Derek
>>
>> > > On Tue, Oct 4, 2011 at 7:24 PM, Saang-Yoon <shyu... at gmail.com>
>> > wrote:> > Ian.
>> > > > Thank you very much for your prompt response. It is always
>> > nice to
>> > > > hear from acquaintace from a long distance. Getting back to the
>> > > > inquiry topic, then which version is more reliable? The
>> > difference> > is small but obvious. Especially the difference in
>> > estimates of
>> > > > 'standard errors' are more obvious.
>> > > > Saang-Yoon
>>
>> > > > On Oct 4, 7:09 pm, Ian Taylor <Ian.Tay... at NOAA.gov> wrote:
>> > > >> Hi Saang-Yoon,
>> > > >> It sounds from your email like you're changing versions AND
>> > changing> >> from 32bit to 64bit. I've found that 32 and 64 provide
>> > very slightly
>> > > >> different results, which I presume is due to small differences
>> > in the
>> > > >> precision of the derivative calculations and thus a different
>> > path> >> through the likelihood surface and a slightly different
>> > point when the
>> > > >> convergence criterion gets met.
>>
>> > > >> You might try comparing 32 to 32 between versions or 32 to 64
>> > within one
>> > > >> version to check whether this is a factor in the change you've
>> > observed.> >> -Ian
>>
>> > > >> On 10/04/2011 03:51 PM, Saang-Yoon wrote:
>>
>> > > >> > Hello, all.
>>
>> > > >> > I would like to report differences in estimates between the
>> > latest> >> > version (admb-ide-452-1 (64bit)) and the previous
>> > version (admb-
>> > > >> > ide-450-1). Today I found differences in estimates between
>> > versions> >> > from my class where I used the latest version and
>> > the other folks all
>> > > >> > used the previous version. Applying the same TPL and DAT
>> > files, my
>> > > >> > results (e.g., STD file) are slightly but obviously
>> > different from the
>> > > >> > others whose values are the same among them. In the two
>> > versions,> >> > estimation were all successful including Hessian
>> > matrix elements.
>>
>> > > >> > I wonder about why. More importantly I wonder which version is
>> > > >> > reliable.
>>
>> > > >> > If you are skeptical of our finding, then please apply a DAT
>> > file and
>> > > >> > a TPL file that has a "non" linear model to the different
>> > versions, or
>> > > >> > request me to send you the DAT and TPL files.
>>
>> > > >> > Thank you,
>>
>> > > >> > Saang-Yoon
>> > > >> > _______________________________________________
>> > > >> > Users mailing list
>> > > >> > Us... at admb-project.org
>> > > >> >http://lists.admb-project.org/mailman/listinfo/users
>>
>> > > >> _______________________________________________
>> > > >> Users mailing list
>> > > >> Us... at admb-project.orghttp://lists.admb-
>> > project.org/mailman/listinfo/users-Hide quoted text -
>>
>> > > >> - Show quoted text -
>> > > > _______________________________________________
>> > > > Users mailing list
>> > > > Us... at admb-project.org
>> > > >http://lists.admb-project.org/mailman/listinfo/users
>>
>> > > _______________________________________________
>> > > Users mailing list
>> > > Us... at admb-project.orghttp://lists.admb-
>> > project.org/mailman/listinfo/users- Hide quoted text -
>>
>> > > - Show quoted text -
>> > _______________________________________________
>> > Users mailing list
>> > Us... at admb-project.org
>> >http://lists.admb-project.org/mailman/listinfo/users
>>
>> _______________________________________________
>> Users mailing list
>> Us... at admb-project.orghttp://lists.admb-project.org/mailman/listinfo/users- Hide quoted text -
>>
>> - Show quoted text -
> _______________________________________________
> Users mailing list
> Users at admb-project.org
> http://lists.admb-project.org/mailman/listinfo/users
>
More information about the Users
mailing list