[ADMB Users] a difference in estimates between versions?
Saang-Yoon
shyunuw at gmail.com
Wed Oct 5 05:41:05 PDT 2011
Allen, Derek, Ian and all.
As requested, I am showing an actual example. Please note obvious
differences between (1) and (2) below, esp. in SE of B1. The common
TPL file is shown below.
DAT file has long rows. I don't see here how to attach the file. If
you need DAT file, let me your email so that I can send it.
Not only difference in the results, but also I am surprised that these
two versions differently work in response to a small change of
parameter bounds. For example in the current TPL, setting the r
range as 0-1 works in both versions, but setting it as 0-2 does NOT
work in the 450-1. It is too soon to conclude, but my (limited)
experience let me prefer older versions prior to 450-1: e.g., 440-2.
Thank you very much for your response.
Saang-Yoon
(1) Results from previous version, ADMB IDE 450-1
index name value std dev
1 r 4.9970e-001 7.2940e-002
2 B1 8.9504e+004 1.3599e+004
3 K 9.1202e+004 7.9562e+003
4 lnq -2.1695e+000 1.9095e-001
(2) Results from the latest version, ADMB-IDE-452-1 (64bit)
index name value std dev
1 r 5.0509e-001 1.4047e-002
2 B1 8.8638e+004 2.9544e+003
3 K 9.0710e+004 1.3304e+003
4 lnq -2.1587e+000 1.1781e-001
========== The following TPL was used.=============
DATA_SECTION
int i; //for loop;
init_vector years(1,2);
int nyrs;
int ncol; //data column
!!nyrs=years(2)-years(1)+1;
!!ncol=3;
init_matrix yieldsurvey(1,nyrs,1,ncol);
vector yrs(1,nyrs);
vector Yt(1,nyrs);
vector survey(1,nyrs); //survey data
!!yrs=trans(yieldsurvey)(1);
!!Yt=trans(yieldsurvey)(2); //yield;
!!survey=trans(yieldsurvey)(3);
number minB1;
number maxB1;
!!minB1=Yt(1)-Yt(1)*15/100;
!!maxB1=minB1+minB1*3000/100;
//!!maxB1=sum(Yt);
number minK;
number maxK;
!!minK=max(Yt)-max(Yt)*15/100;
!!maxK=minK+minK*3000/100;
//!!maxK=sum(Yt)*10;
number minY;
number meanY;
!!minY=min(Yt);
!!meanY=mean(Yt);
!!cout<<"yrs: "<<yrs<<endl;
!!cout<<"Yt: "<<Yt<<endl;
!!cout<<"survey: "<<survey<<endl;
PARAMETER_SECTION
init_bounded_number r(0.0,1.0,4); //
init_bounded_number B1(minB1,maxB1,3); //initial biomass
init_bounded_number K(minK,maxK,2); //aka B_infinitive
init_number lnq(1);
vector Bt(1,nyrs+1);
vector adjBt(1,nyrs);
number q;
vector muI(1,nyrs);
number varlogI;
number MSY;
number Bmsy;
number Fmsy;
number Bpen
vector Ft(1,nyrs);
objective_function_value f;
PROCEDURE_SECTION
f=0.0;
Bpen=0;
q=mfexp(lnq);
//Project B_t
Bt(1)=B1;
for(i=1;i<=nyrs;i++) {
Bt(i+1)=Bt(i)+r*Bt(i)*(1-Bt(i)/K)-Yt(i); //logistic (Schaefer)
model
Bt(i+1)=posfun(Bt(i+1),meanY*2/3,Bpen); //keeps it above
mean(Yt)
adjBt(i)=Bt(i)-Yt(i)/2; //Schnute
adjBt(i)=posfun(adjBt(i),10,Bpen); //keeps it above 10
}
f+=Bpen;
//cout<<"f2: "<<f<<endl;
//cout<<"Bt: "<<Bt<<endl;
//survey data
muI=q*adjBt;
varlogI=norm2(log(survey)-log(muI))/nyrs; //note the analytical
form
f+=0.5*nyrs*log(varlogI)+norm2(log(survey)-log(muI))/(2.0*varlogI);
//cout<<"f3: "<<f<<endl;
//management references
MSY=r*K/4;
Bmsy=K/2;
Fmsy=r/2;
Ft=elem_div(Yt(1,nyrs),Bt(1,nyrs));
REPORT_SECTION
report<<"yr Yt It Bt fitIt Ft Ft/Fmsy (Bt-Yt) varlogI f"<<endl;
for(i=1;i<=nyrs;i++) {
report<<yrs(i)<<" "<<Yt(i)<<" "<<survey(i)<<" "<<Bt(i)<<"
"<<muI(i)<<" ";
report<<Ft(i)<<" "<<Ft(i)/Fmsy<<" "<<Bt(i)-Yt(i)<<" "<<varlogI<<"
"<<f<<endl;
};
report<<"MSY, Bmsy, Fmsy: "<<MSY<<" "<<Bmsy<<" "<<Fmsy<<endl;
report<<"K range: "<<minK<<" "<<maxK<<endl;
report<<"B1 range: "<<minB1<<" "<<maxB1<<endl;
GLOBALS_SECTION
#include <admodel.h>
#include <math.h>
#include <stdio.h>
#include <stddef.h>
#include <stdlib.h>
On Oct 5, 2:40 am, Allan.Hi... at noaa.gov wrote:
> Hi Saang-Yoon,
>
> I agree with Ian Taylor here. It would be useful to determine if the
> difference is due to 64-bit vs 32-bit, ADMB 9 vs ADMB 10, or a
> combination of both.
>
> Also, if the large differences are seen in the standard errors, maybe a
> different starting point would result in a better estimate of the
> Hessian, but you may have already tried this. If you use the .bar file
> (-binp) as the starting point, and do not estimate the parameters, is
> there still a large difference (basically trying to determine if the
> standard errors are largely different at the same converged values)?
>
> If you can send me a tpl and dat file, I will see if I can replicate
> your differences. Ian Taylor and I have been testing SS3 compiled with
> 32-bit and 64-bit versions, as well as ADMB 9 and ADMB 10 and we have
> not found significant differences. But, we should probably make sure
> that we are checking the standard errors.
>
> Thanks for pointing this out, and hopefully we can solve this without
> too much difficulty.
>
> Allan
>
>
>
>
>
> ----- Original Message -----
> From: Saang-Yoon <shyu... at gmail.com>
> Date: Tuesday, October 4, 2011 7:07 pm
> Subject: Re: [ADMB Users] a difference in estimates between versions?
>
> > For example, the difference in estimate of the standard error of a
> > parameter estimate is about in order of 10 (ten). I can name more
> > examples. The difference magnitude is NOT about decimal points. The
> > "64bit" ADMB is somewhat suspicious.
>
> > On Oct 4, 9:15 pm, Derek Seiple <dseipl... at gmail.com> wrote:
> > > What do you mean by "small but obvious" can you give an example?
>
> > > If by small you mean they are the same out to say 7 decimal places,
> > > this is to be somewhat expected. In moving from version 9 to version
> > > 10 several of the mathematical functions had to be changes for
> > > licensing reasons. Having slightly different algorithms could result
> > > in slightly different round off, and as Ian said a slightly
> > different> path through the likelihood surface. If that is the case
> > then I'm not
> > > sure you could say one is more reliable than the other, per say.
>
> > > If they are big changes, then that might be worth looking into.
>
> > > Derek
>
> > > On Tue, Oct 4, 2011 at 7:24 PM, Saang-Yoon <shyu... at gmail.com>
> > wrote:> > Ian.
> > > > Thank you very much for your prompt response. It is always
> > nice to
> > > > hear from acquaintace from a long distance. Getting back to the
> > > > inquiry topic, then which version is more reliable? The
> > difference> > is small but obvious. Especially the difference in
> > estimates of
> > > > 'standard errors' are more obvious.
> > > > Saang-Yoon
>
> > > > On Oct 4, 7:09 pm, Ian Taylor <Ian.Tay... at NOAA.gov> wrote:
> > > >> Hi Saang-Yoon,
> > > >> It sounds from your email like you're changing versions AND
> > changing> >> from 32bit to 64bit. I've found that 32 and 64 provide
> > very slightly
> > > >> different results, which I presume is due to small differences
> > in the
> > > >> precision of the derivative calculations and thus a different
> > path> >> through the likelihood surface and a slightly different
> > point when the
> > > >> convergence criterion gets met.
>
> > > >> You might try comparing 32 to 32 between versions or 32 to 64
> > within one
> > > >> version to check whether this is a factor in the change you've
> > observed.> >> -Ian
>
> > > >> On 10/04/2011 03:51 PM, Saang-Yoon wrote:
>
> > > >> > Hello, all.
>
> > > >> > I would like to report differences in estimates between the
> > latest> >> > version (admb-ide-452-1 (64bit)) and the previous
> > version (admb-
> > > >> > ide-450-1). Today I found differences in estimates between
> > versions> >> > from my class where I used the latest version and
> > the other folks all
> > > >> > used the previous version. Applying the same TPL and DAT
> > files, my
> > > >> > results (e.g., STD file) are slightly but obviously
> > different from the
> > > >> > others whose values are the same among them. In the two
> > versions,> >> > estimation were all successful including Hessian
> > matrix elements.
>
> > > >> > I wonder about why. More importantly I wonder which version is
> > > >> > reliable.
>
> > > >> > If you are skeptical of our finding, then please apply a DAT
> > file and
> > > >> > a TPL file that has a "non" linear model to the different
> > versions, or
> > > >> > request me to send you the DAT and TPL files.
>
> > > >> > Thank you,
>
> > > >> > Saang-Yoon
> > > >> > _______________________________________________
> > > >> > Users mailing list
> > > >> > Us... at admb-project.org
> > > >> >http://lists.admb-project.org/mailman/listinfo/users
>
> > > >> _______________________________________________
> > > >> Users mailing list
> > > >> Us... at admb-project.orghttp://lists.admb-
> > project.org/mailman/listinfo/users-Hide quoted text -
>
> > > >> - Show quoted text -
> > > > _______________________________________________
> > > > Users mailing list
> > > > Us... at admb-project.org
> > > >http://lists.admb-project.org/mailman/listinfo/users
>
> > > _______________________________________________
> > > Users mailing list
> > > Us... at admb-project.orghttp://lists.admb-
> > project.org/mailman/listinfo/users- Hide quoted text -
>
> > > - Show quoted text -
> > _______________________________________________
> > Users mailing list
> > Us... at admb-project.org
> >http://lists.admb-project.org/mailman/listinfo/users
>
> _______________________________________________
> Users mailing list
> Us... at admb-project.orghttp://lists.admb-project.org/mailman/listinfo/users- Hide quoted text -
>
> - Show quoted text -
More information about the Users
mailing list