[ADMB Users] a difference in estimates between versions?

Saang-Yoon shyunuw at gmail.com
Wed Oct 5 05:41:05 PDT 2011


Allen, Derek, Ian and all.

As requested, I am showing an actual example.  Please note obvious
differences between (1) and (2) below, esp. in SE of B1.  The common
TPL file is shown below.
DAT file has long rows.  I don't see here how to attach the file.  If
you need DAT file, let me your email so that I can send it.

Not only difference in the results, but also I am surprised that these
two versions differently work in response to a small change of
parameter bounds.   For example  in the current TPL, setting the r
range as 0-1 works in both versions, but setting it as 0-2 does NOT
work in the 450-1.   It is too soon to conclude, but my (limited)
experience let me prefer older versions prior to 450-1: e.g., 440-2.
Thank you very much for your response.

Saang-Yoon


(1) Results from previous version, ADMB IDE 450-1

 index   name   value      std dev
     1   r   4.9970e-001 7.2940e-002
     2   B1  8.9504e+004 1.3599e+004
     3   K   9.1202e+004 7.9562e+003
     4   lnq -2.1695e+000 1.9095e-001


(2) Results from the latest version, ADMB-IDE-452-1 (64bit)
 index   name   value      std dev
     1   r   5.0509e-001 1.4047e-002
     2   B1  8.8638e+004 2.9544e+003
     3   K   9.0710e+004 1.3304e+003
     4   lnq -2.1587e+000 1.1781e-001


========== The following TPL was used.=============

DATA_SECTION
  int i;  //for loop;

  init_vector years(1,2);
  int nyrs;
  int ncol;  //data column
  !!nyrs=years(2)-years(1)+1;
  !!ncol=3;

  init_matrix yieldsurvey(1,nyrs,1,ncol);
  vector yrs(1,nyrs);
  vector Yt(1,nyrs);
  vector survey(1,nyrs);  //survey data

  !!yrs=trans(yieldsurvey)(1);
  !!Yt=trans(yieldsurvey)(2);  //yield;
  !!survey=trans(yieldsurvey)(3);

  number minB1;
  number maxB1;
  !!minB1=Yt(1)-Yt(1)*15/100;
  !!maxB1=minB1+minB1*3000/100;
  //!!maxB1=sum(Yt);

  number minK;
  number maxK;
  !!minK=max(Yt)-max(Yt)*15/100;
  !!maxK=minK+minK*3000/100;
  //!!maxK=sum(Yt)*10;

  number minY;
  number meanY;
  !!minY=min(Yt);
  !!meanY=mean(Yt);

  !!cout<<"yrs: "<<yrs<<endl;
  !!cout<<"Yt: "<<Yt<<endl;
  !!cout<<"survey: "<<survey<<endl;

PARAMETER_SECTION
  init_bounded_number r(0.0,1.0,4);         //
  init_bounded_number B1(minB1,maxB1,3);   //initial biomass
  init_bounded_number K(minK,maxK,2);      //aka B_infinitive

  init_number lnq(1);

  vector Bt(1,nyrs+1);
  vector adjBt(1,nyrs);

  number q;
  vector muI(1,nyrs);

  number varlogI;

  number MSY;
  number Bmsy;
  number Fmsy;

  number Bpen

  vector Ft(1,nyrs);

  objective_function_value f;

PROCEDURE_SECTION
  f=0.0;
  Bpen=0;
  q=mfexp(lnq);

  //Project B_t
  Bt(1)=B1;
  for(i=1;i<=nyrs;i++)  {
	 Bt(i+1)=Bt(i)+r*Bt(i)*(1-Bt(i)/K)-Yt(i);  //logistic (Schaefer)
model
     Bt(i+1)=posfun(Bt(i+1),meanY*2/3,Bpen);  //keeps it above
mean(Yt)

     adjBt(i)=Bt(i)-Yt(i)/2;           //Schnute
     adjBt(i)=posfun(adjBt(i),10,Bpen); //keeps it above 10
     }
  f+=Bpen;
  //cout<<"f2: "<<f<<endl;
  //cout<<"Bt: "<<Bt<<endl;

  //survey data
  muI=q*adjBt;
  varlogI=norm2(log(survey)-log(muI))/nyrs;  //note the analytical
form

  f+=0.5*nyrs*log(varlogI)+norm2(log(survey)-log(muI))/(2.0*varlogI);
  //cout<<"f3: "<<f<<endl;

  //management references

  MSY=r*K/4;
  Bmsy=K/2;
  Fmsy=r/2;

  Ft=elem_div(Yt(1,nyrs),Bt(1,nyrs));


REPORT_SECTION
  report<<"yr Yt It Bt fitIt Ft Ft/Fmsy (Bt-Yt) varlogI f"<<endl;
  for(i=1;i<=nyrs;i++)  {
    report<<yrs(i)<<" "<<Yt(i)<<" "<<survey(i)<<" "<<Bt(i)<<"
"<<muI(i)<<" ";
    report<<Ft(i)<<" "<<Ft(i)/Fmsy<<" "<<Bt(i)-Yt(i)<<" "<<varlogI<<"
"<<f<<endl;
  };
  report<<"MSY, Bmsy, Fmsy: "<<MSY<<" "<<Bmsy<<" "<<Fmsy<<endl;
  report<<"K range: "<<minK<<" "<<maxK<<endl;
  report<<"B1 range: "<<minB1<<" "<<maxB1<<endl;


GLOBALS_SECTION
  #include <admodel.h>
  #include <math.h>
  #include <stdio.h>
  #include <stddef.h>
  #include <stdlib.h>




On Oct 5, 2:40 am, Allan.Hi... at noaa.gov wrote:
> Hi Saang-Yoon,
>
> I agree with Ian Taylor here. It would be useful to determine if the
> difference is due to 64-bit vs 32-bit, ADMB 9 vs ADMB 10, or a
> combination of both.
>
> Also, if the large differences are seen in the standard errors, maybe a
> different starting point would result in a better estimate of the
> Hessian, but you may have already tried this.  If you use the .bar file
> (-binp) as the starting point, and do not estimate the parameters, is
> there still a large difference (basically trying to determine if the
> standard errors are largely different at the same converged values)?
>
> If you can send me a tpl and dat file, I will see if I can replicate
> your differences.  Ian Taylor and I have been testing SS3 compiled with
> 32-bit and 64-bit versions, as well as ADMB 9 and ADMB 10 and we have
> not found significant differences. But, we should probably make sure
> that we are checking the standard errors.
>
> Thanks for pointing this out, and hopefully we can solve this without
> too much difficulty.
>
> Allan
>
>
>
>
>
> ----- Original Message -----
> From: Saang-Yoon <shyu... at gmail.com>
> Date: Tuesday, October 4, 2011 7:07 pm
> Subject: Re: [ADMB Users] a difference in estimates between versions?
>
> > For example, the difference in estimate of the standard error of a
> > parameter estimate is about  in order of 10 (ten).   I can name more
> > examples.  The difference magnitude is NOT about decimal points.   The
> > "64bit" ADMB is somewhat suspicious.
>
> > On Oct 4, 9:15 pm, Derek Seiple <dseipl... at gmail.com> wrote:
> > > What do you mean by "small but obvious" can you give an example?
>
> > > If by small you mean they are the same out to say 7 decimal places,
> > > this is to be somewhat expected. In moving from version 9 to version
> > > 10 several of the mathematical functions had to be changes for
> > > licensing reasons. Having slightly different algorithms could result
> > > in slightly different round off, and as Ian said a slightly
> > different> path through the likelihood surface. If that is the case
> > then I'm not
> > > sure you could say one is more reliable than the other, per say.
>
> > > If they are big changes, then that might be worth looking into.
>
> > > Derek
>
> > > On Tue, Oct 4, 2011 at 7:24 PM, Saang-Yoon <shyu... at gmail.com>
> > wrote:> > Ian.
> > > > Thank you very much for your prompt response.  It is always
> > nice to
> > > > hear from acquaintace from a long distance.   Getting back to the
> > > > inquiry topic, then which version is more reliable?   The
> > difference> > is small but obvious.  Especially the difference in
> > estimates of
> > > > 'standard errors' are more obvious.
> > > > Saang-Yoon
>
> > > > On Oct 4, 7:09 pm, Ian Taylor <Ian.Tay... at NOAA.gov> wrote:
> > > >> Hi Saang-Yoon,
> > > >> It sounds from your email like you're changing versions AND
> > changing> >> from 32bit to 64bit. I've found that 32 and 64 provide
> > very slightly
> > > >> different results, which I presume is due to small differences
> > in the
> > > >> precision of the derivative calculations and thus a different
> > path> >> through the likelihood surface and a slightly different
> > point when the
> > > >> convergence criterion gets met.
>
> > > >> You might try comparing 32 to 32 between versions or 32 to 64
> > within one
> > > >> version to check whether this is a factor in the change you've
> > observed.> >> -Ian
>
> > > >> On 10/04/2011 03:51 PM, Saang-Yoon wrote:
>
> > > >> > Hello, all.
>
> > > >> > I would like to report differences in estimates between the
> > latest> >> > version (admb-ide-452-1 (64bit)) and the previous
> > version (admb-
> > > >> > ide-450-1).   Today I found differences in estimates between
> > versions> >> > from my class where I used the latest version and
> > the other folks all
> > > >> > used the previous version.  Applying the same TPL and DAT
> > files, my
> > > >> > results (e.g., STD file) are slightly but obviously
> > different from the
> > > >> > others whose values are the same among them.   In the two
> > versions,> >> > estimation were all successful including Hessian
> > matrix elements.
>
> > > >> > I wonder about why.  More importantly I wonder which version is
> > > >> > reliable.
>
> > > >> > If you are skeptical of our finding, then please apply a DAT
> > file and
> > > >> > a TPL file that has a "non" linear model to the different
> > versions, or
> > > >> > request me to send you the DAT and TPL files.
>
> > > >> > Thank you,
>
> > > >> > Saang-Yoon
> > > >> > _______________________________________________
> > > >> > Users mailing list
> > > >> > Us... at admb-project.org
> > > >> >http://lists.admb-project.org/mailman/listinfo/users
>
> > > >> _______________________________________________
> > > >> Users mailing list
> > > >> Us... at admb-project.orghttp://lists.admb-
> > project.org/mailman/listinfo/users-Hide quoted text -
>
> > > >> - Show quoted text -
> > > > _______________________________________________
> > > > Users mailing list
> > > > Us... at admb-project.org
> > > >http://lists.admb-project.org/mailman/listinfo/users
>
> > > _______________________________________________
> > > Users mailing list
> > > Us... at admb-project.orghttp://lists.admb-
> > project.org/mailman/listinfo/users- Hide quoted text -
>
> > > - Show quoted text -
> > _______________________________________________
> > Users mailing list
> > Us... at admb-project.org
> >http://lists.admb-project.org/mailman/listinfo/users
>
> _______________________________________________
> Users mailing list
> Us... at admb-project.orghttp://lists.admb-project.org/mailman/listinfo/users- Hide quoted text -
>
> - Show quoted text -



More information about the Users mailing list