[ADMB Users] a difference in estimates between versions?

[Allan.Hicks at noaa.gov]

Hi Saang-Yoon,

I agree with Ian Taylor here. It would be useful to determine if the
difference is due to 64-bit vs 32-bit, ADMB 9 vs ADMB 10, or a
combination of both.

Also, if the large differences are seen in the standard errors, maybe a
different starting point would result in a better estimate of the
Hessian, but you may have already tried this.  If you use the .bar file
(-binp) as the starting point, and do not estimate the parameters, is
there still a large difference (basically trying to determine if the
standard errors are largely different at the same converged values)?

If you can send me a tpl and dat file, I will see if I can replicate
your differences.  Ian Taylor and I have been testing SS3 compiled with
32-bit and 64-bit versions, as well as ADMB 9 and ADMB 10 and we have
not found significant differences. But, we should probably make sure
that we are checking the standard errors.

Thanks for pointing this out, and hopefully we can solve this without
too much difficulty.

Allan



----- Original Message -----
From: Saang-Yoon <shyunuw at gmail.com>
Date: Tuesday, October 4, 2011 7:07 pm
Subject: Re: [ADMB Users] a difference in estimates between versions?

> For example, the difference in estimate of the standard error of a
> parameter estimate is about  in order of 10 (ten).   I can name more
> examples.  The difference magnitude is NOT about decimal points.   The
> "64bit" ADMB is somewhat suspicious.
> 
> 
> On Oct 4, 9:15 pm, Derek Seiple <dseipl... at gmail.com> wrote:
> > What do you mean by "small but obvious" can you give an example?
> >
> > If by small you mean they are the same out to say 7 decimal places,
> > this is to be somewhat expected. In moving from version 9 to version
> > 10 several of the mathematical functions had to be changes for
> > licensing reasons. Having slightly different algorithms could result
> > in slightly different round off, and as Ian said a slightly 
> different> path through the likelihood surface. If that is the case 
> then I'm not
> > sure you could say one is more reliable than the other, per say.
> >
> > If they are big changes, then that might be worth looking into.
> >
> > Derek
> >
> >
> >
> >
> >
> > On Tue, Oct 4, 2011 at 7:24 PM, Saang-Yoon <shyu... at gmail.com> 
> wrote:> > Ian.
> > > Thank you very much for your prompt response.  It is always 
> nice to
> > > hear from acquaintace from a long distance.   Getting back to the
> > > inquiry topic, then which version is more reliable?   The 
> difference> > is small but obvious.  Especially the difference in 
> estimates of
> > > 'standard errors' are more obvious.
> > > Saang-Yoon
> >
> > > On Oct 4, 7:09 pm, Ian Taylor <Ian.Tay... at NOAA.gov> wrote:
> > >> Hi Saang-Yoon,
> > >> It sounds from your email like you're changing versions AND 
> changing> >> from 32bit to 64bit. I've found that 32 and 64 provide 
> very slightly
> > >> different results, which I presume is due to small differences 
> in the
> > >> precision of the derivative calculations and thus a different 
> path> >> through the likelihood surface and a slightly different 
> point when the
> > >> convergence criterion gets met.
> >
> > >> You might try comparing 32 to 32 between versions or 32 to 64 
> within one
> > >> version to check whether this is a factor in the change you've 
> observed.> >> -Ian
> >
> > >> On 10/04/2011 03:51 PM, Saang-Yoon wrote:
> >
> > >> > Hello, all.
> >
> > >> > I would like to report differences in estimates between the 
> latest> >> > version (admb-ide-452-1 (64bit)) and the previous 
> version (admb-
> > >> > ide-450-1).   Today I found differences in estimates between 
> versions> >> > from my class where I used the latest version and 
> the other folks all
> > >> > used the previous version.  Applying the same TPL and DAT 
> files, my
> > >> > results (e.g., STD file) are slightly but obviously 
> different from the
> > >> > others whose values are the same among them.   In the two 
> versions,> >> > estimation were all successful including Hessian 
> matrix elements.
> >
> > >> > I wonder about why.  More importantly I wonder which version is
> > >> > reliable.
> >
> > >> > If you are skeptical of our finding, then please apply a DAT 
> file and
> > >> > a TPL file that has a "non" linear model to the different 
> versions, or
> > >> > request me to send you the DAT and TPL files.
> >
> > >> > Thank you,
> >
> > >> > Saang-Yoon
> > >> > _______________________________________________
> > >> > Users mailing list
> > >> > Us... at admb-project.org
> > >> >http://lists.admb-project.org/mailman/listinfo/users
> >
> > >> _______________________________________________
> > >> Users mailing list
> > >> Us... at admb-project.orghttp://lists.admb-
> project.org/mailman/listinfo/users-Hide quoted text -
> >
> > >> - Show quoted text -
> > > _______________________________________________
> > > Users mailing list
> > > Us... at admb-project.org
> > >http://lists.admb-project.org/mailman/listinfo/users
> >
> > _______________________________________________
> > Users mailing list
> > Us... at admb-project.orghttp://lists.admb-
> project.org/mailman/listinfo/users- Hide quoted text -
> >
> > - Show quoted text -
> _______________________________________________
> Users mailing list
> Users at admb-project.org
> http://lists.admb-project.org/mailman/listinfo/users
>