[ADMB Users] solid state drive and ADMB
Allan Hicks - NOAA Federal
allan.hicks at noaa.gov
Fri Oct 25 11:42:37 PDT 2013
Hi All,
We bought very fast desktops with 10000K RPM disks and fast laptops with
SSD's last year. The chip for the desktops was rated to perform
calculations faster than the chip in the laptops, so I expected SS (built
with ADMB) to run faster on the desktops. To my surprise, the example I
used, (which did not write any gradient files on either system) performed
slightly faster on the laptop with the SSD. I think that the reading of
data and writing of the Report file were big contributors to the time (the
difference was 1 or 2 seconds out of an overall 80 second run time). I did
not investigate further, but, it reminded me that the bottleneck of reading
and writing may be important, depending on your model.
My ideal desktop would be to have an SSD with the operating system and
programs installed on it, and extra space used for simulations and storage
needed to access quickly. I would then have a larger hard disk for general
storage (or more SSD's if cost wasn't an issue).
Just my thoughts,
Allan
On Sat, Sep 21, 2013 at 2:37 AM, CHRIS GRANDIN <cgrandin at shaw.ca> wrote:
> SSD's are far faster than hard disks due to no need for movement of a read
> head, but the main bottleneck is still the data bus. Only one thing can be
> written onto or read from the data bus at a given time, and the OS and its
> background programs are constantly using it as well as your running
> program. As Ian said, RAM is located in very close proximity to the CPUs,
> so most definitely fill up your RAM. You can still have disk thrashing
> occuring with a SSD if you don't get enough RAM, which is a situation you
> want to avoid at all costs!
>
> Chris
>
> ----- Original Message -----
> From: "John Sibert" <sibert at hawaii.edu>
> To: "Ian Taylor - NOAA Federal" <ian.taylor at noaa.gov>
> Cc: "users at admb-project.org Group" <users at admb-project.org>
> Sent: Friday, September 20, 2013 11:56:44 AM
> Subject: Re: [ADMB Users] solid state drive and ADMB
>
> RAM is cheap, and requires no thought. Figuring out which parts of the
> derivative chain to keep in RAM and which parts to write to storage is
> costly in terms of time, errors, and headache. Max out the RAM your
> machine, especially if you are developing multithreaded applications.
>
> John Sibert
> Emeritus Researcher, SOEST
> University of Hawaii at Manoa
> Honolulu HI (GMT-10)
> 808-294-3842
>
> Visit the ADMB project http://admb-project.org/
>
> On 09/20/2013 08:25 AM, Ian Taylor - NOAA Federal wrote:
> > I agree with the comments in favor of SSDs in general, but I'm not
> > surprised by John's finding that they don't magically solve the
> > problems of running out of RAM. I think the issue is that the
> > read/write speeds are slower than for RAM and the physical distance
> > from the CPU is greater (perhaps also with less throughput in that
> > connection). Allan Hicks knows more about this stuff but he's out to
> > sea right now. In theory, the partitioning of the calculations to have
> > some elements written to disk and others to RAM might also make the
> > computations less efficient, but probably only Dave could know whether
> > that's true.
> >
> >
> > On Fri, Sep 20, 2013 at 11:17 AM, John Sibert <sibert at hawaii.edu
> > <mailto:sibert at hawaii.edu>> wrote:
> >
> > Generally I love SSDs. My new laptop boots up from power on to
> > login screen in about 12 seconds. I/O is usually really fast, but
> > it is not nearly as fast as RAM. Below are timings to do 10
> > function evaluations with (1) writing temporary gradient data to
> > SSD and (2) keeping the the same data in RAM. It is probably
> > faster than writing to a spinning disk, but I no longer have a
> > computer with one of those to test it.
> >
> > (1) Writing to SSD
> > $ time ~/movemod/25/tpl/tagest -gbs 500000 -cbs 500000 -maxfn 10 -est
> > real 2m4.024s
> > user 1m38.994s
> > sys 0m23.509s
> >
> > -rwxrwxr-x 1 jsibert jsibert 62000000 Sep 20 08:04 cmpdiff.tmp*
> > -rwxrwxr-x 1 jsibert jsibert 175440000 Sep 20 08:04 gradfil1.tmp*
> > -rwxrwxr-x 1 jsibert jsibert 0 Sep 20 08:03 gradfil2.tmp*
> > -rwxrwxr-x 1 jsibert jsibert 0 Sep 20 08:03 varssave.tmp*
> >
> >
> > (2) retaining in RAM
> > $ time ~/movemod/25/tpl/tagest -maxfn 10 -est
> > real 1m35.423s
> > user 1m34.906s
> > sys 0m0.124s
> >
> > -rwxrwxr-x 1 jsibert jsibert 0 Sep 20 08:05 cmpdiff.tmp*
> > -rwxrwxr-x 1 jsibert jsibert 0 Sep 20 08:05 gradfil1.tmp*
> > -rwxrwxr-x 1 jsibert jsibert 0 Sep 20 08:05 gradfil2.tmp*
> > -rwxrwxr-x 1 jsibert jsibert 0 Sep 20 08:05 varssave.tmp*
> >
> >
> >
> > John Sibert
> > Emeritus Researcher, SOEST
> > University of Hawaii at Manoa
> > Honolulu HI (GMT-10)
> > 808-294-3842 <tel:808-294-3842>
> >
> > Visit the ADMB project http://admb-project.org/
> >
> > On 09/19/2013 03:35 PM, Mark Maunder wrote:
> >
> > I don't have an application on mind. Just buying some new
> > modeling machines and working out what is important.
> >
> > Regards,
> >
> > Mark
> > IATTC
> >
> > On Sep 19, 2013, at 5:36 PM, "John Sibert" <sibert at hawaii.edu
> > <mailto:sibert at hawaii.edu>> wrote:
> >
> > I'm not sure whether you are asking about swap memory
> > controlled by the operating system or to writing ADMB
> > gradient data to the disk. The former is almost always a
> > mistake. The latter would be interesting to test. I have
> > an old linux laptop with a brand new SSD. If you send me
> > your tpl and dat files, I could run some tests.
> > Cheers,
> > John
> >
> > John Sibert
> > Emeritus Researcher, SOEST
> > University of Hawaii at Manoa
> > Honolulu HI (GMT-10)
> > 808-294-3842 <tel:808-294-3842>
> >
> > Visit the ADMB project http://admb-project.org/
> >
> > On 09/19/2013 12:11 PM, Mark Maunder wrote:
> >
> > Has anyone tried to run a big ADMB model that swaps
> > memory to the hard drive with a solid state drive?
> > Does this perform reasonably or is it still really slow?
> >
> > *From:*users-bounces at admb-project.org
> > <mailto:users-bounces at admb-project.org>
> > [mailto:users-bounces at admb-project.org
> > <mailto:users-bounces at admb-project.org>] *On Behalf Of
> > *Mollie Brooks
> > *Sent:* Thursday, September 19, 2013 1:43 PM
> > *To:* users at admb-project.org
> > <mailto:users at admb-project.org> Group
> > *Subject:* [ADMB Users] dnbinom
> >
> > Hi ADMB users,
> >
> > Has anyone besides me used the dnbinom function in
> > their code?
> >
> > There's a little discussion about the possibility of
> > reparameterizing it, but we don't want to disrupt too
> > many people.
> >
> > Thanks,
> >
> > Mollie
> >
> > ------------------------
> >
> > Mollie Brooks, PhD
> > Postdoctoral Researcher, Population Ecology Research
> > Group http://www.popecol.org <http://www.popecol.org>
> > Institute of Evolutionary Biology & Environmental
> > Studies, University of Zürich
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > Users at admb-project.org <mailto:Users at admb-project.org>
> > http://lists.admb-project.org/mailman/listinfo/users
> >
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > Users at admb-project.org <mailto:Users at admb-project.org>
> > http://lists.admb-project.org/mailman/listinfo/users
> >
> >
>
>
> _______________________________________________
> Users mailing list
> Users at admb-project.org
> http://lists.admb-project.org/mailman/listinfo/users
> _______________________________________________
> Users mailing list
> Users at admb-project.org
> http://lists.admb-project.org/mailman/listinfo/users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.admb-project.org/pipermail/users/attachments/20131025/b7908be7/attachment.html>
More information about the Users
mailing list