[ADMB Users] solid state drive and ADMB
CHRIS GRANDIN
cgrandin at shaw.ca
Sat Sep 21 02:37:11 PDT 2013
SSD's are far faster than hard disks due to no need for movement of a read head, but the main bottleneck is still the data bus. Only one thing can be written onto or read from the data bus at a given time, and the OS and its background programs are constantly using it as well as your running program. As Ian said, RAM is located in very close proximity to the CPUs, so most definitely fill up your RAM. You can still have disk thrashing occuring with a SSD if you don't get enough RAM, which is a situation you want to avoid at all costs!
Chris
----- Original Message -----
From: "John Sibert" <sibert at hawaii.edu>
To: "Ian Taylor - NOAA Federal" <ian.taylor at noaa.gov>
Cc: "users at admb-project.org Group" <users at admb-project.org>
Sent: Friday, September 20, 2013 11:56:44 AM
Subject: Re: [ADMB Users] solid state drive and ADMB
RAM is cheap, and requires no thought. Figuring out which parts of the
derivative chain to keep in RAM and which parts to write to storage is
costly in terms of time, errors, and headache. Max out the RAM your
machine, especially if you are developing multithreaded applications.
John Sibert
Emeritus Researcher, SOEST
University of Hawaii at Manoa
Honolulu HI (GMT-10)
808-294-3842
Visit the ADMB project http://admb-project.org/
On 09/20/2013 08:25 AM, Ian Taylor - NOAA Federal wrote:
> I agree with the comments in favor of SSDs in general, but I'm not
> surprised by John's finding that they don't magically solve the
> problems of running out of RAM. I think the issue is that the
> read/write speeds are slower than for RAM and the physical distance
> from the CPU is greater (perhaps also with less throughput in that
> connection). Allan Hicks knows more about this stuff but he's out to
> sea right now. In theory, the partitioning of the calculations to have
> some elements written to disk and others to RAM might also make the
> computations less efficient, but probably only Dave could know whether
> that's true.
>
>
> On Fri, Sep 20, 2013 at 11:17 AM, John Sibert <sibert at hawaii.edu
> <mailto:sibert at hawaii.edu>> wrote:
>
> Generally I love SSDs. My new laptop boots up from power on to
> login screen in about 12 seconds. I/O is usually really fast, but
> it is not nearly as fast as RAM. Below are timings to do 10
> function evaluations with (1) writing temporary gradient data to
> SSD and (2) keeping the the same data in RAM. It is probably
> faster than writing to a spinning disk, but I no longer have a
> computer with one of those to test it.
>
> (1) Writing to SSD
> $ time ~/movemod/25/tpl/tagest -gbs 500000 -cbs 500000 -maxfn 10 -est
> real 2m4.024s
> user 1m38.994s
> sys 0m23.509s
>
> -rwxrwxr-x 1 jsibert jsibert 62000000 Sep 20 08:04 cmpdiff.tmp*
> -rwxrwxr-x 1 jsibert jsibert 175440000 Sep 20 08:04 gradfil1.tmp*
> -rwxrwxr-x 1 jsibert jsibert 0 Sep 20 08:03 gradfil2.tmp*
> -rwxrwxr-x 1 jsibert jsibert 0 Sep 20 08:03 varssave.tmp*
>
>
> (2) retaining in RAM
> $ time ~/movemod/25/tpl/tagest -maxfn 10 -est
> real 1m35.423s
> user 1m34.906s
> sys 0m0.124s
>
> -rwxrwxr-x 1 jsibert jsibert 0 Sep 20 08:05 cmpdiff.tmp*
> -rwxrwxr-x 1 jsibert jsibert 0 Sep 20 08:05 gradfil1.tmp*
> -rwxrwxr-x 1 jsibert jsibert 0 Sep 20 08:05 gradfil2.tmp*
> -rwxrwxr-x 1 jsibert jsibert 0 Sep 20 08:05 varssave.tmp*
>
>
>
> John Sibert
> Emeritus Researcher, SOEST
> University of Hawaii at Manoa
> Honolulu HI (GMT-10)
> 808-294-3842 <tel:808-294-3842>
>
> Visit the ADMB project http://admb-project.org/
>
> On 09/19/2013 03:35 PM, Mark Maunder wrote:
>
> I don't have an application on mind. Just buying some new
> modeling machines and working out what is important.
>
> Regards,
>
> Mark
> IATTC
>
> On Sep 19, 2013, at 5:36 PM, "John Sibert" <sibert at hawaii.edu
> <mailto:sibert at hawaii.edu>> wrote:
>
> I'm not sure whether you are asking about swap memory
> controlled by the operating system or to writing ADMB
> gradient data to the disk. The former is almost always a
> mistake. The latter would be interesting to test. I have
> an old linux laptop with a brand new SSD. If you send me
> your tpl and dat files, I could run some tests.
> Cheers,
> John
>
> John Sibert
> Emeritus Researcher, SOEST
> University of Hawaii at Manoa
> Honolulu HI (GMT-10)
> 808-294-3842 <tel:808-294-3842>
>
> Visit the ADMB project http://admb-project.org/
>
> On 09/19/2013 12:11 PM, Mark Maunder wrote:
>
> Has anyone tried to run a big ADMB model that swaps
> memory to the hard drive with a solid state drive?
> Does this perform reasonably or is it still really slow?
>
> *From:*users-bounces at admb-project.org
> <mailto:users-bounces at admb-project.org>
> [mailto:users-bounces at admb-project.org
> <mailto:users-bounces at admb-project.org>] *On Behalf Of
> *Mollie Brooks
> *Sent:* Thursday, September 19, 2013 1:43 PM
> *To:* users at admb-project.org
> <mailto:users at admb-project.org> Group
> *Subject:* [ADMB Users] dnbinom
>
> Hi ADMB users,
>
> Has anyone besides me used the dnbinom function in
> their code?
>
> There's a little discussion about the possibility of
> reparameterizing it, but we don't want to disrupt too
> many people.
>
> Thanks,
>
> Mollie
>
> ------------------------
>
> Mollie Brooks, PhD
> Postdoctoral Researcher, Population Ecology Research
> Group http://www.popecol.org <http://www.popecol.org>
> Institute of Evolutionary Biology & Environmental
> Studies, University of Zürich
>
>
>
> _______________________________________________
> Users mailing list
> Users at admb-project.org <mailto:Users at admb-project.org>
> http://lists.admb-project.org/mailman/listinfo/users
>
>
>
>
> _______________________________________________
> Users mailing list
> Users at admb-project.org <mailto:Users at admb-project.org>
> http://lists.admb-project.org/mailman/listinfo/users
>
>
_______________________________________________
Users mailing list
Users at admb-project.org
http://lists.admb-project.org/mailman/listinfo/users
More information about the Users
mailing list