[Developers] How to build admb source with mingw 64 - finally!

Tue Dec 10 07:40:04 PST 2013

Thanks for your feedback Wade, I am going to try the msys version Jon pointed out and perhaps we should adopt that as a starting point for 64 bit mingw for the admb project.
C

-----Original Message-----
From: Cooper, Wade [mailto:Wade.Cooper at MyFWC.com]
Sent: Tue 12/10/2013 6:17 AM
To: Grandin, Chris; Jon Schnute; John Sibert; Johnoel Ancheta; developers at admb-project.org
Cc: Haigh, Rowan
Subject: RE: How to build admb source with mingw 64 - finally!

Good to hear it's up and running!

Regarding msys and Jon's suggestion, I've been using a slightly older version of the build that Jon recommended (rev11 vs 13; below) and that has worked straight out of the box without any need to copy the find.exe.

http://sourceforge.net/projects/mingwbuilds/files/external-binary-packages/msys+7za+wget+svn+git+mercurial+cvs-rev11.7z

Wade

From: Grandin, Chris [mailto:Chris.Grandin at dfo-mpo.gc.ca]
Sent: Tuesday, December 10, 2013 8:26 AM
To: Jon Schnute; Cooper, Wade; John Sibert; Johnoel Ancheta; developers at admb-project.org
Cc: Haigh, Rowan
Subject: RE: How to build admb source with mingw 64 - finally!

The 64-bit debug build also works perfectly. The 64-bit gdb.exe debugger is included with mingw64 and works with the emacs IDE. Whichever file we use for msys 64 is fine with me as long as it works!

Arni had a problem with us using MSYS, I'm not sure what we can do about that but I think it is more important to evolve the software to 64-bit than to shelter some people from this.

Jon - I have always used the shell because it ensures your commands are the msys/mingw ones. For example windows has a find.exe command, and you don't want your build to use this, you want it to use the mingw find.exe. The source may build how it currently stands in DOS but remember that some future changes could break this.

If you want to try making the debug build, just do 'make debug' instead of 'make' in the instructions I sent out.

Cheers
Chris

-----Original Message-----
From: Jon Schnute [mailto:schnutej-dfo at shaw.ca]
Sent: Tue 12/10/2013 1:42 AM
To: Grandin, Chris; 'Cooper, Wade'; 'John Sibert'; 'Johnoel Ancheta'; developers at admb-project.org<mailto:developers at admb-project.org>
Cc: Haigh, Rowan
Subject: RE: How to build admb source with mingw 64 - finally!

Chris and Wade - This works for me too. Fantastic work!

However, I still think that we should use the file
"msys+7za+wget+svn+git+mercurial+cvs-rev13.7z" at

http://sourceforge.net/projects/mingwbuilds/files/external-binary-packages/

rather than "MSYS-20111123.zip" at

http://sourceforge.net/projects/mingw-w64/files/External%20binary%20packages
%20(Win64%20hosted)/MSYS%20(32-bit)/

Both of them have been built as part of the new MinGW-w64 project, but the
first one isn't missing the file "find.exe" and it also contains flex,
bison, and a number of other useful utilities. Chris, I too read that MSYS
will always be 32-bit. As I understand it, the compilations we're talking
about have come from the new MinGW-w64 project, rather than the original
MinGW project. However, MSYS itself is still 32-bit.

I strongly encourage everyone to look at Rioki's Corner
(http://www.rioki.org/2013/10/16/mingw-w64.html), which explicitly mentions
the first web site (mingwbuilds) above. When I saw that Rioki's choice
matched the one that Wade had already used, I thought I could recommend it
with confidence.

Could one of you write a very simple example "Big.tpl" that won't work with
a 32-bit compiler? I think it just needs to declare a huge array (I'm not
sure how big) and use it in the calculation. It would be great to have a
simple example that everyone could test. For example, the
PRELIMINARY_CALCS_SECTION might generate a huge data array used in a simple
linear regression.

Thanks again for the great work! I think we've solved the problem. I also
think we should post files ADMBtools32 and ADMBtools64 on the ADMB web site.
ADMB users could download these exactly as R users download Rtools. It would
also give all developers a standard toolkit for working on the Windows
platform.

Jon

PS to Wade - The problem I described occurred on only one computer. When I
tried a different one, the same procedure (the one that you tested) also
worked for me. I've tried using a registry cleaner on the first computer.

PS to Chris - Does it really matter whether or not a normal command window
or a bash window is used to issue the "make" command? If so, I'd appreciate
understanding why.

From: Grandin, Chris [mailto:Chris.Grandin at dfo-mpo.gc.ca]
Sent: December-09-13 7:08 PM
To: Cooper, Wade; Jon Schnute; John Sibert; Johnoel Ancheta;
developers at admb-project.org<mailto:developers at admb-project.org>
Cc: Haigh, Rowan
Subject: How to build admb source with mingw 64 - finally!

Hi Wade,
You definitely have a 64-bit build.  I have it working now too, and I put
together a little howto so that other developers can get it working. The
trick is getting msys64 as well as the tdm mingw64/32 compiler. I read in
many places that MSYS64 didn't exist and was led down many blind alleys
before finding the right combination with your suggestion of TDM. This is
the bleeding edge and I must give you your kudos due for getting it working
first, nice job!

I'll try building the debug version tomorrow and let you all know how that
goes.

Jon, if you could try this howto out I would really appreciate it!
Cheers & thanks
Chris

-----Original Message-----
From: Cooper, Wade [mailto:Wade.Cooper at MyFWC.com]
Sent: Mon 12/9/2013 6:18 PM
To: Grandin, Chris; Jon Schnute; John Sibert; Johnoel Ancheta;
developers at admb-project.org<mailto:developers at admb-project.org>
Cc: Haigh, Rowan
Subject: RE: Replacement for MinGW; bug in ADMB revision 1461

Hi Chris,

You're right this could be.  I don't have VS dumpbin, so tried another
approach.  Checking on my home machine I built from source in the same
fashion a few months back, I ran the spatial model checked my task manager
while running - it comes up as spatial.exe (not spatial.exe *32).  Not sure
if that says a lot, but the website you directed me to suggests that may be
64-bit.

As a 2nd approach I tried objdump -a filename.obj, which comes up as
spatial.obj:  file format pe-x86-64

This is outside my expertise in interpreting since i'm not a developer, ADMB
is about the only source building I do.

Wade

________________________________________
From: Grandin, Chris [Chris.Grandin at dfo-mpo.gc.ca]
Sent: Monday, December 09, 2013 8:32 PM
To: Cooper, Wade; Jon Schnute; John Sibert; Johnoel Ancheta;
developers at admb-project.org<mailto:developers at admb-project.org>
Cc: Haigh, Rowan
Subject: RE: Replacement for MinGW; bug in ADMB revision 1461

Hi Wade,
  Thanks for your help with this.
But.. I think you are compiling 32-bit objects.  I also got TDM64 4.8.1 (it
is 32/64 bit) and compiled the same way you do but the object files created
are 32-bit Windows objects. I checked this by running:

dumpbin /headers
C:\admb-trunk\build\objects\dist\optlp-df1b2-separable-adpool.obj

(You need Visual studio for dumpbin.exe or look here for another way to
tell:
http://superuser.com/questions/358434/how-to-check-if-a-binary-is-32-or-64-b
it-on-windows)

The (truncated) output follows, the line that says 14C machine(x86) should
read x64 instead of x86

Dump of file optlp-contrib-alk.obj

File Type: COFF OBJECT

FILE HEADER VALUES
             14C machine (x86)
               7 number of sections
               0 time date stamp Wed Dec 31 16:00:00 1969
             F30 file pointer to symbol table
              33 number of symbols
               0 size of optional header
             104 characteristics
                   Line numbers stripped
                   32 bit word machine

Cheers
Chris

-----Original Message-----
From: Cooper, Wade [mailto:Wade.Cooper at MyFWC.com]
Sent: Mon 12/9/2013 7:43 AM
To: Jon Schnute; 'John Sibert'; 'Johnoel Ancheta';
developers at admb-project.org<mailto:developers at admb-project.org>
Cc: Haigh, Rowan; Grandin, Chris
Subject: RE: Replacement for MinGW; bug in ADMB revision 1461

Hi Jon,

I just pulled the latest version from the svn this morning (revision 1461),
built using my normal approach (make through msys shell) with my existing
mingw-w64 (4.8.1 from tdm64) and it works fine.  I checked simple and polio
and both ran.

I then tried the mingw-w64 version you had a link for in your ADMBtools.pdf
document
(http://sourceforge.net/projects/mingwbuilds/files/host-windows/releases/4.8
.1/64-bit/threads-posix/seh/x64-4.8.1-release-posix-seh-rev5.7z/download).
This worked fine too on my machine (Win 7 64 bit).

Good luck with diagnosing the issue,

Wade

-----Original Message-----
From: Jon Schnute [mailto:schnutej-dfo at shaw.ca]
Sent: Monday, December 09, 2013 4:06 AM
To: 'John Sibert'; 'Johnoel Ancheta'; developers at admb-project.org<mailto:developers at admb-project.org>
Cc: 'Haigh, Rowan'; Cooper, Wade; 'Chris Grandin'
Subject: Replacement for MinGW; bug in ADMB revision 1461

John and others - I hope you just got my message about problems with the old
MinGW. Next I enclose my proposed solution for moving to MinGW-w64 (see
ADMBtools.pdf). The file includes links to the download sites for each
software package. I ended up with something close to that used by Wade
Cooper. I also used the discussion at Rioki's Corner
(http://www.rioki.org/2013/10/16/mingw-w64.html).

Having assembled these tools on my computer, I used them to build the latest
revision 1461. Unfortunately, I encountered a bug that you can see in the
following output:

**********
g++ -c -Wall -O3  -DOPT_LIB -D__GNUDOS__ -Dlinux -D__SPDLL__
g++ -DUSE_LAPLACE
-I../
build/dist/include
-o../build/objects/dist/optlp-df1b2-separable-fquadpri.obj df
1b2-separable/fquadpri.cpp
g++ -c -Wall -O3  -DOPT_LIB -D__GNUDOS__ -Dlinux -D__SPDLL__
g++ -DUSE_LAPLACE
-I../
build/dist/include -o../build/objects/dist/optlp-df1b2-separable-gammdev.obj
df1
b2-separable/gammdev.cpp
GNUmakefile:267: recipe for target
`../build/objects/dist/optlp-df1b2-separable-
gammdev.obj' failed
make[2]: *** [../build/objects/dist/optlp-df1b2-separable-gammdev.obj] Error
1
make[2]: Leaving directory `/d/ADMbuild/admb1461/src'
Makefile:63: recipe for target `g++-src' failed
make[1]: *** [g++-src] Error 2
make[1]: Leaving directory `/d/ADMbuild/admb1461'
Makefile:60: recipe for target `g++-all' failed
make: *** [g++-all] Error 2
**********

A strange window popped up referring to "cc1plus.exe". Is it possible that
the makefile confuses the new version of MinGW with Microsoft Visual Studio?

Wade, it would be great if you could test the latest version of ADMB with
your software.

Best wishes to all,

Jon

-----Original Message-----
From: Jon Schnute [mailto:schnutej-dfo at shaw.ca]
Sent: December-09-13 12:41 AM
To: 'John Sibert'; 'Johnoel Ancheta'; 'developers at admb-project.org'
Cc: 'Haigh, Rowan'; 'Cooper, Wade'
Subject: MinGW "nasty problem"

Hi John - Your experience with the MinGW "nasty problem" matches mine
exactly. I've done a lot of research on this, and I've written the attached
report "MinGW problem with drives that don't exist". I think you'll find
that it offers a credible explanation. It contains many Internet links that
connect you with other sources.

Along with other developers (cited in the report), I've concluded that the
original MinGW project has fallen off the rails and can no longer be
considered reliable. We need to move to the independent new project
MinGW-w64. In the long run, we need to do this anyway to get 64-bit support.

I hope my little report convinces everyone involved. I'm copying Wade Cooper
on this because he reported success with ADMB in the message he sent on
November 18.

In my next message, I'll send my recommendations for the explicit software I
think we should use.

Best wishes,

Jon

-----Original Message-----
From: John Sibert [mailto:sibert at hawaii.edu]
Sent: December-06-13 9:54 AM
To: Jon Schnute; 'Johnoel Ancheta'; developers at admb-project.org<mailto:developers at admb-project.org>
Cc: 'Haigh, Rowan'
Subject: Re: [Developers] Big changes in Revision 1438

Hi Jon,
Thank you for your patience in testing this stuff. It can be onerous in
windows.

Regarding the "nasty problem with MinGW" that you mentioned, I may have
encountered something similar.

I was trying to install admb on my wife's new windows 7 machine and the
procedure ran smoothly until it started to build the libraries. As soon as
it started to compile the first source file, the system popped up an error
window titled "g++" containing the error message "There is no disk in the
drive. Please insert a disk in drive \Device\Harddisk2\DR2."
Further experimentation revealed the error window any time g++ starts.
So it appears not to be anything to do with admb; it is an windows/mingw
issue.

Searching for a solution was pretty fruitless. I lot of know-nothing
responses to similar questions and BS remarks from microsoft flacks about
how important solving the problem is to microsoft. But the responses pointed
to usb drive issues.

So I'm wonder if this is the "nasty problem" you mentioned, and if so, how
it was solved.

Cheers,
John

John Sibert
Emeritus Researcher, SOEST
University of Hawaii at Manoa
Honolulu HI (GMT-10)
808-294-3842

Visit the ADMB project http://admb-project.org/

On 12/05/2013 11:17 PM, Jon Schnute wrote:
>
> I've tested revisions 1444 and 1447 on Windows 7 using Msys and MinGW,
> obtained independently from the Internet. The build process worked in
> both cases with no problems. Also, the executable files and scripts in
> ..\build\dist successfully converted .tpl files to .exe files.
>
> In the process, I discovered a nasty problem with MinGW that stems
> from a bug in Windows. It will occur if you connect and disconnect a
> USB drive before using MinGW.
>
> I'm planning to write installation instructions that document exactly
> what I've done, along with a warning about the potential MinGW problem
> and its resolution. I'll circulate this document when I have a draft.
>
> I should add that Rowan Haigh, using the official QuickStart procedure
> on revision 1444, encountered the problem with "banner.cpp" that I've
> described in earlier email. When we have the opportunity, we'll check
> into this further and send a bug report if we can reproduce the problem.
>
> Again, thanks to everyone for your good work cleaning up the code.
>
> Jon
>
> *From:*developers-bounces at admb-project.org
> [mailto:developers-bounces at admb-project.org] *On Behalf Of *Johnoel
> Ancheta
> *Sent:* December-05-13 1:32 AM
> *To:* developers at admb-project.org<mailto:developers at admb-project.org>
> *Subject:* [Developers] Big changes in Revision 1438
>
> Hi all,
>
> The ADMB build files have again been reworked and improved. After
> getting
>
> feedback,
>
> * the build files have been simplified a bit more
>
> - Dave Fournier
>
> * works with Rtools (but not with Rtools->make)
>
> - Jon Schnute
>
> * should build in Windows if Msys or Rtools are in the system PATH
>
> A lot of time and effort was spent by everyone to test and develop
>
> the build scripts.
>
> Please test the installation by following documentation in
>
> http://www.admb-project.org/buildbot/documentation/
>
> Changes and QuickStart pages are in the the directory.
>
> Thanks to everyone who participated in the testing.
>
> Johnoel
>
>
>
> _______________________________________________
> Developers mailing list
> Developers at admb-project.org<mailto:Developers at admb-project.org>
> http://lists.admb-project.org/mailman/listinfo/developers

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.admb-project.org/pipermail/developers/attachments/20131210/a3491b34/attachment-0001.html>