I was able to compile a win64 version of the ADMB libaries using the mingw64 linux/win64 cross compiler. I used these libriries to compile a 64bit DLL for the bimix example. This ran fine with 64 bit R so this compiler seems to be a good way to support win64 DLL's.