[RADS] RADS nightmare

Remko Scharroo remko at altimetrics.com
Fri May 2 19:13:44 CEST 2008


Hi Helen,

I'm CC-ing this to the RADS group for edification, and showing that I  
don't have the ultimate answer to all questions.

I see your problem with having to deal with multiple systems running  
from a single (distributed) filesystem.
In the future, I'll add a "configure" tool so that things will become  
easier. Also the Fortran90 version will abandon all C routines  
compiled in and all I/O will be either netCDF or ASCII, which will  
simplify things considerably (no more -bytereclen, no more byte  
swapping).

On 2 May 2008, at 11:50, Helen Snaith wrote:

> Hi Remko
>
> thanks for all the comments
>
> One of my big problems here is that I don't (can't) administer the  
> system.
> We have a single file system which is accessed from all the machines  
> - a combination of linux & solaris flavours.
> To keep track of which software is actually run, we use links (some  
> at very high levels in the files system) so, eg, $RADSROOT/bin will  
> actually be $SATHOME/linux/bin/rads or $SATHOME/solaris/bin/rads
>
> This works pretty well normally - worked very well when we used a  
> mix of solaris & sgi machines. Compiling on any "solaris" machine  
> gave the same result and the resultant libraries & executables could  
> be linked / run on any other solaris machine. The problems have  
> arisen with linux, when nearly every machine has a subtly different  
> OS release - so (eg) make and gmake may be different on each  
> machine. Combined the the plethora of possible compilers (g77, f90,  
> gfortran, ifort)  we now have a bit of a mess.
>
> We can control which versions of SOME packages are accessed using a  
> setup command - eg "setup v9.1 intel_compilers" or "setup v8.1  
> intel_compilers". Unfortunately, we have to lobby IT to get our  
> favourite versions of different packages installed - and at the  
> moment, it seems 3.6.0 is the latest version of netcdf installed  
> (I'll push for a later one)

NetCDF 3.6.0-p1 was released in Feb 2005, 3.6.2 in March 2007. It is  
not unrealistic to ask to replace a 3-year old software by a 1-year  
old software. It's not like you are asking to replace something  
relatively new with something on the cutting edge. And it's FREE!

> We're about to get ifort 10, which is supposed to be much less  
> "flaky" than the previous ones, and better than gfortran, but I  
> think for now, I'll follow your advice, stick with gfortran for all  
> the code (I was already switching other code anyway) and get netcdf  
> upgraded.

So you are getting ifort 10. At least we all agree about the  
"flakiness" of ifort in the past.

> I've actually got the make to work up to radsstats now - using  
> gfortran and the ifort version of netcd, supposedly compatible!.
> The slip-up now is :
>
> use typesizes
>
> Fatal Error: File 'typesizes.mod' opened at (1) is not a GFORTRAN  
> module file

That is understandable. The module was probably created ("compiled")  
with another compiler (version). typesizes.mod comes with netCDF, by  
the way, after compilation. There are ways around it, and I could  
create code that would produce typesizes.mod by itself.

Alternatively, get typeSizes.f90 out of the netCDF package and compile  
it. It will create typesizes.mod. Put that in $(INCDIR)/$(HOSTTYPE)  
and you should be set. That directory is checked before the location  
of the netcdf include files, so the netcdf version will be ignored.  
I'll do that trick in the CVS version, so that it's always there.

What typesizes.mod does, BTW, is to define the values "fourbyteint",  
"twobyteint", etc, as in:
integer(fourbyteint) :: i
On most systems they are simply 4 and 2, but I read that they can be  
surprisingly different on some systems, so I borrowed typesizes.mod  
from netCDF.

Have a good weekend,
Remko

> On 2 May 2008, at  3:57PM, Remko Scharroo wrote:
>
>> Hi again Helen,
>>
>> (The "commute" was rather uneventful, other than two cats needing  
>> attention).
>>
>> The -I$(INCDIR)/$(HOSTTYPE) was already in the options, so all I  
>> needed to do was to make sure that rads.mod was moved into that  
>> directory.
>>
>> OK, now your second e-mail. I think I have a solution for both.
>>
>> On 2 May 2008, at 06:47, Helen Snaith wrote:
>>
>>> Hi Remko
>>>
>>> I'm having a nightmare updating the RADS software to use NetCDF on  
>>> our (rather complex) system
>>>
>>> Essentially, whichever OS (solaris or linux) I use and whichever  
>>> compiler, there is a problem!
>>>
>>> 1) I'll start with my preferred option:
>>>
>>> running suse linux (HOSTTYPE is just reported as linux so I don't  
>>> have to mess up any of your existing sysdep stuff)
>>> on Opteron 150 or Sun Ultra 40 M2
>>> Using intel compilers (8.1, 9.0 or 9.1 all available)
>>> with netcdf v3.6.0-pl1 for ifort
>>
>> I suggest you upgrade to NetCDF 3.6.2, no matter what. There were a  
>> lot of bugs in 3.6.0 and 3.6.1.
>>
>>> Compiles ok until I get to putnc_dump.f line 247:
>>>
>>> ifort -I. -I/nerc/packages/satprogs/src/rads/src/include/linux -I/ 
>>> nerc/packages/satprogs/src/rads/src/include  -I/nerc/packages/ 
>>> netcdf/v3.6.0-pl1/include -O  -c -o putnc_dump.o putnc_dump.f
>>> fortcom: Error: putnc_dump.f, line 247: Invalid  
>>> character_kind_parameter. No underscore
>>>    |                   "coordinates",7,"lon lat"),*1310)
>>> --------------------------------------^
>>> fortcom: Error: putnc_dump.f, line 247: Syntax error, found  
>>> CHARACTER_CONSTANT 'coordinates' when expecting one of:  
>>> = .EQV. .NEQV. .XOR. .OR. .AND. .LT. < .LE. <= .EQ. == .NE. / 
>>> = .GT. > ...
>>>    |                   "coordinates",7,"lon lat"),*1310)
>>> -------------------------^
>>> compilation aborted for putnc_dump.f (code 1)
>>> make[1]: *** [putnc_dump.o] Error 1
>>
>> I'm not sure what the error is, but it is probably the ancient use  
>> of the "goto on return value" that is in that call. The "*1310"  
>> says to jump to line 1310 when the return value is 1.
>>
>>> I compiled sysdep, using ifort and icc, and it came up with:
>>> sysdep.h:
>>> #define SWAP
>>> #undef CAPITALS
>>> #define UNDERSCOREAFTER
>>> #undef UNDERSCOREBEFORE
>>>
>>> sysdep.mk
>>> ###################################################
>>> # The record length is in words. You are like SGI #
>>> ###################################################
>>> FFLAGS2 = -bytereclen
>>>
>>> to which I've added
>>> FC = ifort
>>> CC = icc
>>>
>>> 2) As above but using gfortran and gcc instead of ifort & icc and  
>>> using the standard (not ifort version) of netcdf)
>>> sysdep.h is the same
>>> sysdep.mk now claims I'm NOT sgi, and I've changed FC, CC & added  
>>> fix for line length
>>
>> The SGI is a leftover from when we had SGIs in Delft. If was the  
>> only machine I knew that required the -bytereclen flag to make sure  
>> that the compiler understood that the rec=<reclen> in an "open"  
>> statement meant the size in bytes. On SGI it was considered as  
>> number of 4-byte words.
>>
>>> ::::::::::::::
>>> ##################################################
>>> # The record length is in bytes. You are not SGI #
>>> ##################################################
>>> FC = gfortran
>>> CC = gcc
>>> FFLAGS += -ffixed-line-length-132
>>>
>>> Now I can compile the library fine, but linking rads2asc I get:
>>> gfortran -o /nerc/packages/satprogs/src/rads/bin/rads2asc  
>>> rads2asc.o /nerc/packages/satprogs/src/rads/lib/librads.a -L/nerc/ 
>>> packages/netcdf/v3.6.0-pl1/lib -lnetcdf
>>> /nerc/packages/satprogs/src/rads/lib/librads.a(load_nc.o): In  
>>> function `load_nc_':
>>> load_nc.f:(.text+0x251): undefined reference to `nf_open_'
>>
>> The NetCDF package you relate to probably wasn't compiled with  
>> Fortran support, hence does not contain nf_open.
>> Again, get 3.6.2 and compile it. That REALLY works out of the box,  
>> first with 3.6.2.
>> Note that since 3.6.2 there will be two separate netcdf libraries:  
>> libnetcdf.so and libnetcdff.so (with two ff). Link both!
>>
>>> And then hundreds of other similar - ie it can't find the nf_  
>>> versions of all the netcdf calls
>>>
>>> Have you any ideas where to start with this?
>>
>> Here are my honest list of suggestions, hopefully without bias:
>> - Throw ifort overboard and use gfortran. Eelco was a big fan of  
>> ifort for a long time, but caved in too when gfortran came to the  
>> standard it is now. Eelco also showed gfortran is faster in  
>> compiling and its compiles run faster.
>> - Download and install netcdf 3.6.2. I know this is a bit of work,  
>> but you will be rewarded and surprised how easy it is. Just check ./ 
>> configure --help to see your options.
>> - Then compile all of RADS with gfortran.
>> - Eventually, compile ALL your fortran code with gfortran. There  
>> may be a few snags, but they proved minor when I did the conversion  
>> from g77.
>>
>> Have a good weekend,
>> Remko
>




More information about the RADS mailing list