Calling Fortran Routines Externally from Unix Versions of IDL.
              Jeff Valenti (Valenti@Soleil.Berkeley.Edu)
                             Mar 26, 1992

Additional Credits: Fen Tamanaha, Gwyn Fireman, Phil Turet Disclaimer: It worked for me, but it may not work for you.

Many people have requested help calling fortran routines externally from Unix versions of IDL. The makers of IDL (RSI) can't help much, as they apparently do not have fortran on their Unix machines, and thus they lack the required experience. I have succeeded in calling a few fortran routines from IDL, so perhaps it would be appropriate for me to discuss some of the problems involved and provide a working example.

The following discussion applies specifically to a Sun IPC running SunOS 4.1.1, but it will probably provide useful hints for other Unix environments as well. Please let me know if you get this working on other machines, so I can include the information in later revisions of this letter. Corrections, additions, and comments should be sent to me at Valenti@Soleil.Berkeley.Edu.

All right, my first task is to try to convince you to turn back from your goal. Calling fortran from Unix IDL is rather painful and should only be attempted if it is absolutely necessary. Here are some other options to consider first.

1. Translate your fortran code into IDL. Most anything written in fortran, can be rewritten in IDL, assuming you have complete source code. Your code will look prettier too! If your fortran program is *very* long (e.g. more than 500 lines of code) it *might* take you longer to translate, than to figure out how to call it from IDL. The only real problem with this option is that some fortran programs do not vectorize well, in which case the IDL version will be significantly slower. If execution time is of paramount importance and your code will not vectorize, then you probably can't get away with translating to IDL.
2. Compile and link your fortran program as a separate executable. Then invoke it from within IDL using IDL's "spawn" command. Data may be passed between IDL and the fortran program either via a disk file or via a Unix pipe. One form of pipe captures the fortran program's standard output in a string array, which is returned as the second argument of the spawn command. ASCII formatted numbers captured in the string array can be converted back into numeric data with the IDL functions float(), long(), etc. A second form of pipe ("bidirectional") connects the fortran program's standard input and output to one of IDL's logical units. Formatted data are then transferred using IDL's readf and printf commands and fortran's read(*,*) and write(*,*) commands. (Has anyone sent unformatted data back and forth between spawned fortran programs?) Before discussing how to call fortran programs externally, let me give you three examples of how to communicate with spawned fortran jobs.

============================ IDL Spawn Examples ============================ soleil:idl% cat fexample1.f

• Example of communication between IDL and fortran via a disk file. program fexample1

integer*2 n
real*4 x(50), y(50)

• Read input arguments from disk file. open(1,file='temp_file',type='old') !open input disk file read(1,*) n,(x(i),i=1,n) !read argument from stdin close(1) !close input disk file
• Calculate the square of each element of input vector. do i=1,n !loop thru elements y(i) = x(i)*x(i) !calculate square enddo
• Write output arguments back to disk file open(1,file='temp_file') !open output disk file write(1,*) (y(i),i=1,n) !write result to stdout close(1) !close output disk file

end

soleil:idl% idl
IDL> n = 10 ;length of data vectors IDL> x = findgen(n) ;init data to fortran IDL> y = fltarr(n) ;init data from fortran IDL> openw,unit,'temp_file',/get_lun ;open temporary file IDL> printf,unit,n,x ;write data for fortran IDL> free_lun,unit ;close file, release lun IDL> spawn,'fexample1' ;fortran program runs, exits IDL> openr,unit,'temp_file',/get_lun ;open temporary file IDL> readf,unit,y ;read data from fortran IDL> free_lun,unit ;close file, release lun IDL> print,y

      0.00000      1.00000      4.00000      9.00000      16.0000
      16.0000      36.0000      49.0000      64.0000      81.0000

IDL> exit

• Example of capturing fortran's standard output into an IDL string array. program fexample2

integer*2 n
real*4 x(50), y(50)

• Read input arguments from disk file. open(1,file='temp_file',type='old') !open input disk file read(1,*) n,(x(i),i=1,n) !read argument from stdin close(1) !close input disk file
• Calculate the square of each element of input vector. do i=1,n !loop thru elements y(i) = x(i)*x(i) !calculate square enddo
• Write output arguments to standard output (which will be piped to IDL).
• Don't use write(*,*) below. IDL's string to numeric conversion functions
• (e.g. float) won't read multiple numbers from a single string element. write(*,'(f)') (y(i),i=1,n) !write result to stdout

end

soleil:idl% idl
IDL> n = 10 ;length of data vectors IDL> x = findgen(n) ;init data to fortran IDL> openw,unit,'temp_file',/get_lun ;open temporary file IDL> printf,unit,n,x ;write data for fortran IDL> free_lun,unit ;close file, release lun IDL> spawn,'fexample2',result ;run program, capture output IDL> y = float(result) ;convert strings to real*4 IDL> help,result
RESULT STRING = Array(10)
IDL> print,y

      0.00000      1.00000      4.00000      9.00000      16.0000
      16.0000      36.0000      49.0000      64.0000      81.0000

IDL> exit

• Example of communication between IDL and fortran via bi-directional pipe. program fexample3

integer*2 n
real*4 x(50), y(50)

• Read input arguments from standard input (which will be piped from IDL). read(*,*) n,(x(i),i=1,n) !read argument from stdin
• Calculate the square of each element of input vector. do i=1,n !loop thru elements y(i) = x(i)*x(i) !calculate square enddo
• Write output arguments to standard output (which will be piped to IDL). write(*,*) (y(i),i=1,n) !write result to stdout

end

soleil:idl% idl
IDL> n = 10 ;length of data vectors IDL> x = findgen(n) ;init data to fortran IDL> y = fltarr(n) ;init data from fortran IDL> spawn,'fexample3',unit=unit ;run program, establish pipe IDL> printf,unit,n,x ;write data to pipe IDL> readf,unit,y ;read data from fortran IDL> free_lun,unit ;close pipe, kill program IDL> print,y

      0.00000      1.00000      4.00000      9.00000      16.0000
      16.0000      36.0000      49.0000      64.0000      81.0000

IDL> exit

======================== End of IDL Spawn Examples =========================

If these alternatives simply will not solve your problem, then it's time to discuss some of the difficulties involved in calling fortran externally from Unix IDL. Again, these comments are specific to SunOS 4.1.1, but may have more general applicability. Call_external is *not* available in IDL running under DEC's Ultrix operating system; use one of the alternatives described above.

The first problem is that IDL uses the standard C mechanism for passing arguments. Unfortunately, fortran is not flexible enough to handle arguments passed in this manner. So you will have to write a C interface to sit between IDL and your fortran routines and to convert between the two different argument formats. This C interface must be linked with your fortran program. IDL passes the number of arguments followed by the address of a list which contains the addresses of the data for each of the arguments. Fortran expects to recieve just a list of addresses of the data for each argument. I have successfully passed real*8 and integer*4 data types between IDL and fortran via a C interface. There may be problems passing other data types. Let me know if you are successful or even unsuccessful passing other data types.

Now comes the really tricky part. You have to compile your program and link it with the fortran library to make a "shared library", rather than a stand alone executable. This is what allows IDL to interface with your routine at execution time. Begin by compiling your programs (with f77 or cc as appropriate) with the -pic and -c flags. These options allow the code to be loaded anywhere in memory and supress automatic linking respectively. Now you must link the programs with the fortran libraries becasue IDL does not include these libraries (or even know where to find them). Normally, f77 automatically links your programs and the fortran libraries, but we supressed this behavior with the -c flag above because f77 would have linked our program as a stand alone executable.

To find your libraries, type f77 -v /dev/null from your unix prompt. On my system, I get the following output.

/bin/ld -dc -dp -e start -u _MAIN_ -X -o a.out /usr/lang/SC1.0/crt0.o /usr/lang/SC1.0/cg87/_crt1.o -L/usr/lang/SC1.0/cg87 -L/usr/lang/SC1.0 /dev/null -lF77 -lm -lc
ld: /dev/null: premature EOF
Compilation failed

These are the options f77 uses to link your fortran program. The first modifications required for our purpose are to remove the -e start option and to remove the crt0.o and _crt1.o modules from the link list. This will force our output to be a shared object, rather than a stand alone program. I also had to include the math (/usr/lang/SC1.0/libm.a) and fortran (/usr/lang/SC1.0 /libF77.a) libraries as explicit link objects rather than simple library references as in the ld -v /dev/null example above. The -lc is uneccessary for our purpose, since IDL is already linked with the C library. (As a side note, I was unable to dynamically link at run time with the .so versions of these libraries.) In SunOS 4.1 the output file (specified with the -o option) should have the extension ".so". A sample link line is given in the Makefile in the example below.

The last step is to call your shared library from IDL. Make sure to prepend an underscore before the name of your C interface routine (e.g. _cinterface below). When you call your fortran routine from the C interface, you must both prepend and append underscores (e.g. _ftest_ below). It is essential that the data types you specify in IDL prior to the call_external be the types expected by your programs. Failure to heed this warning can lead to very cryptic error messages. As always in IDL, don't expect data to be returned in specifically referenced array elements. Use a scalar temporary variable instead.

Finally, here is an example that works for me. It's probably best if you begin with this example and slowly modify it to match your needs. Feel free to contact me for help or to give me corrections, additions, or comments.

=========================== Call_External Example ============================

Script started on Fri Sep 20 11:42:52 1991

soleil:idl% cat ftest.f

• This function is defined as integer*4, so expect _ftest_ to return integer*4
• in the C interface program. integer*4 function ftest(n,x,y)
• Define arguments types consistant with those being passed by IDL. integer*4 n
  real*8 x(*), y(*), f
• Simple fortran code. [y=x*x; f=total(y)] write(*,*) 'This is FORTRAN at work...' f = 0.0
  do i=1,n
  y(i) = x(i) * x(i)
  f = f + y(i)
  enddo
• The call_external function will evaluate to the value of ftest. ftest = NInt(f)
  return
  end

soleil:idl% cat cinterface.c
/*
Here we let our C interface know our fortran function is integer*4. Perhaps the function could be defined as (void) for greater generality. The symbol PROG will be assigned a value "_ftest_" in the Makefile. Other fortran programs need only specify a different value of PROG. */
extern long PROG();

/*
Here we define the return value of our C interface as integer*4. */
int cinterface(argc, argv)

/*
Here we prepare for IDL's argument passing structure. The double could perhaps be changed to void for greater generality. */
int argc;
double *argv[];
{

/*
This is currently set up for no more than 3 arguments, but can clearly be extended by analogy to more arguments. */
switch (argc) {

        case 0:         return PROG();
        case 1:         return PROG(argv[0]);
        case 2:         return PROG(argv[0], argv[1]);
        case 3:         return PROG(argv[0], argv[1], argv[2]);
        default:        (void) printf("Too many arguments.\n");

}

return -1;
}

soleil:idl% cat Makefile
# Note that PROG in cinterface.c will be replaced by _ftest_. # Also note that the <TAB>s in makefiles like this are essential! cinterface.o: cinterface.c Makefile
$(CC) -pic -c -DPROG=ftest_ cinterface.c

ftest.o: ftest.f
f77 -pic -c ftest.f

ftest: ftest.o cinterface.o
$(LD) $(LDFLAGS) -d -o ftest.so \ cinterface.o ftest.o /usr/lang/SC1.0/libF77.a /usr/lang/SC1.0/libm.a

soleil:idl% make ftest

soleil:idl% idl
IDL. Version 2.0.11 (sunos sparc). Copyright 1989, Research Systems, Inc. All rights reserved. Unauthorized reproduction prohibited. Site: 968.
Licensed for use by: UC Berkeley Astronomy

IDL> m=0L ;integer*4 result of call_external() IDL> n=10L ;integer*4 as expected by ftest IDL> x=dindgen(n) ;real*8 as expected by ftest IDL> y=dblarr(n) ;real*8 as will be returned by ftest IDL> m=call_external('ftest.so','_cinterface',n,x,y) This is FORTRAN at work...
IDL> print,m
285
IDL> print,y

       0.0000000       1.0000000       4.0000000       9.0000000
       16.000000       25.000000       36.000000       49.000000
       64.000000       81.000000

IDL> exit