Want to protect your cyber security and still get fast solutions? Ask a secure question today.Go Premium

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 472
  • Last Modified:

Linking fortran to c in a makefile

I am getting compiling errors when making a program.  This is the advice from the developer:

This looks like BLACS is having problems getting symbols resolved.

This can happen if scalapack and BLACS are not built correctly (although I though this issue was taken care of in newer versions of mkl): scalapack is written in fortran and BLACS in C and one needs to construct a solver build which allows the two to cross boundaries between the language (the infamous underscore—double-underscore problem).  The default build of the solvers did not include the expanded symbol table that allow a Fortran code to call the mixed-language solver libraries.   You might look into that build to be sure that the symbol tables have the redundancy in them that allow this.  As I said, I though this had been “fixed” in the mkl that Intel ships, but perhaps not.

However, I added the following  option to my makefile and they still don't work.

 -assume 2underscores

and

-fsecond-underscore

Here is my errors:

a(BI_BlacsAbort.o): In function `BI_BlacsAbort':
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_BlacsAbort.c:(.text+0x26): undef                                                                                                 ined reference to `ompi_mpi_comm_world'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.                                                                                                 a(BI_Pack.o): In function `BI_Pack':
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_Pack.c:(.text+0x13): undefined r                                                                                                 eference to `ompi_mpi_byte'
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_Pack.c:(.text+0x6b): undefined r                                                                                                 eference to `ompi_mpi_packed'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.                                                                                                 a(BI_GetMpiGeType.o): In function `BI_GetMpiGeType':
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_GetMpiGeType.c:(.text+0x39): und                                                                                                 efined reference to `ompi_mpi_byte'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.                                                                                                 a(BI_GetMpiTrType.o): In function `BI_GetMpiTrType':
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_GetMpiTrType.c:(.text+0x9f): und                                                                                                 efined reference to `ompi_mpi_byte'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.                                                                                                 a(BI_MPI_C_to_f77_trans_comm.o): In function `BI_MPI_C_to_f77_trans_comm':
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_MPI_C_to_f77_trans_comm.c:(.text                                                                                                 +0x6): undefined reference to `MPI_Comm_c2f'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.                                                                                                 a(sys2blacs_handle_.oo): In function `Csys2blacs_handle':
../../../../scalapack/BLACS/SRC/MPI/sys2blacs_handle_.c:(.text+0x1e): undefined                                                                                                  reference to `ompi_mpi_comm_null'
../../../../scalapack/BLACS/SRC/MPI/sys2blacs_handle_.c:(.text+0xaf): undefined                                                                                                  reference to `ompi_mpi_comm_world'
../../../../scalapack/BLACS/SRC/MPI/sys2blacs_handle_.c:(.text+0x17d): undefined                                                                                                  reference to `ompi_mpi_comm_world'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.                                                                                                 a(BI_MPI_F77_to_c_trans_comm.o): In function `BI_MPI_F77_to_c_trans_comm':
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_MPI_F77_to_c_trans_comm.c:(.text                                                                                                 +0x6): undefined reference to `MPI_Comm_f2c'
make: *** [quest_265c.x] Error 1
0
ted_yu
Asked:
ted_yu
  • 8
  • 7
1 Solution
 
gheistCommented:
I think you have mismatch between header and object files
You need to either build both C and F with -fopenmp or without.
0
 
ted_yuAuthor Commented:
thanks, just to clarify, do you mean to build the software...or do you mean building the library that it is referencing?
0
 
gheistCommented:
I think your fortran program expectc C library to be linked with openmp library (would be nice to hear what compiler you use and what OS...)
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
ted_yuAuthor Commented:
Sorry for the delay.  Here is the makefile.  The OS is redhat.

qvrs = 265c

FC = mpif77
FFLAGS = -O2 -xSSE3 -mcmodel medium -shared-intel

LFLAGS = -mcmodel medium -shared-intel
LIBS =

MKL_LIB = /home/rtorres/intel/mkl/lib/

MATH = -L${MKL_LIB} -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5

PMATH = -L${MKL_LIB} -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64

SUBSU = utl_mpi.o

MVRSTP=lcaomain_tp.o

SUBS=lcaosubs.o
TPSUBS=lcaosubs_tp.o

quest: quest_${qvrs}.x

help:
        @echo "Make SeqQuest program - openmpi parallel"
        @echo "Usage: make <platform>, only platform available: quest"
        @echo "For parallel builds, you will have to specify library locations in makefile"
        @echo ""

clean:
        rm -f *.o quest_${qvrs}.x

quest_${qvrs}.x : ${MVRSTP} ${SUBS} ${TPSUBS} ${SUBSU}
        $(FC) $(LFLAGS) -o $@  ${MVRSTP} ${SUBSU} ${TPSUBS} ${SUBS} ${PMATH} ${MATH}
0
 
gheistCommented:
Just two builds disagree about their use of openmpi (I was mistaken about openmp)
So if you want to schedule program on multi-machine cluster - build all C parts with openmpi support, if not remove openmpi references from F99 makefile.
0
 
ted_yuAuthor Commented:
Unfortunately, I am not the author of this program, and I just need to get this thing compiled on my machine.  This program has been successfully compiled on other machines.  Here is the comment from the author of the program about my difficulties:

"This can happen if scalapack and BLACS are not built correctly (although I though this issue was taken care of in newer versions of mkl): scalapack is written in fortran and BLACS in C and one needs to construct a solver build which allows the two to cross boundaries between the language (the infamous underscore—double-underscore problem).  The default build of the solvers did not include the expanded symbol table that allow a Fortran code to call the mixed-language solver libraries.   You might look into that build to be sure that the symbol tables have the redundancy in them that allow this.  As I said, I though this had been “fixed” in the mkl that Intel ships, but perhaps not."

I cannot rewrite the code, so the best way to solve this, as I understand it is to include lines on my makefile such as:
 -assume 2underscores
-fsecond-underscore

If I can understand these things a little better, I can try to fix the problem by changing my makefile.

The mkl math library I am using is by Intel and is already installed in the cluster I am using.  I can try to recompile it as another option.

So, the only thing I can change at the moment are:
1)  the makefile
2)  recompiling the mkl library which contains blacs and scalpack differently.
0
 
gheistCommented:
Change (i.e comment out):
# PMATH = -L${MKL_LIB} -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64
To (i.e add new line):
PMATH = -L${MKL_LIB} -lmkl_scalapack_lp64

You will lose ability to run product on multi-node cluster.
Ahouls you need to run it on multi-node cluster you need to build blacs with openmpi support, and then build your F99 program using original Makefile..
0
 
ted_yuAuthor Commented:
Now I'm getting errors that scalapack needs references that are not there (from blacs)

/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `zgesd2d_'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `Cblacs_barrier'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `dtrbs2d_'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `ctrbr2d_'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `dcputime00_'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `Cigsum2d'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `igebs2d_'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `strrv2d_'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `blacs_pnum_'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `MKL_DOUBLE'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `MKL_MAXLOC'
make: *** [quest_265c.x] Error 1

I also need to run this on multi-nodes.  Do you have any idea what it means to look for the  following that was mentioned?  " The default build of the solvers did not include the expanded symbol table that allow a Fortran code to call the mixed-language solver libraries. "  What is this mixed-language solver library and where is it located?
0
 
gheistCommented:
Take it ready on EL7+EPEL, no need to take long leaps to re-invent the wheel.
http://mirror.uv.es/mirror/fedora-epel/7/x86_64/repoview/blacs-openmpi-devel.html
0
 
ted_yuAuthor Commented:
OK.  I downloaded that file and put it in the cluster.  I may not have changed the Makefile correctly, because I'm still getting an error.  Here are my changes to the makefile.

BLACS_LIB = /shares/teamyu/usr/lib64/openmpi/lib/

PMATH = -L${MKL_LIB} -lmkl_scalapack_lp64 -L${BLACS_LIB} -lmkl_blacs_openmpi_lp64

This is going back to the original error, so it's still trying to use BLACS from the mkl library.:
../../../../scalapack/BLACS/SRC/MPI/sys2blacs_handle_.c:(.text+0x17d): undefined                                                                                                              reference to `ompi_mpi_comm_world'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.                                                                                                             a(BI_MPI_F77_to_c_trans_comm.o): In function `BI_MPI_F77_to_c_trans_comm':
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_MPI_F77_to_c_trans_comm.c:(.text                                                                                                             +0x6): undefined reference to `MPI_Comm_f2c'
make: *** [quest_265c.x] Error 1
0
 
gheistCommented:
I was trying to talk you into getting blacs+scalapack as installable RPMs (you know desktop virtualisation ant the stuff for testing)
EPEL RPMs are build with MPI support, so RHEL7+EPEL will sustain your build.
btw no need to target SSE3 - it is so past, --march-native gives bigger bang for the buck.
0
 
gheistCommented:
Aw pain - you use ICC. You must build both components with same ICC. What I am suggesting is to make POC build with GCC.
0
 
ted_yuAuthor Commented:
I was advised to use Intel mkl because it is the fastest by a factor of two.  I would like to use it if possible.  Any chance to fix this with the underscore issue?  

 -assume 2underscores
-fsecond-underscore

Is it possible scalapack is not calling blacs correctly and it is just an easy fix of the settings?
0
 
gheistCommented:
BLACS is not build with openmpi support.
SSE3 is 10 years old. Are you certain GCC does not generate that?
0
 
ted_yuAuthor Commented:
OK, I've given up on using Intel MKL for this build and am using a different library.  Thanks for your help.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

  • 8
  • 7
Tackle projects and never again get stuck behind a technical roadblock.
Join Now