Solved

Linking fortran to c in a makefile

Posted on 2014-10-20
15
388 Views
Last Modified: 2014-11-21
I am getting compiling errors when making a program.  This is the advice from the developer:

This looks like BLACS is having problems getting symbols resolved.

This can happen if scalapack and BLACS are not built correctly (although I though this issue was taken care of in newer versions of mkl): scalapack is written in fortran and BLACS in C and one needs to construct a solver build which allows the two to cross boundaries between the language (the infamous underscore—double-underscore problem).  The default build of the solvers did not include the expanded symbol table that allow a Fortran code to call the mixed-language solver libraries.   You might look into that build to be sure that the symbol tables have the redundancy in them that allow this.  As I said, I though this had been “fixed” in the mkl that Intel ships, but perhaps not.

However, I added the following  option to my makefile and they still don't work.

 -assume 2underscores

and

-fsecond-underscore

Here is my errors:

a(BI_BlacsAbort.o): In function `BI_BlacsAbort':
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_BlacsAbort.c:(.text+0x26): undef                                                                                                 ined reference to `ompi_mpi_comm_world'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.                                                                                                 a(BI_Pack.o): In function `BI_Pack':
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_Pack.c:(.text+0x13): undefined r                                                                                                 eference to `ompi_mpi_byte'
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_Pack.c:(.text+0x6b): undefined r                                                                                                 eference to `ompi_mpi_packed'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.                                                                                                 a(BI_GetMpiGeType.o): In function `BI_GetMpiGeType':
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_GetMpiGeType.c:(.text+0x39): und                                                                                                 efined reference to `ompi_mpi_byte'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.                                                                                                 a(BI_GetMpiTrType.o): In function `BI_GetMpiTrType':
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_GetMpiTrType.c:(.text+0x9f): und                                                                                                 efined reference to `ompi_mpi_byte'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.                                                                                                 a(BI_MPI_C_to_f77_trans_comm.o): In function `BI_MPI_C_to_f77_trans_comm':
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_MPI_C_to_f77_trans_comm.c:(.text                                                                                                 +0x6): undefined reference to `MPI_Comm_c2f'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.                                                                                                 a(sys2blacs_handle_.oo): In function `Csys2blacs_handle':
../../../../scalapack/BLACS/SRC/MPI/sys2blacs_handle_.c:(.text+0x1e): undefined                                                                                                  reference to `ompi_mpi_comm_null'
../../../../scalapack/BLACS/SRC/MPI/sys2blacs_handle_.c:(.text+0xaf): undefined                                                                                                  reference to `ompi_mpi_comm_world'
../../../../scalapack/BLACS/SRC/MPI/sys2blacs_handle_.c:(.text+0x17d): undefined                                                                                                  reference to `ompi_mpi_comm_world'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.                                                                                                 a(BI_MPI_F77_to_c_trans_comm.o): In function `BI_MPI_F77_to_c_trans_comm':
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_MPI_F77_to_c_trans_comm.c:(.text                                                                                                 +0x6): undefined reference to `MPI_Comm_f2c'
make: *** [quest_265c.x] Error 1
0
Comment
Question by:ted_yu
  • 8
  • 7
15 Comments
 
LVL 62

Expert Comment

by:gheist
ID: 40392844
I think you have mismatch between header and object files
You need to either build both C and F with -fopenmp or without.
0
 

Author Comment

by:ted_yu
ID: 40392876
thanks, just to clarify, do you mean to build the software...or do you mean building the library that it is referencing?
0
 
LVL 62

Expert Comment

by:gheist
ID: 40392896
I think your fortran program expectc C library to be linked with openmp library (would be nice to hear what compiler you use and what OS...)
0
Windows Server 2016: All you need to know

Learn about Hyper-V features that increase functionality and usability of Microsoft Windows Server 2016. Also, throughout this eBook, you’ll find some basic PowerShell examples that will help you leverage the scripts in your environments!

 

Author Comment

by:ted_yu
ID: 40420612
Sorry for the delay.  Here is the makefile.  The OS is redhat.

qvrs = 265c

FC = mpif77
FFLAGS = -O2 -xSSE3 -mcmodel medium -shared-intel

LFLAGS = -mcmodel medium -shared-intel
LIBS =

MKL_LIB = /home/rtorres/intel/mkl/lib/

MATH = -L${MKL_LIB} -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5

PMATH = -L${MKL_LIB} -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64

SUBSU = utl_mpi.o

MVRSTP=lcaomain_tp.o

SUBS=lcaosubs.o
TPSUBS=lcaosubs_tp.o

quest: quest_${qvrs}.x

help:
        @echo "Make SeqQuest program - openmpi parallel"
        @echo "Usage: make <platform>, only platform available: quest"
        @echo "For parallel builds, you will have to specify library locations in makefile"
        @echo ""

clean:
        rm -f *.o quest_${qvrs}.x

quest_${qvrs}.x : ${MVRSTP} ${SUBS} ${TPSUBS} ${SUBSU}
        $(FC) $(LFLAGS) -o $@  ${MVRSTP} ${SUBSU} ${TPSUBS} ${SUBS} ${PMATH} ${MATH}
0
 
LVL 62

Expert Comment

by:gheist
ID: 40421021
Just two builds disagree about their use of openmpi (I was mistaken about openmp)
So if you want to schedule program on multi-machine cluster - build all C parts with openmpi support, if not remove openmpi references from F99 makefile.
0
 

Author Comment

by:ted_yu
ID: 40421134
Unfortunately, I am not the author of this program, and I just need to get this thing compiled on my machine.  This program has been successfully compiled on other machines.  Here is the comment from the author of the program about my difficulties:

"This can happen if scalapack and BLACS are not built correctly (although I though this issue was taken care of in newer versions of mkl): scalapack is written in fortran and BLACS in C and one needs to construct a solver build which allows the two to cross boundaries between the language (the infamous underscore—double-underscore problem).  The default build of the solvers did not include the expanded symbol table that allow a Fortran code to call the mixed-language solver libraries.   You might look into that build to be sure that the symbol tables have the redundancy in them that allow this.  As I said, I though this had been “fixed” in the mkl that Intel ships, but perhaps not."

I cannot rewrite the code, so the best way to solve this, as I understand it is to include lines on my makefile such as:
 -assume 2underscores
-fsecond-underscore

If I can understand these things a little better, I can try to fix the problem by changing my makefile.

The mkl math library I am using is by Intel and is already installed in the cluster I am using.  I can try to recompile it as another option.

So, the only thing I can change at the moment are:
1)  the makefile
2)  recompiling the mkl library which contains blacs and scalpack differently.
0
 
LVL 62

Expert Comment

by:gheist
ID: 40421348
Change (i.e comment out):
# PMATH = -L${MKL_LIB} -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64
To (i.e add new line):
PMATH = -L${MKL_LIB} -lmkl_scalapack_lp64

You will lose ability to run product on multi-node cluster.
Ahouls you need to run it on multi-node cluster you need to build blacs with openmpi support, and then build your F99 program using original Makefile..
0
 

Author Comment

by:ted_yu
ID: 40422172
Now I'm getting errors that scalapack needs references that are not there (from blacs)

/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `zgesd2d_'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `Cblacs_barrier'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `dtrbs2d_'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `ctrbr2d_'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `dcputime00_'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `Cigsum2d'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `igebs2d_'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `strrv2d_'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `blacs_pnum_'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `MKL_DOUBLE'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `MKL_MAXLOC'
make: *** [quest_265c.x] Error 1

I also need to run this on multi-nodes.  Do you have any idea what it means to look for the  following that was mentioned?  " The default build of the solvers did not include the expanded symbol table that allow a Fortran code to call the mixed-language solver libraries. "  What is this mixed-language solver library and where is it located?
0
 
LVL 62

Expert Comment

by:gheist
ID: 40422221
Take it ready on EL7+EPEL, no need to take long leaps to re-invent the wheel.
http://mirror.uv.es/mirror/fedora-epel/7/x86_64/repoview/blacs-openmpi-devel.html
0
 

Author Comment

by:ted_yu
ID: 40422383
OK.  I downloaded that file and put it in the cluster.  I may not have changed the Makefile correctly, because I'm still getting an error.  Here are my changes to the makefile.

BLACS_LIB = /shares/teamyu/usr/lib64/openmpi/lib/

PMATH = -L${MKL_LIB} -lmkl_scalapack_lp64 -L${BLACS_LIB} -lmkl_blacs_openmpi_lp64

This is going back to the original error, so it's still trying to use BLACS from the mkl library.:
../../../../scalapack/BLACS/SRC/MPI/sys2blacs_handle_.c:(.text+0x17d): undefined                                                                                                              reference to `ompi_mpi_comm_world'
/usr/opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.                                                                                                             a(BI_MPI_F77_to_c_trans_comm.o): In function `BI_MPI_F77_to_c_trans_comm':
../../../../scalapack/BLACS/SRC/MPI/INTERNAL/BI_MPI_F77_to_c_trans_comm.c:(.text                                                                                                             +0x6): undefined reference to `MPI_Comm_f2c'
make: *** [quest_265c.x] Error 1
0
 
LVL 62

Expert Comment

by:gheist
ID: 40422756
I was trying to talk you into getting blacs+scalapack as installable RPMs (you know desktop virtualisation ant the stuff for testing)
EPEL RPMs are build with MPI support, so RHEL7+EPEL will sustain your build.
btw no need to target SSE3 - it is so past, --march-native gives bigger bang for the buck.
0
 
LVL 62

Expert Comment

by:gheist
ID: 40422760
Aw pain - you use ICC. You must build both components with same ICC. What I am suggesting is to make POC build with GCC.
0
 

Author Comment

by:ted_yu
ID: 40422988
I was advised to use Intel mkl because it is the fastest by a factor of two.  I would like to use it if possible.  Any chance to fix this with the underscore issue?  

 -assume 2underscores
-fsecond-underscore

Is it possible scalapack is not calling blacs correctly and it is just an easy fix of the settings?
0
 
LVL 62

Accepted Solution

by:
gheist earned 500 total points
ID: 40423339
BLACS is not build with openmpi support.
SSE3 is 10 years old. Are you certain GCC does not generate that?
0
 

Author Comment

by:ted_yu
ID: 40458709
OK, I've given up on using Intel MKL for this build and am using a different library.  Thanks for your help.
0

Featured Post

Simplifying Server Workload Migrations

This use case outlines the migration challenges that organizations face and how the Acronis AnyData Engine supports physical-to-physical (P2P), physical-to-virtual (P2V), virtual to physical (V2P), and cross-virtual (V2V) migration scenarios to address these challenges.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Today companies are subjected to more-and-more data, and it won't stop any time soon.  But there are obvious opportunities for reducing data, particularly data duplicated among companies.
This guide will walk you through the essential considerations and tech stack for building scalable websites. Know how to grow your business the smart way!
The viewer will learn how to successfully download and install the SARDU utility on Windows 7, without downloading adware.
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

830 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question