Creating a C external function for Python, an easier way

Colin Paice

Aug 31

I wrote about my first steps in creating a C extension in Python. Now I've got more experience, I've found an easier way of compiling the program and creating a load module. It is not the official way - but it works, and is easier to do!

The traditional way of building a package is to use the setup.py technique. I've found just compiling it works just as well (and is slighly faster). You still need the setup.py for building Python source.

I set up a cp4.sh file

name=zos  pythonSide='/usr/lpp/IBM/cyp/v3r8/pyz/lib/python3.8/config-3.8/libpython3.8.x'  export _C89_CCMODE=1  p1=" -DNDEBUG -O3 -qarch=10 -qlanglvl=extc99 -q64"  p2="-Wc,DLL -D_XOPEN_SOURCE_EXTENDED -D_POSIX_THREADS"  p2="-D_XOPEN_SOURCE_EXTENDED -D_POSIX_THREADS"  p3="-D_OPEN_SYS_FILE_EXT                     -qstrict          "  p4="-Wa,asa,goff -qgonumber -qenum=int"  p5="-I//'COLIN.MQ930.SCSQC370' -I. -I/u/tmp/zpymqi/env/include"  p6="-I/usr/lpp/IBM/cyp/v3r8/pyz/include/python3.8"  p7="-Wc,ASM,EXPMAC,SHOWINC,ASMLIB(//'SYS1.MACLIB'),NOINFO "  p8="-Wc,LIST(c.lst),SOURCE,NOWARN64,FLAG(W),XREF,AGG -Wa,LIST,RENT"  /bin/xlc $p1 $p2 $p3 $p4 $p5 $p6 $p7 $p8  -c $name.c -o $name.o  -qexportall -qagg -qascii  l1="-Wl,LIST=ALL,MAP,XREF     -q64"  l1="-Wl,LIST=ALL,MAP,DLL,XREF     -q64"  /bin/xlc $name.o  $pythonSide  -o $name.so  $l1 1>a 2>b  oedit a  oedit b

This shell script creates a zos.so load module in the current directory.

You need to copy the output load module (zos.so) to a directory on the PythonPath environment variable.

What do the parameters mean?

Many of the parameters I blindly copied from the setup.py script.

name=zos
- This parametrizes the script, for example $name.c $name.o $name.so
pythonSide='/usr/lpp/IBM/cyp/v3r8/pyz/lib/python3.8/config-3.8/libpython3.8.x'
- This is where the python side deck, for resolving links the to functions in the Python code
export _C89_CCMODE=1
- This is needed to prevent the message "FSUM3008 Specify a file with the correct suffix (.c, .i, .s,.o, .x, .p, .I, or .a), or a corresponding data set name, instead of -o./zos.so."
p1=" -DNDEBUG -O3 -qarch=10 -qlanglvl=extc99 -q64"
- -O3 optimization level
- -qarch=10 is the architectural level of the code to be produced.
- --qlanglvl=extc99 says use the C extensions defined in level 99. (For example defining variables in the middle of a program, rather that only at the top)
- -q64 says make this a 64 bit program
p2="-D_XOPEN_SOURCE_EXTENDED -D_POSIX_THREADS"
- The C #defines to preset
p3="-D_OPEN_SYS_FILE_EXT -qstrict "
- -qstrict Used to prevent optimizations from re-ordering instructions that could introduce rounding errors.
p4="-Wa,asa,goff -qgonumber -qenum=int"
- -Wa,asa,goff options for any assembler compiles (not used)
- -qgonumber include C program line numbers in any dumps etc
- -qenum=int use integer variables for enums
p5="-I//'COLIN.MQ930.SCSQC370' -I. -I/u/tmp/zpymqi/env/include"
- Where to find #includes:
- the MQ libraries,
- the current working directory
- the header files for my component
p6="-I/usr/lpp/IBM/cyp/v3r8/pyz/include/python3.8"
- Where to find #includes
p7="-Wc,ASM,EXPMAC,SHOWINC,ASMLIB(//'SYS1.MACLIB'),NOINFO "
- Support the use of __ASM().. to use inline assembler code.
- Expand macros to show what is generated
- List the data from #includes
- If using__ASM__(...) where to find assembler copy files and macros.
- Do not report infomation messages
p8="-Wc,LIST(c.lst),SOURCE,NOWARN64,FLAG(W),XREF,AGG -Wa,LIST,RENT"
- For C compiles, produce a listing in c.lst,
- include the C source
- do not warn about problems with 64 bit/31 bit
- display the cross references (where used)
- display information about structures
- For Assembler programs generate a list, and make it reentrant
/bin/xlc $p1 $p2 $p3 $p4 $p5 $p6 $p7 $p8 -c $name.c -o $name.o -qexportall
- Compile $name.c into $name.o ( so zos.c into zos.o) export all entry points for DLL processing
L1="-Wl,LIST=ALL,MAP,DLL,XREF -q64"
- bind pararameters -Wl, produce a report,
- show the map of the module
- show the cross reference
- it is a 64 bit object
/bin/xlc $name.o $pythonSide -o $name.so $L1 1>a 2>b
- take the zos.o, the Python side deck and bind them into the zos.so
- pass the parameters defined in L1
- output the cross reference to a and errors to b
oedit a
- This will have the map, cross reference and other output from the bind
oedit b
- This will have any error messages - it should be empty