Thursday, 26 June 2014

Debugging Python extensions in Eclipse

To accelerate your Python code it may be helpful to implement some of the  computationally intensive routines in C++. This is what makes Python a powerful tool for prototyping numerical algorithm that extensively use numpy and scipy routines that were implemented in C++ or Fortran. One way to do it, would be to follow the Python/C API Reference Manual, but unless you want to do some highly specific stuff like specific memory management, swig may be just right for you.

One problem is, how to deal with debugging of such code where you have the combination of script language and a shared library. This post focuses on my experience in setting up a proper developement environment

 developing a C++ code that you compile to a shared library make available as a Python plugin using swig. 


Step 1: Setup the environment

Basically, you need to get the latest Eclipse CDT development environment and install the PyDev plugin for python development. Also install swig binaries. You would also need the header files for Python:

sudo apt-get install swig python-dev


Step 2: Install python-dbg and all supporting numerical packages



This step depends on your operation system and code you are working on. Supposed, like me, you are working on Ubuntu and you are using numpy and scipy  routines in you Python  code:

sudo apt-get install python-dbg python-numpy-dbg python-scipy-dbg


Step 3: Compile your C++ code

Use flags -g -O0 -I/.../python_d/






swig -o example_wrap.cc -c++ -python example.i
gcc -o _example.so -fPIC -g -O0 -shared -I/.../python_d/ -I/usr/include/python2.7 example_wrap.cc

You will get files example.py and _example.so. For simplicity, we will keep them in the same directory with our python code.


Step 4. Debugging

Create a breakpoint in python code and create a debug configuration with python-dbg as interpreter
5. Create the debugging configuration with c++ application
6. Run python debugging configuration
7. Run CDT debugging configuration and hang the listener to your python-dbg process

Some weird behaviour I have encountered

1. If you watch expression and you break point is placed a the line before the expressions can be evaluated, c++ debugger fails and just jumps back to the python debugging process, usually before you can understand what actually happens. Just remove the expressions or change the break point.