Cross Compiling Python for Embedded Linux

This article details how to cross compile Python for the PowerPC platform.  It should apply equally to other platforms such as ARM, just plug in the correct cross-compiler.  The article supports Python versions, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.6.5 and 3.1.1, 3.1.2.

Firstly, download the version of Python that you want to use from http://www.python.org/

Unpack the Python package using tar:

tar -xvzf Python-2.6.5.tgz

This will create a directory called Python-2.6.5.  Goto the directory:

cd Python-2.6.5

Then run these commands to build the host components:

./configure

make python Parser/pgen

mv python hostpython

mv Parser/pgen Parser/hostpgen

make distclean

Download the correct patch for your version of Python:

Then apply the patch:

patch -p1 < Python-2.6.5-xcompile.patch

Then run this (where ~/Python-2.6.5/_install/ is your desired installation path)

CC=ppc_6xx-gcc CXX=ppc_6xx-g++ AR=ppc_6xx-ar RANLIB=ppc_6xx-ranlib ./configure --host=ppc-linux --build=i686-pc-linux-gnu --prefix=/python

make HOSTPYTHON=./hostpython HOSTPGEN=./Parser/hostpgen BLDSHARED="ppc-linux-gcc -shared" CROSS_COMPILE=ppc_6xx- CROSS_COMPILE_TARGET=yes

make  install HOSTPYTHON=./hostpython BLDSHARED="ppc-linux-gcc -shared" CROSS_COMPILE=ppc_6xx- CROSS_COMPILE_TARGET=yes prefix=~/Python-2.6.5/_install

This will install all your python binaries and libraries in ~/Python-2.6.5/_install.

Copy the entire _install directory to the device, setup the PATH environment variable to include the location of the Python executable and run:

python lib/Python-2.6/test/test___all___.py

…and hopefully all the tests will run correctly.

To speed up the importing of Python modules on the target, it is recommended to zip up the lib directory to make a file called python26.zip.  This means that we do not have to copy all the individual Python files to the target, just the one zip file.  This technique might also save space, but that depends on your file system.  On the host machine:

cd _install/lib/python2.6

zip -r -y python26.zip .

Delete libpythonxxx.a site-packages, lib-dynload, config and anything else you do not require from the python26.zip file.

Copy the _install/bin/python to the /usr/bin directory on the target:

Copy the python26.zip file to the /usr/lib directory on the target:

Create a directory on the target called python2.6 in the /usr/lib directory and copy the following directories to that directory:

  • ./lib/python2.6/config
  • ./lib/python2.6/lib-dynload
  • ./lib/python2.6/site-packages

Your directory structure on the target should be as follows:

/usr/lib # ls

python2.6  python26.zip

/python/lib/python2.6 # ls

config  lib-dynload  site-packages

Set the PYTHONHOME environment variable to /usr/ and you are ready to run Python on the target.

The above patches are based on Chris Lambacher’s patches for Python 2.5 here:

http://whatschrisdoing.com/blog/2006/10/06/howto-cross-compile-python-25/

Other links and credits:

http://www.ailis.de/~k/archives/19-ARM-cross-compiling-howto.html

Bookmark and Share

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

16 Comments Post a Comment
  1. Thanks for the 3.1/2.6 patch updates. I have some new notes on how to cross compile extensions.

  2. Todd says:

    Just did this for an ARM computer I have – problem now is the result in _install is 75MB – my machine has 64 MB.

    Any thoughts on optimizing/throwing out whats not needed?

  3. Paul Gibson says:

    The _install directory is really just for initial testing as it contains lots of extra build artifacts. I believe it is much better and faster to use the zip method of deploying to your final system.

    Having said this, here are some ways to get the size down for both methods:

    cd _install
    rm -rf include
    rm -rf share

    cd lib
    rm libpython2.6.a

    cd lib/python2.6
    rm -rf `find . -name test`
    rm `find . -name *.pyo`
    rm `find . -name *.py`

    cd ../../bin
    rm python2.6

    And strip the python executable. On powerpc, the command is:

    ppc_6xx-strip python

    This gets my image down to 20M. Now you can delete modules that you do not think you’ll use. Look in the lib/python2.6 directory and delete any subdirectory that you do not think you’ll use. Example of less used modules would be:
    – email, xml, sqlite3, curses, idlelib, encodings, lib2to3, lib-tk, etc.

    You may even to able to remove the config directory. I removed it and had success.

    This got me down to 11M.

    The last piece of optimization would be the lib-dynload directory. Some of these shared objects would definitely be needed, but not all.

    So, after being very aggressive my _install image is 6M.

  4. Todd says:

    Okay, manually copying each directory was much smaller than 75MB of the tarball, more like 5-10MB. Thanks!

  5. Amit Yedidia says:

    Hi, Thanks for the great tutorial.
    I try following it but I got few problems that I hope you can help with.
    I am compiling Python-2.6.2 on a Ubuntu-x86 and my target is linux on imx27(arm9)

    first, when I compile I got some errors during the compilation and in the end I got:
    ——————————————————
    errors
    Failed to find the necessary bits to build these modules:
    _bsddb _hashlib _sqlite3
    _ssl _tkinter bsddb185
    bz2 dbm gdbm
    readline sunaudiodev
    To find the necessary bits, look in setup.py in detect_modules() for the module’s name.

    Failed to build these modules:
    _curses _curses_panel binascii
    —————————————————————-

    second after I ignored those errors and installed the file on my target, when I am running the test script I got:
    ———————————————————-
    root@freescale /$ python -v /lib/python2.6/test/test___all__.py
    # installing zipimport hook
    import zipimport # builtin
    # installed zipimport hook
    ‘import site’ failed; traceback:
    ImportError: No module named site
    Python 2.6.2 (r262:71600, Nov 3 2009, 21:04:45)
    [GCC 4.1.2] on linux2
    Type “help”, “copyright”, “credits” or “license” for more information.
    Traceback (most recent call last):
    File “/lib/python2.6/test/test___all__.py”, line 1, in
    import unittest
    ImportError: No module named unittest
    # clear __builtin__._
    # clear sys.path
    # clear sys.argv
    # clear sys.ps1
    # clear sys.ps2
    # clear sys.exitfunc
    # clear sys.exc_type
    # clear sys.exc_value
    # clear sys.exc_traceback
    # clear sys.last_type
    # clear sys.last_value
    # clear sys.last_traceback
    # clear sys.path_hooks
    # clear sys.path_importer_cache
    # clear sys.meta_path
    # clear sys.flags
    # clear sys.float_info
    # restore sys.stdin
    # restore sys.stdout
    # restore sys.stderr
    # cleanup __main__
    # cleanup[1] zipimport
    # cleanup[1] signal
    # cleanup[1] exceptions
    # cleanup[1] _warnings
    # cleanup sys
    # cleanup __builtin__
    # cleanup ints: 3 unfreed ints
    # cleanup floats
    ——————————————————————

    I am attaching also the configure+make+install commands:
    —————————————————————]
    CC=arm-none-linux-gnueabi-gcc CXX=arm-none-linux-gnueabi-g++ AR=arm-none-linux-gnueabi-ar RANLIB=arm-none-linux-gnueabi-ranlib ./configure –host=arm-linux –build=i686-pc-linux-gnu –prefix=/python
    make HOSTPYTHON=./hostpython HOSTPGEN=./Parser/hostpgen BLDSHARED=”arm-none-linux-gnueabi-gcc -shared” CROSS_COMPILE=arm-none-linux-gnueabi- CROSS_COMPILE_TARGET=yes
    make install HOSTPYTHON=./hostpython BLDSHARED=”arm-none-linux-gnueabi-gcc -shared” CROSS_COMPILE=arm-none-linux-gnueabi- CROSS_COMPILE_TARGET=yes prefix=~/workspace/DVE/Http/Python-2.6.2/_install
    ————————————————————

  6. Amit Yedidia says:

    Hi,

    I found the problem, it was all about paths.
    now I am facing only one probelm. since the binascii module fail to build lots of test cant run. can you figure why it happens?

  7. Paul Gibson says:

    Hi Amit,

    Can you post the error that the test cases give (for binascii)? Ubuntu is probably missing the package you need to compile it properly. I couldn’t find a binascii package, but it could be part of another package. The error message should help me.

    Thanks,
    Paul

  8. Rachel says:

    Hi,
    A huge thanks for this – worked a treat. I’ve made a 2.6.4 patch if you want it, email me and I’ll send it.

    Being new to Python, I’m still trying to get my head around cross compiling the modules side of it. Looking at the setup.py script, it seems to be looking for include files and library files to link to the modules on the host system not the target. Which I guess is why it’s not able to build modules for which I have valid libraries eg ncurses and readline. Amit, what paths did you change? Anything in setup.py?
    Can anyone give any experience of cross compiling modules for python?
    Did you need any extra patches? Did you modify Setup.local or something else to add your modules to?
    Thanks
    Rachel

  9. nguyen tien dat says:

    Hi Rachel!

    I’ve tried to find on google a patch for cross-compiling python2.6.4 but there is no result. Could you email for me your patch. My email is nguyentiendat05@gmail.com.

    Many thanks for you!

  10. Paul Gibson says:

    Just updated the post for Python 2.6.4.

  11. Anonymous says:

    Great tutorial! If you’re cross-compiling in OS X, the python executable is actually called python.exe (since OS X filenames are case-insensitive). So the make and mv commands should be:

    make python.exe Parser/pgen

    mv python.exe hostpython

  12. Nguyen Thai Binh says:

    Thank you for your work for cross compile python.I followed your instructions to cross compile python2.6.4 for powerpc405 (i used eldk as cross compile tool). The compilation process was ok, and no errors occured.

  13. I was expecting a trackback to pop up, but I posted about using your technique and getting functional _ctypes support in the bargain. It occurs to me you could incorporate the technique into future patches.

  14. Paul Gibson says:

    Thanks Art, appreciate the effort. I will test and incorporate into my patches.

  15. Momo says:

    Message for Nguyen Thai Binh: Can you contact me on my email or give me yours: snice1981@gmail.com

    I would like to use ELDK also but I don’t know how. If you could give me some tips, it would be greate :-)

    Thanks

  16. Wladimir says:

    Thanks for the great explanation and patches. Guido, this should be in the mainline! To encourage the use of Python in embedded development it should be as easy as possible to cross-compile it (at least, no source code patching should be needed).

Leave a Reply