Skip to content
 

Cross Compiling Python for Embedded Linux

This article details how to cross compile Python for the ARM and PowerPC platforms.  It should apply equally to other platforms as well, just plug-in the correct cross-compiler.  The article supports Python versions, 2.6.6 (Thanks to evadeflow), 2.7.2, 2.7.3 (thanks to Lothsahn) and 3.1.1, 3.1.2, 3.1.3, 3.2.2.  For version 2.7.5, please refer to Trevor Bowen’s page.

This page has been translated into http://www.webhostinghub.com/support/es/misc/cruzada-para-linux“>Spanish</a> language by Maria Ramos  from <a href=”http://www.webhostinghub.com/support/edu>Webhostinghub.com/support/edu</a>.

Firstly, download the Python that you want to use from http://www.python.org/

Unpack the Python package using tar:

tar -xvzf Python-2.7.2.tgz

This will create a directory called Python-2.7.2.  Goto the directory:

cd Python-2.7.2

Then run these commands to build the host components:

./configure

make python Parser/pgen

mv python hostpython

mv Parser/pgen Parser/hostpgen

make distclean

Download the correct patch for your version of Python:

Then apply the patch:

patch -p1 < Python-2.7.2-xcompile.patch

Then run this (where ~/Python-2.7.2/_install/ is your desired installation path).  Note also that you must replace all instances of the cross compiler and the host build system in the lines below.  If you are on an x86_64 machine, then you should use x86_64-linux-gnu as the host.

CC=ppc_6xx-gcc CXX=ppc_6xx-g++ AR=ppc_6xx-ar RANLIB=ppc_6xx-ranlib ./configure --host=ppc-linux --build=x86_64-linux-gnu --prefix=/python

make HOSTPYTHON=./hostpython HOSTPGEN=./Parser/hostpgen BLDSHARED="ppc_6xx-gcc -shared" CROSS_COMPILE=ppc_6xx- CROSS_COMPILE_TARGET=yes HOSTARCH=ppc-linux BUILDARCH=x86_64-linux-gnu

make install HOSTPYTHON=./hostpython BLDSHARED="ppc_6xx-gcc -shared" CROSS_COMPILE=ppc_6xx- CROSS_COMPILE_TARGET=yes prefix=~/Python-2.7.2/_install

This will install all your python binaries and libraries in ~/Python-2.7.2/_install.

Copy the entire _install directory to the device, setup the PATH environment variable to include the path the Python executable and run:

python lib/python-2.7/test/test___all__.py

…and hopefully all the tests will run correctly.

To speed up the importing of Python modules on the target, I recommend to zip up the lib directory to make a file called python27.zip.  This means that we do not have to copy all the Python files to the target, just the one zip file.  This technique might also save space, but that depends on your file system.  On the host machine:

cd _install/lib/python2.7

zip -r -y python27.zip .

Delete libpythonxxx.a site-packages, lib-dynload, config and anything else you do not need from the python27.zip file.

Copy the _install/bin/python to the /usr/bin directory on the target:

Copy the python27.zip file to the /usr/lib directory on the target:

Create a directory on the target called python2.7 in the /usr/lib directory and copy the following directories to that directory:

  • ./lib/python2.7/config
  • ./lib/python2.7/lib-dynload
  • ./lib/python2.7/site-packages

Your directory structure on the target must be as follows:

/usr/lib # ls

python2.7 python27.zip

/python/lib/python2.7 # ls

config lib-dynload site-packages

Set the PYTHONHOME environment variable to /usr/ and you are ready to run Python on the target.

 

Building SQLLite and other dependancies

Thanks to Lothsahn’s 2.7.3 patch, it is now possible to build sqlite3, bz2, gz, ssl and ctypes.  Lothsahn provides the following instructions:

To do the compilation, the dependencies must be in the include and lib folder one path up from the python install.   For example:

<base directory>/include <– header files go here

<base directory>/lib <– shared object files go here

/Python-2.7.3 <—Python sources go here during compile

Dependencies include libffi, sqlite, openssl, bzip2, zlib, etc.

 

Credits

The above patches are based on Chris Lambacher’s patches for Python 2.5 here:

http://whatschrisdoing.com/blog/2006/10/06/howto-cross-compile-python-25/

Other links and credits:

http://www.ailis.de/~k/archives/19-ARM-cross-compiling-howto.html

191 Comments

  1. KyleL says:

    Thanks for this writeup. I’m trying to get Python2.7.3 working with ctypes on MIPS. However I seem to be facing a problem where the ctypes module fails to configure:

    checking build system type… Invalid configuration `None’: machine `None’ not recognized
    configure: error: /bin/sh /home/phase6/xc_python/Python-2.7.3/Modules/_ctypes/libffi/config.sub None failed
    Failed to configure _ctypes module

    Any ideas?

    Thanks,
    Kyle

  2. Abhishek says:

    I want to cross-compile Python for a embedded system. The target is a PPC but DOES NOT run linux. Instead it runs a lesser known though capable embedded OS and I was wondering if you have attempted porting Python to a non-linux platform before.

    I have the cross compilation tool chain installed on a linux machine so I can run the configure script pointing to the target OS headers and libraries.

    Do you think this porting would be possible?

  3. Myles Dear says:

    Thanks for the instructions. They got me 90% of the way there for my Python 2.7.3 cross-compilation onto a PPC 85xx Wind River Linux 3.0 system.

    One small problem : my toolchain requires both CPPFLAGS and LDFLAGS to be set.

    In my top-level Makefile, I had to temporarily move the target’s “Makefile” and “pyconfig.h” to the host Python’s installation directory since the hostpython’s sysconfig module builds an environment variable database from these files, and these variables are used by setup.py to set CPPFLAGS and LDFLAGS to compile the Python extensions.

    I’m sure there is a more elegant solution, but this did the trick and I’m trying hard not to do any more patches than necessary to the Python distribution beyond what has been already published.

  4. Jean-Philippe Halimi says:

    Thanks for the instructions ! I’ve used them in order to cross-compile Python for Xeon Phi. I have made a repository which features the patch (https://github.com/jphalimi/PythonForMIC). Is there any problem for you ? Do you have any particular license for these files ?

  5. Vincent says:

    First, thanks for your infomation. When I compile python 2.7.3 for mips-platform, using Python-2.7.3-xcompile.patch ( Thanks to Lothsahn), I meet with such problems(after make command):
    Python build finished, but the necessary bits to build these modules were not found:
    _bsddb _curses _curses_panel
    _sqlite3 _ssl _tkinter
    bsddb185 bz2 dbm
    dl gdbm nis
    readline sunaudiodev zlib
    To find the necessary bits, look in setup.py in detect_modules() for the module’s name.
    My host system is Ubuntu 10.04 32 bit, and cross-compiler is mips-wrs-linux-gnu-mips_74k_softfp-glibc_small-xxx,
    I tried install some libs: libreadline-dev libreadline6 libssl-dev, libbz2-dev, libgdbm-dev, libreadline5-dev? libsqlite3-dev and so on, but nothing different. I don’t understand why.
    My parameters:
    CC=mips-wrs-linux-gnu-mips_74k_softfp-glibc_small-gcc CXX=mips-wrs-linux-gnu-mips_74k_softfp-glibc_small-g++ AR=mips-wrs-linux-gnu-mips_74k_softfp-glibc_small-ar RANLIB=mips-wrs-linux-gnu-mips_74k_softfp-glibc_small-ranlib ./configure –host=mips-linux –build=i686-linux-gnu –prefix=/python

    make HOSTPYTHON=./hostpython HOSTPGEN=./Parser/hostpgen BLDSHARED=”mips-wrs-linux-gnu-mips_74k_softfp-glibc_small-gcc -shared” CROSS_COMPILE=mips-wrs-linux-gnu-mips_74k_softfp-glibc_small- CROSS_COMPILE_TARGET=yes HOSTARCH=mips-linux BUILDARCH=i686-linux-gnu

    • Paul Gibson says:

      Hi Vincent,

      Your build has succeeded, those modules are optional. Do you really need these modules? You cannot fix them by installing host dev packages, you actually need to cross-compile them separately using the setup.py scripts which is an advanced task. There should be a post somewhere on this page that shows instructions for sqlite3.

      If you need the other modules, I recommend you skip this page and use buildroot.

      Paul

      • Vincent says:

        Thanks to Paul Gibson, that is very useful!

      • Vincent says:

        Hi Paul Gibson,
        I have some good news! What I really need is zlib and readline modules, there is a way to build them correctly.
        I have printed some parameters in setup.py, e.g. do_readline = self.compiler.find_library_file(lib_dirs, ‘readline’), and do_readline is None.
        I knew the readline library was localed in /usr/lib, by running command:
        dpkg -L libreadline6-dev

        /usr/lib/libreadline.a

        So, I modified the line to this:
        do_readline = self.compiler.find_library_file(lib_dirs+['/usr/lib'], ‘readline’)
        and make clean, remake it. It works, _curses.so and readline.so appeared in lib/python2.7/lib-dynload floder.
        This method is available to zlib too.
        Now, readline and zlib are both work on my mips-platform!

  6. Sam Jones says:

    Hi Paul,

    I was wondering if you could help me out here. First of all though, thanks for the great work here. I came across these instructions via a script I found while googling: https://github.com/wewei/Build-Python-for-iOS

    I have the libpython2.7.a in my XCode iOS app, but upon Py_Initialize(); I get these errors (with PYTHONVERBOSE=1):

    loadPyFile: 1
    # installing zipimport hook
    import zipimport # builtin
    # installed zipimport hook
    ImportError: No module named site
    # clear __builtin__._
    # clear sys.path
    # clear sys.argv
    # clear sys.ps1
    # clear sys.ps2
    # clear sys.exitfunc
    # clear sys.exc_type
    # clear sys.exc_value
    # clear sys.exc_traceback
    # clear sys.last_type
    # clear sys.last_value
    # clear sys.last_traceback
    # clear sys.path_hooks
    # clear sys.path_importer_cache
    # clear sys.meta_path
    # clear sys.flags
    # clear sys.float_info
    # restore sys.stdin
    # restore sys.stdout
    # restore sys.stderr
    # cleanup __main__
    # cleanup[1] zipimport
    # cleanup[1] signal
    # cleanup[1] exceptions
    # cleanup[1] _warnings
    # cleanup sys
    # cleanup __builtin__
    # cleanup ints: 5 unfreed ints
    # cleanup floats

    The point at which it crashes is, “ImportError: No module named site”. Am I correct in thinking that shared modules are compiled with the static library rather than being .dylib (disallowed on iOS as far as I know). If so, why can my interpreter not find the modules?!

    I’m really at a loss here. Any help would be greatly appreciated. Thanks in advance

    • Paul Gibson says:

      Hi Sam,

      I’m not an iOS expert, but on Linux this normally means that your PYTHONHOME variable is not set or incorrect. I’m not sure the exact format you should use on iOS, but google should help if you search for your error message and PYTHONHOME.

      Good luck!
      Paul

  7. Carl says:

    Hi,

    First, thanks for the tutorial. I cross-compiled python 2.7.3, and using the “file” command on the python2.7 executable, I get: “ELF 32-bit MSB executable, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.0.0, not stripped”.

    However, I get an error when I dump it on the target and try to execute it. It says:

    ~# python
    /bin/python: line 1: syntax error: unexpected “(”

    Any idea why it does that? I would expect to get the python interactive prompt.

  8. Chris Rolls says:

    Hi paul:

    Do you know if this method will work for a Embedded computer running linux2.6.9 which needs the arm-linux-gcc cross compiler.
    Im trying to get python3.2.3 onto the EC so i can run some python code.

    If i just want to run python code on the EC do i need python installed?

    I have tried cx_freeze but this did not work. |It could not read the binary file.

    Sorry im new to linux :(

    Thanks chris

    • Paul Gibson says:

      Hi Chris,

      Yes, this method should work on Linux 2.6.9 if you have the correct cross compiler.

      I googled your question about python installation and got the following answer:
      http://stackoverflow.com/questions/5539736/python-can-i-run-a-python-script-without-actually-installing-python

      cx_freeze should work, but you need to make sure its generating binaries for arm. If it generates x86 binaries, then it will not run.

      Paul

      • Chris Rolls says:

        cheers for the reply paul. Could i be annoying and ask you to explain what you mean by creating the correct binaries and how to check? or maybe give me a direction for things i neeed to learn?

        Thanks in advance

        • Paul Gibson says:

          Hi Chris,

          If you are using cx_freeze on your desktop PC, then it is probably creating a binary that will run on the PC (eg x86 or x86_64 architecture). The ARM architecture is different and will not run x86/x86_64 binaries.

          You can check if the binary is ARM compatible by running the readelf command:
          $ readelf -A program

          Attribute Section: aeabi
          File Attributes
          Tag_CPU_name: “7-A”
          Tag_CPU_arch: v7
          Tag_CPU_arch_profile: Application
          Tag_ARM_ISA_use: Yes
          Tag_THUMB_ISA_use: Thumb-2
          Tag_FP_arch: VFPv3
          Tag_Advanced_SIMD_arch: NEONv1
          Tag_ABI_PCS_wchar_t: 4
          Tag_ABI_FP_rounding: Needed
          Tag_ABI_FP_denormal: Needed
          Tag_ABI_FP_exceptions: Needed
          Tag_ABI_FP_number_model: IEEE 754
          Tag_ABI_align_needed: 8-byte
          Tag_ABI_align_preserved: 8-byte, except leaf SP
          Tag_ABI_enum_size: int
          Tag_ABI_HardFP_use: SP and DP
          Tag_ABI_VFP_args: VFP registers
          Tag_CPU_unaligned_access: v6
          Tag_Virtualization_use: TrustZone

          Paul

    • Lothsahn says:

      Chris:

      2.6.9 is very very old. I was not able to get this to compile correctly on a device running 2.6.8 because the toolchain was very old and not compliant with some C standards. Consider yourself warned.

      You will likely be able to get it to compile (I did), but make sure you run the Python testcases. They were the ones that warned me that Python was very very broken. It was a direct result of the kernel age and toolchain problems.

  9. Dustin Gooding says:

    Thanks for the post Paul. Very helpful.

    I’m attempting to build the bz2 dependency and am running into a problem. Perhaps you could give some advice?

    I’m running Ubuntu 12.04 64, and building for an ARM Linux target. I’ve got a copy of my sysroot at ~/digi-sysroot, and I’ve copied my cross-compiled bz2 headers/libraries into ~/digi-sysroot/include and ~/digi-sysroot/lib respectively. I’ve unzipped Python 2.7.3 sources into ~/digi-sysroot as well (based on the pathing described in your “building dependencies” section), built the host tools, and patched. Then I execute the following

    /**
    ~/digi-sysroot/Python-2.7.3$ CC=arm-cortex_a8-linux-gnueabi-gcc CXX=arm-cortex_a8-linux-gnueabi-g++ AR=arm-cortex_a8-linux-gnueabi-ar RANLIB=arm-cortex_a8-linux-gnueabi-ranlib ./configure –host=arm-cortex_a8-linux-gnueabi –build=arm-linux –prefix=/python

    ~/digi-sysroot/Python-2.7.3$ make HOSTPYTHON=./hostpython HOSTPGEN=./Parser/hostpgen BLDSHARED=”arm-cortex_a8-linux-gnueabi-gcc -shared” CROSS_COMPILE=arm-cortex_a8-linux-gnueabi- CROSS_COMPILE_TARGET=yes HOSTARCH=arm-cortex_a8-linux-gnueabi BUILDARCH=arm-linux
    … snip …
    Python build finished, but the necessary bits to build these modules were not found:
    _bsddb _curses _curses_panel
    _sqlite3 _ssl _tkinter
    bsddb185 bz2 dbm
    dl gdbm imageop
    nis readline sunaudiodev
    zlib
    To find the necessary bits, look in setup.py in detect_modules() for the module’s name.
    **/

    The make command still fails to find dependencies (specifically bz2). But I know its there:

    /**
    ~/digi-sysroot$ ls include/bzlib.h
    include/bzlib.h

    ~/digi-sysroot$ file lib/libbz2.*
    lib/libbz2.a: current ar archive
    lib/libbz2.so.1.0: ELF 32-bit LSB shared object, ARM, version 1 (SYSV), dynamically linked, not stripped
    lib/libbz2.so.1.0.6: ELF 32-bit LSB shared object, ARM, version 1 (SYSV), dynamically linked, not stripped
    **/

    I feel like there should be some way of telling configure/make about my sysroot path. Any advice?

    • Dustin Gooding says:

      Sorry to reply to my own question, but I made a breakthrough. I added “PYTHON_XCOMPILE_DEPENDENCIES_PREFIX=~/digi-sysroot” to my “make” arguments and the bz2 module was found and built.

      /**
      ~/digi_sysroot/Python-2.7.3/_install$ file ./lib/python2.7/lib-dynload/bz2.so
      ./lib/python2.7/lib-dynload/bz2.so: ELF 32-bit LSB shared object, ARM, version 1 (SYSV), dynamically linked, not stripped
      **/

      Might mention that variable in your instructions. I don’t know that the path of Python-x.y.z relative to your sysroot is important if you can just define the sysroot variable directly.

  10. Dawood says:

    Dear Paul
    Thanks for your great tutorial. I used it to cross-compile Python-2.7.2 for Advantech ROM-1210 (ARM) board and it has been installed. Now, I need to install Numpy-1.5 (as a python module), but I couldn’t find any apt instruction to cross-compile this module! Could you please introduce me a reference to cross-compile Python-2.7.2 with Numpy module?
    Thanks in advance.
    Regards,
    Dawoood

  11. Leo says:

    Hi Paul,

    Im a bit of a noob at this as its the first time ive tried compiling anything.

    Ive got to the point after applying the patch, where the actual compile happens, and very confused. Im trying to compile Python for android arm, i know that Android provides toolchains and an NDK, but i have no idea how to use them or plug them in to the above commands.
    My host machine is an ubuntu 12.04LTS, and my target is an Android Nexus 7 running 4.3. (although im hoping to be able to run the compile on other versions of android as its for a rooted app im currently making).

    Any help would be greatly appreciated.

    Thanks, Leo.

  12. xi says:

    Hi Paul.

    I want to know if I want to compile for ARM. What I should write about the following command
    CC=ppc_6xx-gcc CXX=ppc_6xx-g++ AR=ppc_6xx-ar RANLIB=ppc_6xx-ranlib ./configure –host=ppc-linux –build=x86_64-linux-gnu –prefix=/python
    make HOSTPYTHON=./hostpython HOSTPGEN=./Parser/hostpgen BLDSHARED=”ppc_6xx-gcc -shared” CROSS_COMPILE=ppc_6xx- CROSS_COMPILE_TARGET=yes HOSTARCH=ppc-linux BUILDARCH=x86_64-linux-gnu
    make install HOSTPYTHON=./hostpython BLDSHARED=”ppc_6xx-gcc -shared” CROSS_COMPILE=ppc_6xx- CROSS_COMPILE_TARGET=yes prefix=~/Python-2.7.2/_install
    My host machine is common Ubuntu system. What’s the mean about –host and –build.

    Thanks.

    • Paul Gibson says:

      Hi Xi,

      You need to replace ppc_6xx- with the name of your ARM toolchain. For example, if your toolchain name is arm-linux-gnueabi then you would replace ppc_6xx-gcc with arm-linux-gnueabi-gcc.
      Host should be arm-linux
      Build is the tuple of your build machine, either x86_64-linux-gnu or the 32-bit i686 equivalent.

      Paul

  13. PJM says:

    Great info!

    So I want to cross compile and install a package (http://pythonhosted.org/evdev/) on my target. How do I do that? I can download the package and run:

    $ hostpython setup.py build -x bdist_egg –plat-name=arm-linux-gnueabi

    Then theoretically I can move the egg and run easy_install on the target once Python is up and running on that machine.
    I’ve got Python to compile on the dev system and have it located in ~/Python2.7.3/_install as described above.
    Apparently I’ve got to point PYTHONPATH to (and install) setuptools and distutils somewhere. Am I doing this right? Any hints would be useful/ helpful.

    Thanks in advance.

Leave a Reply

(required)