This article details how to cross compile Python for the PowerPC platform. It should apply equally to other platforms such as ARM, just plug in the correct cross-compiler. The article supports Python versions, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.6.5 and 3.1.1, 3.1.2.
Firstly, download the version of Python that you want to use from http://www.python.org/
Unpack the Python package using tar:
This will create a directory called Python-2.6.5. Goto the directory:
Then run these commands to build the host components:
make python Parser/pgen
mv python hostpython
mv Parser/pgen Parser/hostpgen
make distclean
Download the correct patch for your version of Python:
- Python-2.6.1-xcompile.patch
- Python-2.6.2-xcompile.patch
- Python-2.6.3-xcompile.patch
- Python-2.6.4-xcompile.patch
- Python-2.6.5-xcompile.patch
- Python-3.1.1-xcompile.patch
- Python-3.1.2-xcompile.patch
Then apply the patch:
patch -p1 < Python-2.6.5-xcompile.patch
Then run this (where ~/Python-2.6.5/_install/ is your desired installation path)
make HOSTPYTHON=./hostpython HOSTPGEN=./Parser/hostpgen BLDSHARED="ppc-linux-gcc -shared" CROSS_COMPILE=ppc_6xx- CROSS_COMPILE_TARGET=yes
make install HOSTPYTHON=./hostpython BLDSHARED="ppc-linux-gcc -shared" CROSS_COMPILE=ppc_6xx- CROSS_COMPILE_TARGET=yes prefix=~/Python-2.6.5/_install
This will install all your python binaries and libraries in ~/Python-2.6.5/_install.
Copy the entire _install directory to the device, setup the PATH environment variable to include the location of the Python executable and run:
…and hopefully all the tests will run correctly.
To speed up the importing of Python modules on the target, it is recommended to zip up the lib directory to make a file called python26.zip. This means that we do not have to copy all the individual Python files to the target, just the one zip file. This technique might also save space, but that depends on your file system. On the host machine:
zip -r -y python26.zip .
Delete libpythonxxx.a site-packages, lib-dynload, config and anything else you do not require from the python26.zip file.
Copy the _install/bin/python to the /usr/bin directory on the target:
Copy the python26.zip file to the /usr/lib directory on the target:
Create a directory on the target called python2.6 in the /usr/lib directory and copy the following directories to that directory:
- ./lib/python2.6/config
- ./lib/python2.6/lib-dynload
- ./lib/python2.6/site-packages
Your directory structure on the target should be as follows:
python2.6 python26.zip
/python/lib/python2.6 # ls
config lib-dynload site-packages
Set the PYTHONHOME environment variable to /usr/ and you are ready to run Python on the target.
The above patches are based on Chris Lambacher’s patches for Python 2.5 here:
http://whatschrisdoing.com/blog/2006/10/06/howto-cross-compile-python-25/
Other links and credits:
http://www.ailis.de/~k/archives/19-ARM-cross-compiling-howto.html
No related posts.
Related posts brought to you by Yet Another Related Posts Plugin.
Thanks for the 3.1/2.6 patch updates. I have some new notes on how to cross compile extensions.
Just did this for an ARM computer I have – problem now is the result in _install is 75MB – my machine has 64 MB.
Any thoughts on optimizing/throwing out whats not needed?
The _install directory is really just for initial testing as it contains lots of extra build artifacts. I believe it is much better and faster to use the zip method of deploying to your final system.
Having said this, here are some ways to get the size down for both methods:
rm -rf include
rm -rf share
cd lib
rm libpython2.6.a
cd lib/python2.6
rm -rf `find . -name test`
rm `find . -name *.pyo`
rm `find . -name *.py`
cd ../../bin
rm python2.6
And strip the python executable. On powerpc, the command is:
This gets my image down to 20M. Now you can delete modules that you do not think you’ll use. Look in the lib/python2.6 directory and delete any subdirectory that you do not think you’ll use. Example of less used modules would be:
– email, xml, sqlite3, curses, idlelib, encodings, lib2to3, lib-tk, etc.
You may even to able to remove the config directory. I removed it and had success.
This got me down to 11M.
The last piece of optimization would be the lib-dynload directory. Some of these shared objects would definitely be needed, but not all.
So, after being very aggressive my _install image is 6M.
Okay, manually copying each directory was much smaller than 75MB of the tarball, more like 5-10MB. Thanks!
Hi, Thanks for the great tutorial.
I try following it but I got few problems that I hope you can help with.
I am compiling Python-2.6.2 on a Ubuntu-x86 and my target is linux on imx27(arm9)
first, when I compile I got some errors during the compilation and in the end I got:
——————————————————
errors
Failed to find the necessary bits to build these modules:
_bsddb _hashlib _sqlite3
_ssl _tkinter bsddb185
bz2 dbm gdbm
readline sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module’s name.
Failed to build these modules:
_curses _curses_panel binascii
—————————————————————-
second after I ignored those errors and installed the file on my target, when I am running the test script I got:
———————————————————-
root@freescale /$ python -v /lib/python2.6/test/test___all__.py
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
‘import site’ failed; traceback:
ImportError: No module named site
Python 2.6.2 (r262:71600, Nov 3 2009, 21:04:45)
[GCC 4.1.2] on linux2
Type “help”, “copyright”, “credits” or “license” for more information.
Traceback (most recent call last):
File “/lib/python2.6/test/test___all__.py”, line 1, in
import unittest
ImportError: No module named unittest
# clear __builtin__._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.exitfunc
# clear sys.exc_type
# clear sys.exc_value
# clear sys.exc_traceback
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup __main__
# cleanup[1] zipimport
# cleanup[1] signal
# cleanup[1] exceptions
# cleanup[1] _warnings
# cleanup sys
# cleanup __builtin__
# cleanup ints: 3 unfreed ints
# cleanup floats
——————————————————————
I am attaching also the configure+make+install commands:
—————————————————————]
CC=arm-none-linux-gnueabi-gcc CXX=arm-none-linux-gnueabi-g++ AR=arm-none-linux-gnueabi-ar RANLIB=arm-none-linux-gnueabi-ranlib ./configure –host=arm-linux –build=i686-pc-linux-gnu –prefix=/python
make HOSTPYTHON=./hostpython HOSTPGEN=./Parser/hostpgen BLDSHARED=”arm-none-linux-gnueabi-gcc -shared” CROSS_COMPILE=arm-none-linux-gnueabi- CROSS_COMPILE_TARGET=yes
make install HOSTPYTHON=./hostpython BLDSHARED=”arm-none-linux-gnueabi-gcc -shared” CROSS_COMPILE=arm-none-linux-gnueabi- CROSS_COMPILE_TARGET=yes prefix=~/workspace/DVE/Http/Python-2.6.2/_install
————————————————————
Hi,
I found the problem, it was all about paths.
now I am facing only one probelm. since the binascii module fail to build lots of test cant run. can you figure why it happens?
Hi Amit,
Can you post the error that the test cases give (for binascii)? Ubuntu is probably missing the package you need to compile it properly. I couldn’t find a binascii package, but it could be part of another package. The error message should help me.
Thanks,
Paul
Hi,
A huge thanks for this – worked a treat. I’ve made a 2.6.4 patch if you want it, email me and I’ll send it.
Being new to Python, I’m still trying to get my head around cross compiling the modules side of it. Looking at the setup.py script, it seems to be looking for include files and library files to link to the modules on the host system not the target. Which I guess is why it’s not able to build modules for which I have valid libraries eg ncurses and readline. Amit, what paths did you change? Anything in setup.py?
Can anyone give any experience of cross compiling modules for python?
Did you need any extra patches? Did you modify Setup.local or something else to add your modules to?
Thanks
Rachel
Hi Rachel!
I’ve tried to find on google a patch for cross-compiling python2.6.4 but there is no result. Could you email for me your patch. My email is nguyentiendat05@gmail.com.
Many thanks for you!
Just updated the post for Python 2.6.4.
Great tutorial! If you’re cross-compiling in OS X, the python executable is actually called python.exe (since OS X filenames are case-insensitive). So the make and mv commands should be:
make python.exe Parser/pgen
mv python.exe hostpython
Thank you for your work for cross compile python.I followed your instructions to cross compile python2.6.4 for powerpc405 (i used eldk as cross compile tool). The compilation process was ok, and no errors occured.
I was expecting a trackback to pop up, but I posted about using your technique and getting functional _ctypes support in the bargain. It occurs to me you could incorporate the technique into future patches.
Thanks Art, appreciate the effort. I will test and incorporate into my patches.
Message for Nguyen Thai Binh: Can you contact me on my email or give me yours: snice1981@gmail.com
I would like to use ELDK also but I don’t know how. If you could give me some tips, it would be greate
Thanks
Thanks for the great explanation and patches. Guido, this should be in the mainline! To encourage the use of Python in embedded development it should be as easy as possible to cross-compile it (at least, no source code patching should be needed).