Python 64-bit on Mac OS X 10.6 Snow Leopard

After yesterday's disastrous attempt to install various python packages, I started from scratch. First, I got rid of all of my python frameworks (backed up but removed from the path). Then, I compiled python 2.7 from scratch:

I got some help from http://blog.mahmoudimus.com/2009/12/python-2-6-4-and-twisted-9-on-os-x-10-6-snow-leopard/

./configure --enable-framework --enable-universalsdk=/Developer/SDKs/MacOSX10.6.sdk MACOSX_DEPLOYMENT_TARGET=10.6 --with-universal-archs=intel -with-readline-dir=/usr/local
make -j 17
make -j 17 test

make results:

Python build finished, but the necessary bits to build these modules were not found:
_bsddb             dl                 gdbm            
imageop            linuxaudiodev      ossaudiodev     
spwd               sunaudiodev                        
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

I'm not concerned about these - I don't use any of them and I assume I need to install some other packages to get them to work.

During make test, I had two failures that resulted in "python crash" pop-up boxes:

test_subprocess
.
    this bit of output is from a test of stdout in a different process ...
.
    this bit of output is from a test of stdout in a different process ...
test_sunaudiodev

Then, I got some malloc errors:

test_io

Testing large file ops skipped on darwin.
It requires 2147483648 bytes and a long time.
Use 'regrtest.py -u largefile test_io' to run it.

Testing large file ops skipped on darwin.
It requires 2147483648 bytes and a long time.
Use 'regrtest.py -u largefile test_io' to run it.
python.exe(22914,0x7fff70d3ebe0) malloc: *** mmap(size=9223372036854775808) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
python.exe(22914,0x7fff70d3ebe0) malloc: *** mmap(size=9223372036854775808) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
python.exe(22914,0x7fff70d3ebe0) malloc: *** mmap(size=9223372036854775808) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
test_ioctl

Installing Snow Leopard

I'm going to attempt to install snow leopard today. This post will serve as a record of the difficulties I run into.

Things to install (and ensure they are 64-bit):

• ipython
  
• numpy
  
• scipy
  
• matplotlib
  
• stsci-python
  
• starlink
  
• gfortran
  
• latex
  
• idl (check)
  

Things that have happened:

1. Had to restart again (twice) to install additional updates
   
2. My bash command line looked funny - something about bash changed, but I don't know what. The fix was easy: commented out some code from http://pseudogreen.org/blog/set_tab_names_in_leopard_terminal.html that I had been using to set the tab title
   
3. My locate db broke. Needed repair: sudo /usr/libexec/locate.updatedb
   
4. numpy svn failed to build:
   python setup.py build
Running from numpy source directory.non-existing path in 'numpy/distutils': 'site.cfg'
F2PY Version 2_8111
numpy/core/setup_common.py:86: MismatchCAPIWarning: API mismatch detected, the C API version numbers have to be updated. Current C api version is 4, with checksum 59750b518272c8987f02d66445afd3f1, but recorded checksum for C API version 4 in codegen_dir/cversions.txt is 3d8940bf7b0d2a4e25be4338c14c3c85. If functions were added in the C API, you have to update C_API_VERSION in numpy/core/setup_common.pyc.
MismatchCAPIWarning)
blas_opt_info:
FOUND:
extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
define_macros = [('NO_ATLAS_INFO', 3)]
extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers']

lapack_opt_info:
FOUND:
extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
define_macros = [('NO_ATLAS_INFO', 3)]
extra_compile_args = ['-faltivec']

running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building py_modules sources
building library "npymath" sources
customize NAGFCompiler
Found executable /usr/local/bin/f95
customize AbsoftFCompiler
Could not locate executable f90
Found executable /usr/bin/f77
absoft: no Fortran 90 compiler found
absoft: no Fortran 90 compiler found
customize IBMFCompiler
Could not locate executable xlf90
Could not locate executable xlf
customize IntelFCompiler
Could not locate executable ifort
Could not locate executable ifc
customize GnuFCompiler
Found executable /usr/local/bin/g77
gnu: no Fortran 90 compiler found
gnu: no Fortran 90 compiler found
customize Gnu95FCompiler
Found executable /usr/local/bin/gfortran
customize Gnu95FCompiler
customize Gnu95FCompiler using config
C compiler: gcc -arch i386 -arch ppc -arch ppc64 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.5.sdk -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes

compile options: '-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c'
gcc: _configtest.c
gcc _configtest.o -o _configtest
ld: library not found for -lcrt1.10.5.o
collect2: ld returned 1 exit status
ld: library not found for -lcrt1.10.5.o
collect2: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
Traceback (most recent call last):
File "setup.py", line 210, in
setup_package()
File "setup.py", line 203, in setup_package
configuration=configuration )
File "/Users/adam/repos/numpy-svn/numpy/distutils/core.py", line 186, in setup
return old_setup(**new_attr)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py", line 152, in setup
dist.run_commands()
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py", line 975, in run_commands
self.run_command(cmd)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py", line 995, in run_command
cmd_obj.run()
File "/Users/adam/repos/numpy-svn/numpy/distutils/command/build.py", line 37, in run
old_build.run(self)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/command/build.py", line 134, in run
self.run_command(cmd_name)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/cmd.py", line 333, in run_command
self.distribution.run_command(command)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py", line 995, in run_command
cmd_obj.run()
File "/Users/adam/repos/numpy-svn/numpy/distutils/command/build_src.py", line 152, in run
self.build_sources()
File "/Users/adam/repos/numpy-svn/numpy/distutils/command/build_src.py", line 163, in build_sources
self.build_library_sources(*libname_info)
File "/Users/adam/repos/numpy-svn/numpy/distutils/command/build_src.py", line 298, in build_library_sources
sources = self.generate_sources(sources, (lib_name, build_info))
File "/Users/adam/repos/numpy-svn/numpy/distutils/command/build_src.py", line 385, in generate_sources
source = func(extension, build_dir)
File "numpy/core/setup.py", line 670, in get_mathlib_info
raise RuntimeError("Broken toolchain: cannot link a simple C program")
RuntimeError: Broken toolchain: cannot link a simple C program
   
   SOLUTION: Use the Mac OS X 10.6 python (/usr/bin/python). I will do this until I run into another problem. Numpy build successfully
   
5. Build/install matplotlib - failed! Completely!
   
6. Acquired gcc/gfortran from hpc
   
7. Followed instructions from hyperjeff on fortran install...
   
8. Get rid of numpy 1.2.1: mv /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/numpy /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/.not_numpy.bak
   
9. Try to install scipy. Fail on missing umfpack, follow hyperjeff's instructions (but make sure to edit the site.cfg in scipy, not just the one in numpy)
   
   Had to add the following code:
   
sudo cp AMD/Lib/libamd.a /System/Library/Frameworks/Python.framework/Versions/2.6/lib
sudo cp UMFPACK/Lib/libumfpack.a /System/Library/Frameworks/Python.framework/Versions/2.6/lib
sudo cp AMD/Include/amd.h /System/Library/Frameworks/Python.framework/Versions/2.6/include
sudo cp UFconfig/UFconfig.h /System/Library/Frameworks/Python.framework/Versions/2.6/include
sudo cp UMFPACK/Include/*.h /System/Library/Frameworks/Python.framework/Versions/2.6/include

   
10. Installed fftw from fftw.org with simple ./configure, make, sudo make install - no compiler opts as they killed the install
    
11. Get SoundSource from rogueamoeba
    
12. Updated istatmenus
    

PSDs, cross-correlation...

scipy is capable of doing fft-base cross-correlation, convolution, etc., but it requires the stsci package, which is not generally easy to install. For that matter, scipy can be a pain some of the time. So agpy now includes a 2D cross-correlation code and a power spectrum / power spectral density code. These are pure-numpy codes that should be easy to use without any other bothersome dependencies.

EDIT: I have them check for scipy (which can cause crashes if you have a bad scipy install, e.g. 32 bit executables on a 64 bit system) because scipy uses FFTW and numpy appears not to. Also, this code & related stuff has been discussed on astrobetyter

agpy
correlate2d
psds

Logarithmic Colormap / Other Colormap in Matplotlib

This is kind of a pain to find out:


from matplotlib.colors import LogNorm

im = imshow(.... cmap=... , norm=LogNorm(vmin=clevs[0], vmax=clevs[-1]))


It also works for contours, and can be particularly useful if you only want to display contours at a few levels, but you want the colormap to start at a different point. e.g.:

contour(xx,levels=[2,3,4,5,6,7,8,9,10],norm=matplotlib.colors.Normalize(vmin=0,vmax=10))

will start at light blue instead of dark blue in the default colormap

wrapping text around a figure in latex

An example from Devin:

%\begin{wrapfigure}{l}{0.5\textwidth}
% \vspace{-27pt}
% \begin{center}
% \includegraphics[width=0.48\textwidth]{nsf_fig3.ps}
% \end{center}
% \vspace{-27pt}
% \caption{\it{}}
% \vspace{-12pt}
%\end{wrapfigure}
1:16
\usepackage{wrapfig}

my month of travel

I figure most people spending a month away from home would bring more than a small duffel full of clothes. I was quite satisfied with what I had. The things I missed (as a reminder for next time):
My desktop, eta
My music collection
Powerbars

...nothin else, really

Detexify!

for when you can't remember how to draw an angstrom:
detexify