Thursday, April 22, 2010

Gildas CLASS

It's absurdly difficult to find help on GILDAS Class, probably because you can't google "class" and most people probably don't label every piece of code with "GILDAS class".

Anyway, here are some scripts that I refer back to often:

file in August2009BGPS.dat
file out August2009fits.dat multiple
on error "file out August2009fits.dat"
say "READ IN FILES"
define character sourcelist*10[300]
accept sourcelist /column observed_sources.txt
on error "continue"
get 1001
set window -100 160
set mask -400 -100 160 400
set mode x -400 400
set align velocity
for i 1 to 161
say "Working on SOURCE "'i'
find /source 'sourcelist[i]' /telescope "CSO 4GHZ IF1" /offset 0 0 /quality 5
average
on error "@avplot2 'sourcelist[i]' 'i'; next"
base 3
line 0
min
plot
vis
write i
! on error "continue"
next


and


file in araya-2004.cls
find
define character filename*20
for i 1 to 20
say "Working on source "'i'
get next
let filename "araya-2004_"'i'".fits"
say "fits write "'filename'" /mode spectrum"
fits write 'filename' /mode spectrum
next

!file in araya-2002.cls
!find
!define character filename*20
!for i 1 to 42
! say "Working on source "'i'
! get i
! let filename "araya-2002_"'i'".fits"
! say "fits write "'filename'" /mode spectrum"
! fits write 'filename' /mode spectrum
!next

Thursday, April 15, 2010

Montage wrapper

(I'm going to try to gradually shift my blogging to this one...)

I wrote a bash wrapper for Tom Robitaille's montage wrapper to allow fits wildcards.


#!/bin/bash

origdir=`pwd`

#echo $# $*

if [ $# -gt 0 ]
then
for ii in $*
do
if [ ${ii%=*} == 'header' ]
then
/usr/local/bin/montage/mGetHdr ${ii#*=} mosaic.hdr
elif [ ${ii%=*} == 'outfile' ]
then
outfile=${ii#*=}
elif [ `echo $ii | grep =` ]
then
params="$params,${ii%=*}='${ii#*=}'"
elif [ `echo $ii | grep ".fits"` ]
then
files=( ${files[@]} $ii )
fi
done
fi
echo ${files[@]} ${#files}
if [ ${#files} -gt 0 ]
then
mkdir tmp
cp ${files[@]} tmp/
cp mosaic.hdr tmp/
cd tmp/
fi

if [ -f mosaic.hdr ]
then
echo "mosaic.hdr exists, continuing"

dir=`pwd`
echo python -c "import montage; montage.wrappers.mosaic('$dir','$dir/mosaic',header='$dir/mosaic.hdr'$params)"
python -c "import montage; montage.wrappers.mosaic('$dir','$dir/mosaic',header='$dir/mosaic.hdr'$params)"

cd $origdir
if [ -d tmp ]
then
if [ $outfile ]
then
mv tmp/mosaic/mosaic.fits $outfile
else
mv tmp/mosaic mosaic
fi
rm -r tmp
fi

else
echo "mosaic.hdr does not exist. Quitting."
cd $origdir
fi

Sunday, March 28, 2010

IRAF append/prepend/replace

IRAF has special syntax to append/replace, so you can do something like:
imarith *.fits - dark.fits ds_//*.fits
...though I can't remember right now how to do replacement etc.

Thursday, March 18, 2010

command line & escape key

Problem: in both the bash command line and ipython, if I hit escape twice, I can never again do history-search-backwards (i.e. when you type part of a command and type "up" and reverse-search through your command history). Any idea how to fix this?

Friday, March 5, 2010

Rebuilding python from scratch again

I got scipy working a week or two ago, but doing so killed matplotib's tkagg. So, I switched to the MacOSX backend, which worked ok until I realized that the interactive (connect) features of macosx failed miserably. This led me to try to get matplotlib working.... which broke with those awful "symbol not found" errors in ft2font.so and _path.so, which I've determined all have to do with linking to the wrong library files.

The most worrisome part of this process was discovering that a full Time Machine recovery of /usr and /Library/Frameworks and /Library/Python did *not* restore python - it stayed dead with IDENTICAL errors. So there are probably additional layers of hidden links.

The process below is based on hyperjeff's blog post but differs substantially based on Sam Skillman's recommendations and the very big issue I ran into that my /usr/local files appeared to be corrupted. After this install, my path no longer includes /usr/local/bin and /sw has been moved to /_sw... hopefully one of these days I'll be ballsy enough to delete it.

  1. Install python 2.6.4
    1. Needed a clean terminal with no flags set at all. Don't know why - all I had set were a bunch of -arch x86_64 flags.

      export LD_LIBRARY_PATH="/usr/local/lib:/usr/X11/lib"
      ./configure --enable-framework=/Library/Frameworks MACOSX_DEPLOYMENT_TARGET=10.6
      make -j 17
      sudo make install

    2. Reset PYTHONPATH to blank
    3. alias clearflags='export CFLAGS=""; export CCFLAGS=""; export CXXFLAGS=""; export LDFLAGS=""; export FFLAGS="";' to make sure

  2. Install FFTW

    cd ~/tmp
    curl -O http://www.fftw.org/fftw-3.2.2.tar.gz
    tar xf fftw-3.2.2.tar.gz
    cd fftw-3.2.2
    clearflags
    ./configure CC="gcc -arch x86_64" CXX="g++ -arch x86_64" CPP="gcc -E" CXXCPP="g++ -E"
    make -j 17
    sudo make install

  3. Install UMFPACK

    cd ~/tmp
    curl -O http://www.cise.ufl.edu/research/sparse/umfpack/current/UMFPACK.tar.gz
    curl -O http://www.cise.ufl.edu/research/sparse/UFconfig/current/UFconfig.tar.gz
    curl -O http://www.cise.ufl.edu/research/sparse/amd/current/AMD.tar.gz
    tar xf AMD.tar.gz
    tar xf UFconfig.tar.gz
    tar xf UMFPACK.tar.gz
    sed -ibck 's/F77 = f77/F77 = gfortran/' UFconfig/UFconfig.mk
    sed -ibck '299,303s/# //' UFconfig/UFconfig.mk
    cp UFconfig/UFconfig.h AMD/Include/
    cp UFconfig/UFconfig.h UMFPACK/Include/
    cd UMFPACK
    make -j 17
    make hb
    make clean

  4. Install numpy
    1. Set environment variables

      export MACOSX_DEPLOYMENT_TARGET=10.6
      export CFLAGS="-arch x86_64"
      export FFLAGS="-m64"
      export LDFLAGS="-Wall -undefined dynamic_lookup -bundle -arch x86_64"
      export PYTHONPATH="/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/"
      echo "[amd]
      library_dirs = /Users/adam/tmp/AMD/Lib
      include_dirs = /Users/adam/tmp/AMD/Include
      amd_libs = amd

      [umfpack]
      library_dirs = /Users/adam/tmp/UMFPACK/Lib
      include_dirs = /Users/adam/tmp/UMFPACK/Include
      umfpack_libs = umfpack" > site.cfg

    2. Setup & Install

      python setup.py build --fcompiler=gnu95
      sudo python setup.py install

    3. Test: python -c "import numpy"

  5. Install scipy. The important thing is to use g++-4.2 because g++-4.5 doesn't accept the -arch flag. Also, get rid of /sw if it's on your computer at all.

    sudo mv /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/numpy /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/numpyX
    cd ~/repos/scipy-0.7.1
    python setup.py clean
    rm -rf build
    clearflags
    FFLAGS="-m64" CFLAGS="-arch x86_64 -I/usr/local/include/freetype2 -I/usr/X11/include -L/usr/X11/lib" LDFLAGS="-Wall -undefined dynamic_lookup -bundle -lpng -arch x86_64" CXX="/usr/bin/g++-4.2" CC="/usr/bin/gcc-4.2" python setup.py build
    python setup.py install

    Test the install:

    python -c "import scipy, scipy.fftpack, scipy.interpolate"

    1. Install matplotlib. MAKE SURE /usr/bin/texbin is in front of /usr/local/bin and /sw/bin so that dvipng comes from MacTEX. I also ended up having to remove /usr/local/bin from my path completely

      sudo mv /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/numpy /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/numpyX
      cd ~/repos/matplotlib-svn
      python setup.py clean
      rm -rf build
      clearflags

    2. Do hyperjeff's recommended edits except don't use /usr/local because it's f'd:
      make.osx:

      MACOSX_DEPLOYMENT_TARGET=10.6
      PREFIX=/usr
      PYTHON=/Library/Frameworks/Python.framework/Versions/Current/bin/python

      ## You shouldn't need to configure past this point (and yet…)

      PKG_CONFIG_PATH="${PREFIX}/lib/pkgconfig"
      CFLAGS="-arch i386 -arch x86_64 -I${PREFIX}/include -I${PREFIX}/include/freetype2 -isysroot /Developer/SDKs/MacOSX10.6.sdk"
      LDFLAGS="-arch i386 -arch x86_64 -L${PREFIX}/lib -syslibroot,/Developer/SDKs/MacOSX10.6.sdk"
      FFLAGS="-arch i386 -arch x86_64"

      setup.cfg:

      wxagg = False

    3. Do the install (different from hyperjeff b/c I don't want root)
      sudo make -f make.osx fetch deps
      make -f mpl_build mpl_install
      python setup.py install


  6. Install setuptools
  7. easy_install ipython
  8. install everything else pythonically

Monday, March 1, 2010

Thursday, February 25, 2010

macvim crash

well, it finally happened.... my reliable, trusty editor crashed. That should be impossible. I am ready to call it quits for the week....


Process: MacVim [650]
Path: /Applications/Vim.app/Contents/MacOS/MacVim
Identifier: org.vim.MacVim
Version: 7.2 (49)
Code Type: X86 (Native)
Parent Process: Vim [649]

Date/Time: 2010-02-25 13:12:43.001 -0700
OS Version: Mac OS X 10.6.2 (10C540)
Report Version: 6

Interval Since Last Report: 871676 sec
Crashes Since Last Report: 26
Per-App Interval Since Last Report: 938504 sec
Per-App Crashes Since Last Report: 1
Anonymous UUID: 03159B9E-2257-4E38-8C4A-4D4DAF5641A7

Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: 0x000000000000000d, 0x0000000000000000
Crashed Thread: 0 Dispatch queue: com.apple.main-thread

Thread 0 Crashed: Dispatch queue: com.apple.main-thread
0 com.apple.CoreFoundation 0x99119480 __CFSetCallback + 0
1 com.apple.CoreFoundation 0x990c78bc ___CFBasicHashFindBucket1 + 444
2 com.apple.CoreFoundation 0x990cfaac CFBasicHashFindBucket + 252
3 com.apple.CoreFoundation 0x990e8293 CFSetGetValue + 131
4 com.apple.AppKit 0x961bae7e -[NSWindow _discardTrackingRect:] + 59
5 com.apple.AppKit 0x961badca -[NSView(NSInternal) _uninstallTrackingArea:] + 123
6 com.apple.AppKit 0x960d2c32 -[NSView(NSInternal) _uninstallRemovedTrackingAreas] + 293
7 com.apple.AppKit 0x960dac40 -[NSView(NSInternal) _updateTrackingAreas] + 646
8 com.apple.CoreFoundation 0x990ea4e0 CFArrayApplyFunction + 224
9 com.apple.AppKit 0x960daefb -[NSView(NSInternal) _updateTrackingAreas] + 1345
10 com.apple.CoreFoundation 0x990ea4e0 CFArrayApplyFunction + 224
11 com.apple.AppKit 0x960daefb -[NSView(NSInternal) _updateTrackingAreas] + 1345
12 com.apple.CoreFoundation 0x990ea4e0 CFArrayApplyFunction + 224
13 com.apple.AppKit 0x960daefb -[NSView(NSInternal) _updateTrackingAreas] + 1345
14 com.apple.AppKit 0x960da8db _handleInvalidCursorRectsNote + 392
15 com.apple.CoreFoundation 0x99135892 __CFRunLoopDoObservers + 1186
16 com.apple.CoreFoundation 0x990f218d __CFRunLoopRun + 557
17 com.apple.CoreFoundation 0x990f1864 CFRunLoopRunSpecific + 452
18 com.apple.CoreFoundation 0x990f1691 CFRunLoopRunInMode + 97
19 com.apple.HIToolbox 0x936f6f0c RunCurrentEventLoopInMode + 392
20 com.apple.HIToolbox 0x936f6bff ReceiveNextEventCommon + 158
21 com.apple.HIToolbox 0x936f6b48 BlockUntilNextEventMatchingListInMode + 81
22 com.apple.AppKit 0x960b0ac5 _DPSNextEvent + 847
23 com.apple.AppKit 0x960b0306 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 156
24 com.apple.AppKit 0x9607249f -[NSApplication run] + 821
25 com.apple.AppKit 0x9606a535 NSApplicationMain + 574
26 org.vim.MacVim 0x0000238b _start + 209
27 org.vim.MacVim 0x000022b9 start + 41

Thread 1: Dispatch queue: com.apple.libdispatch-manager
0 libSystem.B.dylib 0x98d0c0ea kevent + 10
1 libSystem.B.dylib 0x98d0c804 _dispatch_mgr_invoke + 215
2 libSystem.B.dylib 0x98d0bcc3 _dispatch_queue_invoke + 163
3 libSystem.B.dylib 0x98d0ba68 _dispatch_worker_thread2 + 234
4 libSystem.B.dylib 0x98d0b4f1 _pthread_wqthread + 390
5 libSystem.B.dylib 0x98d0b336 start_wqthread + 30

Thread 2:
0 libSystem.B.dylib 0x98ce58da mach_msg_trap + 10
1 libSystem.B.dylib 0x98ce6047 mach_msg + 68
2 com.apple.CoreFoundation 0x990f277f __CFRunLoopRun + 2079
3 com.apple.CoreFoundation 0x990f1864 CFRunLoopRunSpecific + 452
4 com.apple.CoreFoundation 0x990f1691 CFRunLoopRunInMode + 97
5 com.apple.Foundation 0x91b24430 +[NSURLConnection(NSURLConnectionReallyInternal) _resourceLoadLoop:] + 329
6 com.apple.Foundation 0x91aeb8d8 -[NSThread main] + 45
7 com.apple.Foundation 0x91aeb888 __NSThread__main__ + 1499
8 libSystem.B.dylib 0x98d12fbd _pthread_start + 345
9 libSystem.B.dylib 0x98d12e42 thread_start + 34

Thread 3:
0 libSystem.B.dylib 0x98d04856 select$DARWIN_EXTSN + 10
1 com.apple.CoreFoundation 0x99131ddd __CFSocketManager + 1085
2 libSystem.B.dylib 0x98d12fbd _pthread_start + 345
3 libSystem.B.dylib 0x98d12e42 thread_start + 34

Thread 4:
0 libSystem.B.dylib 0x98d0b182 __workq_kernreturn + 10
1 libSystem.B.dylib 0x98d0b718 _pthread_wqthread + 941
2 libSystem.B.dylib 0x98d0b336 start_wqthread + 30

Thread 0 crashed with X86 Thread State (32-bit):
eax: 0x00515db0 ebx: 0x990c7711 ecx: 0x00516460 edx: 0xbfffcabc
edi: 0x00001041 esi: 0x00504270 ebp: 0xbfffca38 esp: 0xbfffc99c
ss: 0x0000001f efl: 0x00010246 eip: 0x99119480 cs: 0x00000017
ds: 0x0000001f es: 0x0000001f fs: 0x00000000 gs: 0x00000037
cr2: 0x97a20000