There's a new post on a new idea for guiding with, e.g., APO at adamginsburg.blogspot.com, which is where I intend to move all my blogging.
Friday, October 5, 2012
Thursday, September 6, 2012
Grad Student Software Usage
I sent a poll to the APS grad students to determine what language(s) they use in their daily work and which they'd be comfortable teaching.
IDL is still clearly dominant, with python second. Matlab and mathematica are, to me surprisingly, prevalent as well.
Far more students are comfortable teaching IDL than any other language.
There's a large dispersion in the version control software preference, with mercurial and dropbox as the winners. Mercurial has a strong following among enzo/yt users, who make up a large fraction of those who use version control at all.
Tuesday, September 4, 2012
Research idea: NIR variation
I really want to know how near-infrared absorption lines correlate with emission lines; understanding this is essential for near infrared calibration.
The emission lines correlate reasonably well, but not perfectly. There are models out there of how these should behave, but they seem to be proprietary and non-free so I have no interest in them - I won't pay before I know they work, and I can't test them.
But perhaps more careful measurements of the night sky lines could provide some information about the absorption, particularly in the Brackett-delta and Brackett-epsilon region we've been interested in lately.
I'm pretty sure the change in slope observed above is from observing at different airmasses (2.3 vs 1.05)
The emission lines correlate reasonably well, but not perfectly. There are models out there of how these should behave, but they seem to be proprietary and non-free so I have no interest in them - I won't pay before I know they work, and I can't test them.
But perhaps more careful measurements of the night sky lines could provide some information about the absorption, particularly in the Brackett-delta and Brackett-epsilon region we've been interested in lately.
I'm pretty sure the change in slope observed above is from observing at different airmasses (2.3 vs 1.05)
Sunday, September 2, 2012
Research Idea: Stacking Finders
Idea: Stack all of the finders from spectroscopic observations. Finder images tend to be on lower-quality CCDs with no filter, but they frequently produce very deep observations. For example, the open K-band finder on TripleSpec (though it's technically not a CCD).
In order to stack them, you would need to mask out the bad pixels (already done) and compute astrometic solutions for the CCD. Un-warping the images will take some work, but there should be plenty of information available from thousands of observations of different fields to make this computation nearly ideal. Similarly, it should be possible to calibrate different pixels on the imager based on response to 2MASS standards.
Applications? Very deep imaging of spectroscopic targets. Short- and long-term variability (typical finder cadence is ~a few seconds). Deep imaging around stars and galaxies of interest - probably far deeper than you could get with classical observing requests.
This project should be achievable by a motivated undergraduate, but I think the tools for astrometric solutions need to be in place first. Astrometry.net is a great tool for this, but I think operates on spatial scales that are too large. Once basic astrometric solutions are available (e.g., pointing center for the image), I think IRAF tools could be automated to compute the complete solution, which would then be applied to all images.
Calibration might end up being the most challenging component, since there is variable atmospheric emission (absorption) that is not filtered by the finder. Depending on the application, though, large calibration errors may be acceptable. i.e., for deep nebular observations, morphology will be more important than absolute brightness, since the line responsible for the brightness cannot be directly determined. Whereas, for variability, calibration is important, but it can be computed directly from other stars in the field.
Sunday, August 12, 2012
Connecting to ipython notebook with SSH tunneling
My typical ssh tunnel looks something like:
ssh -N -f -L 8889:SERVER.colorado.edu:8889 ginsbura@SERVER.colorado.edu
For ipython notebooks, this approach was giving me the error:
channel 2: open failed: connect failed: Connection refused
. The ipython notebook is at http://127.0.0.1:8888/ locally. Therefore, the correct ssh tunnel command is:
ssh -N -f -L localhost:8888:localhost:8888 adam@SERVER.colorado.edu
Tuesday, May 29, 2012
Orion in the Infrared and Millimeter
This composite image of the Orion A Giant Molecular Cloud star-forming complex shows infrared emission from the WISE and MSX missions in 4 micron (red), 12 micron (blue), and 22 micron (green) emission with Bolocam Galactic Plane Survey 1.1mm emission overlaid in yellow/orange. The Orion A region is frequently featured in astronomical images:
http://apod.nasa.gov/apod/ap110917.html
http://apod.nasa.gov/apod/ap120212.html
http://apod.nasa.gov/apod/ap120206.html
http://www.eso.org/public/images/eso1219c/
http://www.eso.org/public/news/eso1209/
but its tail tends to be ignored. This quiescent region is the source of the next generation of stars, although the relatively small mass concentrations imply that no massive stars like the bright Theta 1C that powers the Orion Nebula will form.
The infrared colors show all sorts of stars including protostars. The infrared can pierce through the dust and find young stars still forming. The green and blue bands also see diffuse clouds of dust being illuminated by the central stars of the Orion nebula.
The yellow 1.1 mm dust emission shows the coldest dust that is shielded from external radiation. These cold clumps contain enough mass to form new stars...
http://apod.nasa.gov/apod/ap110917.html
http://apod.nasa.gov/apod/ap120212.html
http://apod.nasa.gov/apod/ap120206.html
http://www.eso.org/public/images/eso1219c/
http://www.eso.org/public/news/eso1209/
but its tail tends to be ignored. This quiescent region is the source of the next generation of stars, although the relatively small mass concentrations imply that no massive stars like the bright Theta 1C that powers the Orion Nebula will form.
The infrared colors show all sorts of stars including protostars. The infrared can pierce through the dust and find young stars still forming. The green and blue bands also see diffuse clouds of dust being illuminated by the central stars of the Orion nebula.
The yellow 1.1 mm dust emission shows the coldest dust that is shielded from external radiation. These cold clumps contain enough mass to form new stars...
Thursday, March 22, 2012
Sync a fork with the original repository on Git
A seemingly simple operation that I just can't seem to get right.
This page gives me useless information:
If you do that, it frankly doesn't work. Why? "upstream/master" etc. all have special meanings.
Here's the real process & explanation (thanks to Erik Tollerud for some help):
This page gives me useless information:
$ git checkout -b upstream/master
$ git remote add upstream git://github.com/upstream_maintainer/master.git
$ git pull upstream remote
$ git checkout master
$ git merge upstream/master
If you do that, it frankly doesn't work. Why? "upstream/master" etc. all have special meanings.
Here's the real process & explanation (thanks to Erik Tollerud for some help):
git checkout master # (assuming you have a local branch named master - otherwise, pick whatever branch you want synced)
git remote add original git@github.com:thing/thing.git # "upstream" = "original" = "remote" - the place you're trying to sync from
git fetch original # 'original' being the name YOU gave for the "remote/original" repository
git merge master original/master # now merge the "original/master" branch (they should have a branch named "master" too, otherwise you have to figure out which branch to get) into your master
git push # push your now-merged stuff back to github
# I was instructed to use these commands. They didn't work.
# git reset original # the word "original" here matches the word "original" on the previous line
# git reset --hard original # this will overwrite local changes
NaN-friendly convolution
NaN-friendly convolution is important for, e.g., masked data sets in which you want to interpolate across the masked region.
Astropy has gained this functionality with pull request 155:
https://github.com/astropy/astropy/pull/155
but this is a "direct" convolution parallel to IDL's 'convol' routine.
My FFT-based version now works in N dimensions and is a little cleaner:
http://code.google.com/p/agpy/source/browse/trunk/AG_fft_tools/convolve_nd.py
I'm still working on writing unit tests, and I'm really not sure what the "correct" behavior at the edges is for the different cases... right now, it seems counterintuitive to me, but the code is doing what I expect it to.
Also, Boxcar kernels always result in shifts for me... they're never supposed to. This is a bug.
Currently, other links to these codes:
http://stackoverflow.com/questions/1100100/fft-based-2d-convolution-and-correlation-in-python/8454010#8454010
Astropy has gained this functionality with pull request 155:
https://github.com/astropy/astropy/pull/155
but this is a "direct" convolution parallel to IDL's 'convol' routine.
My FFT-based version now works in N dimensions and is a little cleaner:
http://code.google.com/p/agpy/source/browse/trunk/AG_fft_tools/convolve_nd.py
I'm still working on writing unit tests, and I'm really not sure what the "correct" behavior at the edges is for the different cases... right now, it seems counterintuitive to me, but the code is doing what I expect it to.
Also, Boxcar kernels always result in shifts for me... they're never supposed to. This is a bug.
Currently, other links to these codes:
http://stackoverflow.com/questions/1100100/fft-based-2d-convolution-and-correlation-in-python/8454010#8454010
Thursday, January 26, 2012
Compiling vim...
Figured I had to post this...
I've been trying to compile command-line vim 7.3 on Mac OS X 10.7. I have the latest `hg clone`d version of vim. I'm stuck on ncurses.
If I `./configure` with no options, I get the following error:
checking --with-tlib argument... empty: automatic terminal library selection
checking for tgetent in -ltinfo... no
checking for tgetent in -lncurses... no
checking for tgetent in -ltermlib... no
checking for tgetent in -ltermcap... no
checking for tgetent in -lcurses... no
no terminal library found
checking for tgetent()... configure: error: NOT FOUND!
You need to install a terminal library; for example ncurses.
Or specify the name of the library with --with-tlib.
If instead I try `./configure --with-tlib=ncurses`
checking --with-tlib argument... ncurses
checking for linking with ncurses library... configure: error: FAILED
I have Xcode 4.1. As far as I can tell, ncurses is available:
$ file /usr/lib/libncurses.*
/usr/lib/libncurses.5.4.dylib: Mach-O universal binary with 2 architectures
/usr/lib/libncurses.5.4.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64
/usr/lib/libncurses.5.4.dylib (for architecture i386): Mach-O dynamically linked shared library i386
/usr/lib/libncurses.5.dylib: Mach-O dynamically linked shared library i386
/usr/lib/libncurses.dylib: Mach-O universal binary with 2 architectures
/usr/lib/libncurses.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64
/usr/lib/libncurses.dylib (for architecture i386): Mach-O dynamically linked shared library i386
Then I changed my PATH from /usr/local/bin... to /usr/bin.....
The problem was trying to use my /usr/local/bin/gcc instead of the mac default /usr/bin/gcc. Something about my locally installed gcc (4.6.1) caused major problems.
I also eventually had to do this command:
LDFLAGS=-L/usr/lib CFLAGS='-arch i386 -arch x86_64' CCFLAGS='-arch i386 -arch x86_64' CXXFLAGS='-arch i386 -arch x86_64' ./configure --enable-perlinterp --enable-pythoninterp --enable-cscope --with-features=huge
and then had to make sure my default python was NOT pointing to enthought!
I've been trying to compile command-line vim 7.3 on Mac OS X 10.7. I have the latest `hg clone`d version of vim. I'm stuck on ncurses.
If I `./configure` with no options, I get the following error:
checking --with-tlib argument... empty: automatic terminal library selection
checking for tgetent in -ltinfo... no
checking for tgetent in -lncurses... no
checking for tgetent in -ltermlib... no
checking for tgetent in -ltermcap... no
checking for tgetent in -lcurses... no
no terminal library found
checking for tgetent()... configure: error: NOT FOUND!
You need to install a terminal library; for example ncurses.
Or specify the name of the library with --with-tlib.
If instead I try `./configure --with-tlib=ncurses`
checking --with-tlib argument... ncurses
checking for linking with ncurses library... configure: error: FAILED
I have Xcode 4.1. As far as I can tell, ncurses is available:
$ file /usr/lib/libncurses.*
/usr/lib/libncurses.5.4.dylib: Mach-O universal binary with 2 architectures
/usr/lib/libncurses.5.4.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64
/usr/lib/libncurses.5.4.dylib (for architecture i386): Mach-O dynamically linked shared library i386
/usr/lib/libncurses.5.dylib: Mach-O dynamically linked shared library i386
/usr/lib/libncurses.dylib: Mach-O universal binary with 2 architectures
/usr/lib/libncurses.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64
/usr/lib/libncurses.dylib (for architecture i386): Mach-O dynamically linked shared library i386
Then I changed my PATH from /usr/local/bin... to /usr/bin.....
The problem was trying to use my /usr/local/bin/gcc instead of the mac default /usr/bin/gcc. Something about my locally installed gcc (4.6.1) caused major problems.
I also eventually had to do this command:
LDFLAGS=-L/usr/lib CFLAGS='-arch i386 -arch x86_64' CCFLAGS='-arch i386 -arch x86_64' CXXFLAGS='-arch i386 -arch x86_64' ./configure --enable-perlinterp --enable-pythoninterp --enable-cscope --with-features=huge
and then had to make sure my default python was NOT pointing to enthought!
Thursday, January 12, 2012
LaTeX: VIM + Skim
macvim-skim-install.sh is my install script for using MacVim.app with Skim.app.
The agpy wiki page has instructions that are probably more clear; I don't really like the colorscheme / layout of this blog.
You can use synctex to make an editor and viewer work together, but it is far from easy and far harder than it should be. Forward-search is pretty easy, but the latex-suite
I had to do the following:
For VIM->Skim.app (Skim.app is necessary for any of this to work), add these commands to .vimrc:
The
Going the other way (reverse-search / inverse-search) was MUCH more challenging. The code that does this is on agpy . Reproduced here for posterity (I hope to update the agpy version to deal with tabs). [A few hours later, I HAVE replaced the code. Below are the old applescript version, then the new, vim-based version
New code: download link
Save that as an executable in your default path (e.g.,
You need to have mvim on your path. mvim comes with MacVim.app, but is NOT installed by default. Install it by doing something like:
You'll also need to install WhichTab.vim in your
The agpy wiki page has instructions that are probably more clear; I don't really like the colorscheme / layout of this blog.
You can use synctex to make an editor and viewer work together, but it is far from easy and far harder than it should be. Forward-search is pretty easy, but the latex-suite
\ls
only works intermittently and is not easily customizable.I had to do the following:
For VIM->Skim.app (Skim.app is necessary for any of this to work), add these commands to .vimrc:
" Activate skim map ,v :w<CR>:silent !/Applications/Skim.app/Contents/SharedSupport/displayline -r <C-r>=line('.')<CR> %<.pdf %<CR><CR> map ,p :w<CR>:silent !pdflatex -synctex=1 --interaction=nonstopmode %:p <CR>:silent !/Applications/Skim.app/Contents/SharedSupport/displayline -r <C-r>=line('.')<CR> %<.pdf %<CR><CR> map ,m :w<CR>:silent !make <CR>:silent !/Applications/Skim.app/Contents/SharedSupport/displayline -r <C-r>=line('.')<CR> %<.pdf %<CR><CR> " Reactivate VIM map ,r :w<CR>:silent !/Applications/Skim.app/Contents/SharedSupport/displayline -r <C-r>=line('.')<CR> %<.pdf %<CR>:silent !osascript -e "tell application \"MacVim\" to activate" <CR><CR> map ,t :w<CR>:silent !pdflatex -synctex=1 --interaction=nonstopmode %:p <CR>:silent !/Applications/Skim.app/Contents/SharedSupport/displayline -r <C-r>=line('.')<CR> %<.pdf %<CR>:silent !osascript -e "tell application \"MacVim\" to activate" <CR><CR>
The
,m
command will reload the file and put your cursor where the text is. ,t
will return VIM to the front afterwards.Going the other way (reverse-search / inverse-search) was MUCH more challenging. The code that does this is on agpy . Reproduced here for posterity (I hope to update the agpy version to deal with tabs). [A few hours later, I HAVE replaced the code. Below are the old applescript version, then the new, vim-based version
#!/bin/bash
file="$1"
line="$2"
[ "${file:0:1}" == "/" ] || file="${PWD}/$file"
# Use Applescript to activate VIM, find file, and load it
# the 'delay' command is needed to prevent command/control/shift from sticking when this
# is activated (e.g., from Skim, where the command is command-shift-click)
#
# key code 53 is "escape" to escape to command mode in VIM
exec osascript \
-e "delay 0.2" \
-e "tell application \"MacVim\" to activate" \
-e "tell application \"System Events\"" \
-e " tell process \"MacVim\"" \
-e " key code 53 "\
-e " keystroke \":set hidden\" & return " \
-e " keystroke \":if bufexists(bufname('$file'))\" & return " \
-e " keystroke \":exe \\\":buffer \\\" . bufnr(bufname('$file'))\" & return " \
-e " keystroke \":else \" & return " \
-e " keystroke \":echo \\\"Could not load file\\\" \" & return " \
-e " keystroke \":endif\" & return " \
-e " keystroke \":$line\" & return " \
-e " end tell" \
-e "end tell"
New code: download link
#!/bin/bash # Install directions: # Put this file somewhere in your path and make it executable # To set up in Skim, go to Preferences:Sync # Change Preset: to Custom # Change Command: to macvim-load-line # Change Arguments: to "%file" %line file="$1" line="$2" debug="$3" echo file: $file echo line: $line echo debug: $debug for server in `mvim --serverlist` do foundfile=`mvim --servername $server --remote-expr "WhichTab('$file')"` if [[ $foundfile > 0 ]] then mvim --servername $server --remote-expr "foreground()" if [[ $debug ]] ; then echo mvim --servername $server --remote-send ":exec \"tabnext $foundfile\""; fi mvim --servername $server --remote-send ":exec \"tabnext $foundfile\" " if [[ $debug ]] ; then echo mvim --servername $server --remote-send ":$line "; fi mvim --servername $server --remote-send ":$line " fi done
Save that as an executable in your default path (e.g.,
/usr/local/bin/macvim-load-line
) and open Skim.app, go to Preferences:Sync and make the command look like this:You need to have mvim on your path. mvim comes with MacVim.app, but is NOT installed by default. Install it by doing something like:
cp /Users/adam/Downloads/MacVim-7_3-53/mvim /usr/local/bin/mvim
You'll also need to install WhichTab.vim in your
~/.vim/plugins/
directory. It's available here ( download link ). Here's the source:function! WhichTab(filename) " Try to determine whether file is open in any tab. " Return number of tab it's open in let buffername = bufname(a:filename) if buffername == "" return 0 endif let buffernumber = bufnr(buffername) " tabdo will loop through pages and leave you on the last one; " this is to make sure we don't leave the current page let currenttab = tabpagenr() let tab_arr = [] tabdo let tab_arr += tabpagebuflist() " return to current page exec "tabnext ".currenttab " Start checking tab numbers for matches let i = 0 for tnum in tab_arr let i += 1 echo "tnum: ".tnum." buff: ".buffernumber." i: ".i if tnum == buffernumber return i endif endfor endfunction function! WhichWindow(filename) " Try to determine whether the file is open in any GVIM *window* let serverlist = split(serverlist(),"\n") "let currentserver = ???? for server in serverlist let remotetabnum = remote_expr(server, \"WhichTab('".a:filename."')") if remotetabnum != 0 return server endif endfor endfunction
Sunday, January 8, 2012
API documentation on agpy
I finally processed agpy through sphinx and made some nice html documentation.
http://agpy.googlecode.com/svn/trunk/doc/html/agpy.html
http://agpy.googlecode.com/svn/trunk/doc/html/agpy.html
Subscribe to:
Posts (Atom)