Using IDL within Python

So you have a robust and well-tested program someone gave you that runs in ITT's IDL programming environment, and you don't want to rewrite it in Python so you can use Python.  Don't worry, you can use Python to run IDL programs (and probably the other way 'round, too).

Yes, you can have your cake and eat it too.  There are actually multiple ways to do this, including two different Python modules (pyIDL and pIDLy), using python's os.system() to run an IDL script that outputs to a file, then reading in that file with Python, and so on...

I'm going to show off pIDLy. There's a nice page here that explains it all, but here's a quick summary:

pIDLy requires numpy and Pexpect to run.

pip install pidly pexpect

Now you set it up in your program:

import pidly
...
idl = pidly.IDL('/Applications/itt/idl706/bin/idl')

IDL seems to preferentially installs itself by adding a bunch of aliases and $shell_variables to your .bashrc or .tcshrc file, so the IDL executable is actually not in your $PATH like executables really ought to be.  That means you have to give pIDLy the full path to your IDL executable (check your .bashrc or .tcshrc)

Now, to use it:  pIDLy works by creating an IDL session within your Python session (I called my session 'idl') ; you can pass it commands to execute, and retrieve results.

idl.pro('kinematics,r_of_t,phi_of_t,z_of_t,time_t',tra[i],tdec[i],tplx[i],tu[i],tv[i],tw[i],time,timestep)

I'm sending the IDL procedure some of my python variables, but ultimately whatever's in those parentheses has to follow IDL syntax.  You'll get all the printed messages the IDL program would normally produce, and if something crashes here, you'll end up at the interactive IDL prompt. Next time you run IDL itself you'll find all of the commands pIDLy sent to IDL in the scrollback buffer.  Also, notice one big difference between IDL procedures and Python programs: In Python, you get results from a procedures by setting the program equal to its output.

outputs = program(inputs)

In IDL, you get results from a procedure by creating a variable, passing it to a function that fills it, and then using the results. (I THINK what's happening is that IDL is using the argument list to say "this data object has this variable name inside the subroutine, and this other variable name outside the subroutine" - this page supports that theory)

outputs = dblarr(datalen)
program(inputs,outputs)

In my above example, r_of_t, phi_of_t, z_of_t, and time_t are created within the 'kinematics' subroutine, but they had to exist in some in the main program so IDL would make their values accessible outside of the 'kinematics' subroutine.  Because they CANNOT exist beforehand in Python, I put their names in strings.

Now, getting results out is easy.  Once the IDL session is done running its command, Python resumes on the next line.  The IDL session keeps running, though, which allows you to pull out the results.

r_t[i] = idl.r_of_t
phi_t[i] = idl.phi_of_t
z_t[i] = idl.z_of_t
time_t = idl.time_t

They are returned as numpy datatypes, and pIDLy's developer claims nearly everything works, if a little slowly.  The same IDL session can run multiple programs, so you don't need to close and reopen IDL sessions every time you go through a loop.

Closing the IDL session is easy:

idl.close()

5 thoughts on “Using IDL within Python

  1. Does python have an equivalent to IDLs system variables, !X, !Y,!D, !P

    When a plot is made, these are automatically created/updated for a given plot.
    Finally, can you contour an image and retrieve the pixel positions of the contours?
    IDL uses PathXY and PathInfo variables to the contour procedure.

    • I believe so. Everything I've used the !P, !X and !Y system variables for (which isn't much) can be accomplished within matplotlib, but not as settings that would affect the entire python session.

      !P.MULTI is a regular part of the figure generation process. figure.add_subplot(321) for instance will return an axes object attached to a figure named 'figure', that you can attach plots to; it's the first (top left) plot in a 3x2 grid. The one to its immediate right would be 322, and so on. figure.add_subplot(111) (or no argument at all) would be a regular one-plot graph.
      !P.FONT is a property of axes.text() objects, and can be set to different values for each text object. It uses system TrueType fonts.
      !P.CHARSIZE is also a property of axes.text() objects as above, and can be set to TrueType point sizes or recognized names ("smaller", "medium", "x-large")
      !P.THICK is now a property of a line plot, as in axes.plot(x,y,linewidth=n)
      !X.THICK and !Y.THICK are properties of the axes object itself, and can be set with axes.axhline(linewidth=n) or axes.axvline(linewidth=n) respectively.
      I didn't use many other system commands, but I'm sure they're all there.

      My impression is that IDL builds up the plotted figure as it goes, item by item - if a program crashes you'll find a half-drawn diagram left in its wake. In contrast, with matplotlib you create a figure object, create axes objects that are attached to the figure object, and then you attach plot commands to each axis object. The objects are only evaluated and turned into plots at the very end when the program reaches a savefig() or show() command. So, it should be (and apparently is) easy to access and probably even modify elements of a plot after they're created. Matplotlib's own documentation states that trying to save the axes.contour() object returns a QuadContourSet object. QuadContourSet objects contain LineCollections objects (line segments) you can probably analyze.

      • Thanks Adric. Most helpful. IDL allows me to contour an
        image without axes in device[image pixel] coordinates, and
        returns a structure array(list) to parse a 2xN array of vertices(N and OFFSET tags),
        whether contour is a hill/valley(highlow tag). I need these to
        select, track and numerically integrate around features. To
        verify, I need to replot the image and plot individual contours.
        Finally, if I can understand how Python can be used the way IDL uses keywords, I will be set.
        Further, ITT-IDL doesn't use its own product the way it
        was designed. Most IDL programmers don't use it as designed, either. I teach IDL but need to learn Python.
        For your readers, IDL has three types of scripts. Procedures, Functions and main routines. Functions must return something. Procedures do something and should never be used to routinely return something. Finally, a main routine should only be used to develop code and is simply a listing of command line commands.
        Tim

  2. Would it be possible to re-post the webpage with the full explanation of how to use IDL within Python ? The link highlighted in blue ("nice page here") is not working.

    Thanks,
    George

    • The original website we linked to was deleted, but the information still exists on the Github page. The link now points there.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>