Editing PYTHONPATH (or “Where’s my module?!”)

Python not finding your modules with import MyModule for some reason? It’s probably your PYTHONPATH. Here’s how you edit it so that Python sees all your modules.

A Word About Directories

Firstly, if you are the kind of person who keeps files scattered about your computer, dumps everything into the root directory, or is afraid to use those nice little blue folders, then perhaps programming and computers in general are not for you.

Logical and neat directory structure will make your own, your computer’s and your collaborators’ lives much easier.

My recommendation: Create a master code directory (called “Modules” or something) in your Documents folder. This will be the new home of all your future Python projects.

Now every time you create a .py file, it should go into a project folder inside your Modules directory, i.e. /Documents/Modules/NewProject/MyModule.py. Note that you should have no actual modules inside your Modules directory! Keep those puppies in their warm, snuggly project subdirectories.

This way you can also initialize that project folder (i.e. /NewProject) as a Git repository and just push and pull the contents to keep it up-to-date!

Editing PYTHONPATH

Python won’t just search your computer for the MyModule.py file you’re trying to import. You have to tell it explicitly each time where to get it. The PYTHONPATH is a list of directories for your computer to check whenever you type import MyModule into the interpreter.

To add a path, launch ipython and type:

import sys
sys.path.append("path/to/Modules")
print sys.path

Note: You only have to update your PYTHONPATH once, not every time you use Python!

So now your path is updated but this is only the path to your master Python folder.

In order to have Python see the modules inside each subdirectory, add a blank file called __init__.py to each subdirectory (with two underscores on each side).

Now to import the module and use a function called foo() do:

from NewProject import MyModule as m
m.foo()

That is, it’s checking the main python directory you added to your PYTHONPATH, then looking within the NewProject subdirectory via the __init__.py file for the module you’re trying to import.

11 thoughts on “Editing PYTHONPATH (or “Where’s my module?!”)

  1. The directory structure I encourage is ~/Code/Python/Project.

    and isn’t there a way to set the PYTHONPATH in the .cshrc so it’s explicitly written somewhere and clear where and how to change it?

  2. I’ve found that trying to edit the PYTHONPATH within Python doesn’t stick. When you open Python again, you’re back to the same old list.

    The persistent solution: edit .bashrc or .tcshrc with commands to change PYTHONPATH. (not sure what shell your terminal is using? Run ‘echo $0’ That’s the number zero.)

    For TCSH shells, edit .tcshrc and add a line like this:
    setenv PYTHONPATH ${PYTHONPATH}:/Users/riedel/Code/Python/:/Users/riedel/Code/Python/Astrotools:/Users/riedel/Code/Python/Database:/Users/riedel/Code/Python/Miscellaneous

    For BASH shells, edit .bashrc and add something like these lines:
    PYTHONPATH=”${PYTHONPATH}:/Users/riedel/Code/Python/:/Users/riedel/Code/Python/Astrotools:/Users/riedel/Code/Python/Database:/Users/riedel/Code/Python/Miscellaneous”

    export PYTHONPATH

    The changes will take effect in newly opened terminals.

    According to what I’ve read, all you SHOULD need is to specify the base code folder (mine is /Users/riedel/Code/Python) and add __init__.py files in the subfolders below it. That doesn’t seem to work for the BDNYC module that comes with the python database, hence this approach.

    • Great! Thank you soooo much. Finally find the solution. I used to edit $PYTHONPATH every time before I run the code. You save me from it…

  3. Is PYTHONPATH a defined variable? iPython told me it wasn’t, so I tried using just the word “path” in my .tcshrc file, which it allowed me to do, until I then tried to call a module, and it still could not find it.

  4. So, the more complete explanation:
    http://docs.python.org/2/tutorial/modules.html#the-module-search-path
    http://docs.python.org/2/using/cmdline.html#envvar-PYTHONPATH

    Python looks for modules in three places when it starts up:
    1.) The directory that python or ipython was launched from
    2.) A default location relative to where the actual python executable is- for instance, my active python is /Library/Frameworks/EPD64.framework/Versions/7.3/bin/python so all the python modules are stored in directories below ../lib/python2.7/
    3.) $PYTHONPATH, if it exists. By default on Mac OS X + Enthought Python Distribution, it doesn’t – so the commands in my previous posts only needed to be:
    (.tcshrc)
    setenv PYTHONPATH /Users/riedel/Code/Python/:/Users/riedel/Code/Python/BDNYC:/Users/riedel/Code/Python/BDNYC/BDtools:/Users/riedel/Code/Python/BDNYC/BDdatabase:/Users/riedel/Code/Python/BDNYC/BDmisc
    and
    (.bashrc)
    PYTHONPATH=”/Users/riedel/Code/Python/:/Users/riedel/Code/Python/BDNYC:/Users/riedel/Code/Python/BDNYC/BDtools:/Users/riedel/Code/Python/BDNYC/BDdatabase:/Users/riedel/Code/Python/BDNYC/BDmisc”
    export ${PYTHONPATH}

    (i.e. no $PYTHONPATH at the beginning, to include anything else already defined – you definitely DO want to do that with $PATH and other UNIX system variables, though)

    python and ipython read the directories out of those three locations when they start, and set up python’s internal sys.path variable accordingly. Once you’ve loaded python or ipython, they don’t know or care about $PYTHONPATH, just whatever was set in sys.path.

  5. Can’t you just execute sys.path (after importing sys), put your modules folder in one of the locations displayed (adding __init__.py to the modules folder), take out the sub folders, delete the main folder and put shortcuts where ever you want? Just a suggestion that worked for me.

  6. Pingback: PYTHONPATH | monsteR#bash

  7. What about a .py file that loads a .ui file?

    I have a .ui file that uses a widget from a dll plugin. When I run the .py file I get the ‘No Module’ error for the widget.

    Is this a path / __init__.py issue or a widget issue? (I have the widget compiled to a dll from Qt Creator, but perhaps needs something else in order for python to use it?)

  8. The problem with dynamic adding to path within code is that you have (very much) duplicate code in hundreds of files, especially when your code need to work cross platform.

Leave a Reply to kelle Cancel reply

Your email address will not be published. Required fields are marked *