XClose

MPHY0021: Research Software Engineering With Python

Home
Menu

Python not in the Notebook

We will often want to save our Python classes, for use in multiple Notebooks. We can do this by writing text files with a .py extension, and then importing them.

Writing Python in Text Files

You can use a text editor like VS Code or Spyder. If you create your own Python files ending in .py, then you can import them with import just like external libraries.

You can also maintain your library code in a Notebook, and use %%writefile to create your library, though this is not encouraged!

Libraries are usually structured with multiple files, one for each class.

We will be turning the code we have written for the maze into a library, so that other code can reuse it.

We group our modules into packages, by putting them together into a folder. You can do this with explorer, or using a shell, or even with Python:

In [1]:
import os
if 'mazetool' not in os.listdir(os.getcwd()):
    os.mkdir('mazetool')
In [2]:
%%writefile mazetool/maze.py

from .room import Room
from .person import Person

class Maze(object):
    def __init__(self, name):
        self.name = name
        self.rooms = []
        self.occupants = []
        
    def add_room(self, name, capacity):
        result = Room(name, capacity)
        self.rooms.append(result)
        return result
        
    def add_exit(self, name, source, target, reverse= None):
        source.add_exit(name, target)
        if reverse:
            target.add_exit(reverse, source)
            
    def add_occupant(self, name, room):
        self.occupants.append(Person(name, room))
        room.occupancy += 1
    
    def wander(self):
        "Move all the people in a random direction"
        for occupant in self.occupants:
            occupant.wander()
                
    def describe(self):
        for occupant in self.occupants:
            occupant.describe()
            
    def step(self):
        house.describe()
        print()
        house.wander()
        print()
        
    def simulate(self, steps):
        for _ in range(steps):
            self.step()
Writing mazetool/maze.py
In [3]:
%%writefile mazetool/room.py
from .exit import Exit


class Room(object):
    def __init__(self, name, capacity):
        self.name = name
        self.capacity = capacity
        self.occupancy = 0
        self.exits = []
        
    def has_space(self):
        return self.occupancy < self.capacity
    
    def available_exits(self):
        return [exit for exit in self.exits if exit.valid() ]
            
    def random_valid_exit(self):
        import random
        if not self.available_exits():
            return None
        return random.choice(self.available_exits())
    
    def add_exit(self, name, target):
        self.exits.append(Exit(name, target))
    
Writing mazetool/room.py
In [4]:
%%writefile mazetool/person.py

class Person(object):
    def __init__(self, name, room = None):
        self.name=name
        self.room=room
    
    def use(self, exit):
        self.room.occupancy -= 1
        destination=exit.target
        destination.occupancy +=1
        self.room=destination
        print(self.name, "goes", exit.name, "to the", destination.name)
    
    def wander(self):
        exit = self.room.random_valid_exit()
        if exit:
            self.use(exit)
            
    def describe(self):
        print(self.name, "is in the", self.room.name)
Writing mazetool/person.py
In [5]:
%%writefile mazetool/exit.py

class Exit(object):
    def __init__(self, name, target):
        self.name = name
        self.target = target
    
    def valid(self):
        return self.target.has_space()
Writing mazetool/exit.py

In order to tell Python that our "mazetool" folder is a Python package, we have to make a special file called __init__.py. If you import things in there, they are imported as part of the package:

In [6]:
%%writefile mazetool/__init__.py
from .maze import Maze # Python 3 relative import
Writing mazetool/__init__.py

In this case we are making it easier to import Maze as we are making it available one level above.

Loading Our Package

We just wrote the files, there is no "Maze" class in this notebook yet:

In [7]:
myhouse = Maze('My New House')
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
/tmp/ipykernel_13904/2940705298.py in <module>
----> 1 myhouse = Maze('My New House')

NameError: name 'Maze' is not defined

But now, we can import Maze, (and the other files will get imported via the chained Import statements, starting from the __init__.py file.

In [8]:
import mazetool

Let's see how we can access the files we created:

In [9]:
mazetool.exit.Exit
Out[9]:
mazetool.exit.Exit
In [10]:
from mazetool import Maze
In [11]:
house = Maze('My New House')
living = house.add_room('livingroom', 2)

Note the files we have created are on the disk in the folder we made:

In [12]:
import os
In [13]:
os.listdir(os.path.join(os.getcwd(), 'mazetool') )
Out[13]:
['person.py', 'exit.py', 'maze.py', '__init__.py', 'room.py', '__pycache__']

You may get also .pyc files. Those are "Compiled" temporary python files that the system generates to speed things up. They'll be regenerated on the fly when your .py files change. They may appear inside the __pycache__ directory.

The Python Path

We want to import these from notebooks elsewhere on our computer: it would be a bad idea to keep all our Python work in one folder.

The best way to do this is to learn how to make our code into a proper module that we can install. We'll see more on that in a few lectures' time (notebook).

Alternatively, we can add a folder to the "PYTHONPATH", where python searches for modules:

In [14]:
import sys
print('\n'.join(sys.path[-3:]))
/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages
/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/IPython/extensions
/home/runner/.ipython
In [15]:
from pathlib import Path
sys.path.append(os.path.join(Path.home(), 'devel', 'libraries', 'python'))
In [16]:
print(sys.path[-1])
/home/runner/devel/libraries/python

I've thus added a folder to the list of places searched. If you want to do this permanently, you should set the PYTHONPATH Environment Variable, which you can learn about in a shell course, or can read about online for your operating system.