Building the website

The builder.Builder class is the object that is responsible for assembling the profiling website, as well as the developer documentation. The choice to use a class over a script or function was due to a combination of minimising parameter passing balanced against the ability to break the process of building the website into manageable, human-readable chunks.

The website build can be triggered by passing the website_builder/builder.py script to Python on the command line. You may pass the --help option for additional options, which are just convenience wrappers for builder.Builder class functionality.

python website_builder/builder.py

It is also worth mentioning a few constants that are defined in this module.

• PROFILING_TABLE_MATCH_STRING: This string appears in profiling.rst, and is replaced with the contents of the lookup table that is inserted into that page.

• RUN_PLOTS_MATCH_STRING: This string appears in run-statistics.rst, and is replaced with the various plots that are produced for the profiling_statistics.Statistic instances that require plotting.

• DF_EXTRA_COLUMNS: These are additional columns that need to be computed from the statistics read in, or from examination of the source branch tree structure.

The Builder

The heavy-lifting of the website construction is done by builder.Builder. Build options are set on initialisation (although there is nothing stopping you from editing attribute values of a class instance at runtime), and the construction can be invoked with the builder.Builder.build method.

class builder.Builder(source_branch: str, build_dir: str, flatten_profiling_html: bool = True, clean_build: bool = True, web_source_dir: ~pathlib.Path = PosixPath('/home/runner/work/TLOmodel-profiling/TLOmodel-profiling/source'), stats_file_ext: str = 'stats.json', website_plaintext_format: ~typing.Literal['md', 'rst'] = 'rst', size_of_plots: ~typing.Tuple[int, int] = (12, 6), cols_for_lookup_table: ~typing.List[str] = <factory>)

Handles the construction of the gh-pages website, as per the process detailed in http://github-pages.ucl.ac.uk/TLOmodel-profiling.

Build options are configured on initialisation of a class instance; members with default values can be passed as keyword arguments to the constructor to overwrite this default behaviour.

Use the .build() method to run the website build.

Parameters:

• source_branch (str) – The git repository branch that holds the profiling results.

• build_dir (str) – The directory into which to build the website.

• flatten_profiling_html (bool, optional) – If True (default), static HTML files generated from profiling session outputs will be placed into the profiling_html_dir in a flat structure. If False, they will retain the YYYY/MM/DD/{name}.html structure of the source branch.

• clean_build (bool, optional) – If True (default), the build directory will be purged before starting.

• web_source_dir (Path, optional) – Folder containing website source and template files. Default is ./source.

• stats_file_ext (str, optional) – The file extension of the statistics files on the source branch. Default is .stats.json.

• website_plaintext_format (Literal["md", "rst"], optional) – Whether the website templates to be edited are MarkDown (md) or ReStructured Text (rst) (default).

• size_of_plots (Tuple[int, int], optional) – The figure size to set for all plots that will be produced. Default is (12, 6).

• cols_for_lookup_table (List[str], optional) – The columns from the website DataFrame to include in the profiling runs lookup table, the columns will be ordered as provided in the list. Default is [Start time, Link, Commit, Triggered by, Session duration (s)].

build() → None

Build the website using the set configurations.

fetch_rendered_HTML() → None

Fetch rendered profiling sessions from the source branch, saving the HTML files to the static directory in the staging area.

This also populates the “Link” column of the DataFrame, so that each row now knows how to link to its profiling run output.

format_as_title(text: str, char_if_rst: str = '-') → str

Format the text string provided into a plaintext title, in the format corresponding to self.website_plaintext_format.

Parameters:

• text (str) – String to format into a title.

• char_if_rst (str) – The character to place underneath the title if formatting an .rst file.

make_stats_plots() → None

Create and save plots of the statistics that are flagged as such.

This produces a dictionary whose keys are the names of the plots, and whose values are the paths to the images of those plots. This dictionary populates the self.plot_dict attribute.

parse_stats_files_to_df() → None

Locate all statistics files on the source branch, and populate the DataFrame with their information.

property plot_folder: Path

The folder within the build directory that will contain any plots that are generated from the statistics collected across the profiling runs.

property profiling_html_dir: Path

The folder within the build directory that will contain any HTML files generated from profiling sessions.

property profiling_table_file: Path

The file into which the profiling results lookup table will be inserted.

property staging_dir: Path

The intermediate folder that sphinx-build will use as its source for building the website. This folder will contain static files that have to be generated from the results stored on the source branch; such as:

• Static HTML files from profiling runs,

• The rst files with the lookup tables inserted,

• The run-statistics plots.

property static_folder: Path

The folder containing static html files that are to be linked to.

property statistics_plots_file: Path

The file into which the run statistics plots will be inserted.

write_profiling_lookup_table() → str

Create the source for the profiling results lookup table, by extracting the relevant columns from the DataFrame.

write_run_stats_page() → None

Write the text for the webpage that will display statistics which are plotted across multiple profiling runs, then insert this into the webpage.