Appendix B — Setup & Usage

Note

These instructions are only relevant if you want to contribute to the handbook’s content or want to recreate the book locally.

B.1 Overview

This repository uses the following technologies:

  • conda environments (part of Anaconda) for creating and managing virtual environments

  • jupyter notebooks for course materials combining rich-text and code

  • quarto to generate the handbook from the course’s notebooks and some extra md files.

B.1.1 Outline

The workflow’s outline is as follows:

  1. (only once per machine) Setting up the environment
    1. conda and dependencies (Section B.2)
    2. quarto (Section B.3)
    3. git filters (Section B.4)
  2. Activating conda environment
  3. Edit changes
    1. in content (by editing the notebooks in content/folder)
    2. in book structure
  4. Preview changes locally using quarto preview
  5. Adding changes to repo (Section B.6)
    1. Source code: by pushing commits to the repo
    2. Live handbook: by publishing it to Github Pages using quarto publish gh-pages

B.2 Setting up the virtual environment

Virtual environments are a way to install all the dependencies (and their right version) required for a certain project by isolating python and libraries’ specific versions. Every person who recreatesthe virtual environment will be using the same packages and versions, reducing errors and increasing reproducibility.

This project, uses a conda environment called IM939, which will install all the packages and versions (as well as their dependencies) as defined in the file environment.yml at the root of this project.

B.2.1 Installing Anaconda

You will need to install Anaconda distribution your machine if it is not already installed. You can run the following command to see if it is already present in your system:

conda --help

If you get a command not found error, you will need to install Anaconda following these instructions on their website).

B.2.2 Creating the virtual environment

Note

You only need to do this once in the same machine. After the virtual environment is created we can always update it to follow any change in the file environment.yml

To recreate the virtual environment from environment.yml, run the following command:

conda env create -f environment.yml

or, if we want to install the environment within the project:

conda env create --prefix env -f environment.yml

B.2.3 Activating the virtual environment

Once the environment has been created, it needs to be activated by typing the following command

conda activate IM939

or, if it is stored in env/ folder:

conda activate env/
Once per session

You will need to activate the environment every time you open your editor anew.

Deactivate virtual environment:

If you want, you can always deactivate your environment (and, actually open the default one, called base) by running:

conda deactivate

Update virtual environment from environment.yml:

conda env update -f environment.yml

Freeze used dependencies into a file

We can create a file (in this case environment.yml) containing the exact libraries and versions used in the current environment. This can be useful to update the versions used in the environment in the future.

conda env export > environment.yml

B.3 Installing quarto

Quarto is a binary that needs to be downloaded from https://quarto.org/docs/get-started/ and manually installed by running the installer.

B.4 Git setup - Preventing commits with execution cells

This handbook relies on jupyter notebooks. Quarto renders any *.ipynb file into a handbook, and displays the output of any code block, according to the settings. Regretfully, that means that it executes every code cell and therefore, jupyter notebooks stores the results in the notebook too, which is not what we’d like to do.

To prevent executed cells from being pushed to the repo in an automated way, the following command must be run once within the repository’s root:

nbstripout --install

This will setup a git filter and is only needed once. Any notebooks being committed to the repo will be striped out from executed cells.

B.5 Editing the book’s structure

This handbook is organised in sections and chapters. The book’s structure is defined in the project’s configuration file: _quarto.yml, particularly in the chapters’ section (Line 38).

If you want to add new structure

Please, refer to this page at quarto’s official documentation for more information: https://quarto.org/docs/books/book-structure.html

B.5.1 Crosslinking

Quarto cross references provide automatic numbering and reference creation for figures, tables, equations, sections, listings, theorems, and proofs. In books, cross references work the same way except they can reach across chapters.

Please, refer to these pages at quarto’s official documentation for more information: https://quarto.org/docs/books/book-crossrefs.html and https://quarto.org/docs/authoring/cross-references.html

B.6 Recreating the handbook

Tip

Quarto has extensive documentation at their website (specifically in this page about authoring document and this other on managing books).

The workflow can be summarised as follows (detailed instructions below):

  1. Create new content or edit existing one

    1. Create a .md, .iypnb or .qmd file and put it in the /content/ folder
    2. Add an entry to to the table of contents in _quarto.yml (more info in quarto

  2. Edit existing content in /content/ folder

  3. Render book locally to see changes

  4. Commit & Push changes to the source documents in /content/ folder

    git commit -a -m "My fancy message"
git push origin main

  5. Publish book online

B.6.1 Rendering the book locally:

Rendering the book will convert jupyternotebooks, qmd files or md files listed in _quarto.yml’s table of contents into a a book, applying the styles and configurations from _quarto.yml. Do this to preview how your changes would look like as a book. From the repo’s root, run:

quarto render
First time render

On its first run, this command will take several minutes to process. This is because quarto will run and execute the computations in every cell within every jupyter notebook, some of which are really time consuming. The good news, is that quarto will create a cached version of it (stored in the /_freeze/ folder) , which means that further runs of quarto render will not need to execute the cells again (unless the original jupyternotebook is changed or the corresponding folder is deleted).

If you want the cache to be regenerated:

quarto render --cache-refresh

B.6.2 Publishing book to github pages

This book is published using GitHub pages, and assumes that your rendered book will be located on a dedicated branch called gh-pages which needs to be created in the repo if not present already. Also, the repository needs to be configured as to use that branch’s root for publishing a GithubPage.

Once gh-pagesbranch has been created, run the following command:

quarto publish gh-pages

This command will render the book in the branch gh-pages and will push it to the corresponding branch in our repo and then checking out again to the previous branch (usually, main). More info about it here: https://quarto.org/docs/publishing/github-pages.html

Other useful resources
  • Notebook embedding: https://quarto.org/docs/authoring/notebook-embed.html