diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 7fffe30..70bcc8c 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -23,8 +23,9 @@ before_script: # This job tests if the book compiles gracefully after a new addition test-build: stage: test - only: - - main + rules: + - if: $CI_PIPELINE_SOURCE == "merge_request_event" + - if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE == "push" script: - pip install -r requirements.txt - jupyter-book clean book/ diff --git a/book/_toc.yml b/book/_toc.yml index 3d8a946..f04c1ac 100644 --- a/book/_toc.yml +++ b/book/_toc.yml @@ -7,6 +7,18 @@ parts: - file: installation-and-setup/git-setup - file: installation-and-setup/vscode-setup - file: installation-and-setup/jupyter-book-setup + - caption: Workflows + chapters: + - file: workflows/git-basic + sections: + - file: workflows/git-clone + - file: workflows/git-branch + - file: workflows/git-stage-commit-push + - file: workflows/git-merge + - file: workflows/git-advanced + sections: + - file: workflows/git-merge-conflict + - file: workflows/gitlab - caption: Cookbook chapters: - file: cookbook/recipes.md diff --git a/book/images/git-branch-menu-main.png b/book/images/git-branch-menu-main.png new file mode 100644 index 0000000..27e0e96 Binary files /dev/null and b/book/images/git-branch-menu-main.png differ diff --git a/book/images/git-branch-selection-menu.png b/book/images/git-branch-selection-menu.png new file mode 100644 index 0000000..8fd7998 Binary files /dev/null and b/book/images/git-branch-selection-menu.png differ diff --git a/book/images/git-file-added.png b/book/images/git-file-added.png new file mode 100644 index 0000000..bb5312c Binary files /dev/null and b/book/images/git-file-added.png differ diff --git a/book/images/git-new-file.png b/book/images/git-new-file.png new file mode 100644 index 0000000..fd7a104 Binary files /dev/null and b/book/images/git-new-file.png differ diff --git a/book/images/git-push.png b/book/images/git-push.png new file mode 100644 index 0000000..f1105d6 Binary files /dev/null and b/book/images/git-push.png differ diff --git a/book/images/gitlab-branch-name.png b/book/images/gitlab-branch-name.png new file mode 100644 index 0000000..b99aa95 Binary files /dev/null and b/book/images/gitlab-branch-name.png differ diff --git a/book/images/gitlab-branch.png b/book/images/gitlab-branch.png new file mode 100644 index 0000000..d92b330 Binary files /dev/null and b/book/images/gitlab-branch.png differ diff --git a/book/images/gitlab-clone.png b/book/images/gitlab-clone.png new file mode 100644 index 0000000..46f8c3e Binary files /dev/null and b/book/images/gitlab-clone.png differ diff --git a/book/images/gitlab-finished-merge-request.png b/book/images/gitlab-finished-merge-request.png new file mode 100644 index 0000000..567679f Binary files /dev/null and b/book/images/gitlab-finished-merge-request.png differ diff --git a/book/images/gitlab-merge-request-description.png b/book/images/gitlab-merge-request-description.png new file mode 100644 index 0000000..691aaf1 Binary files /dev/null and b/book/images/gitlab-merge-request-description.png differ diff --git a/book/images/gitlab-merge-request.png b/book/images/gitlab-merge-request.png new file mode 100644 index 0000000..4083b88 Binary files /dev/null and b/book/images/gitlab-merge-request.png differ diff --git a/book/images/gitlab-new-branch-repo.png b/book/images/gitlab-new-branch-repo.png new file mode 100644 index 0000000..b772845 Binary files /dev/null and b/book/images/gitlab-new-branch-repo.png differ diff --git a/book/images/vscode-welcome.png b/book/images/vscode-welcome.png new file mode 100644 index 0000000..83d7f2b Binary files /dev/null and b/book/images/vscode-welcome.png differ diff --git a/book/installation-and-setup/git-setup.md b/book/installation-and-setup/git-setup.md index 8bbf93e..b806060 100644 --- a/book/installation-and-setup/git-setup.md +++ b/book/installation-and-setup/git-setup.md @@ -64,4 +64,8 @@ The final step is to provide a username and email address to Git, so our commits And type the following command to set the email address: - git config --global user.email "" \ No newline at end of file + git config --global user.email "" + +## Configuring Git Bash + +Git Bash is a shell (also referred to as "the terminal") that is automatically installed with git. It provides Unix-like commands on a Windows OS, and is recommended for working with open source projects like Jupyter (for example `pip install`, building Jupyter books, using Git Bash in VS Code, etc). To do this, `conda` needs to be setup properly; this is described in the {ref}`instructions for setting up VS Code `. \ No newline at end of file diff --git a/book/installation-and-setup/vscode-setup.md b/book/installation-and-setup/vscode-setup.md index b162149..446ee0f 100644 --- a/book/installation-and-setup/vscode-setup.md +++ b/book/installation-and-setup/vscode-setup.md @@ -14,8 +14,64 @@ When working with Jupyter Book (or Notebooks in general), these extensions are h - [MyST-Markdown](https://marketplace.visualstudio.com/items?itemName=ExecutableBookProject.myst-highlight) - adds support for MyST markdown, including previews. - [YAML](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) - enables validation of `.yml` files (the format of the Jupyter Book configuration files). +## Using the Terminal in VS Code (Windows only) + +By default, Powershell is the default terminal used in VS Code. However, Git Bash is recommended because it provides Unix-like commands, making it much easier to find help on sites like Stack Exchange. It also works better with open source projects like Jupyter. Git Bash installs automatically with git; to set as default: +* Type `Ctrk+Shift+P` to open the command palette (it's also the top item in the "View" menu list) +* Start type "terminal default" and you will soon see and be able to select the setting "Terminal: Select Default Profile" (hit `Enter`) +* select Git Bash and hit "Enter" again to set as default + +When creating a new terminal via the menu bar you should now automatically start with a Git Bash terminal. Note that if a Powershell terminal still opens by default, you can always create a new terminal and manually select Git Bash. + ## Getting Anaconda to work in the VS Code terminal -```{tab} Windows +(vscode-setup-bash)= +````{tab} Windows - Git Bash (preferred) + +There are two things to do before you can use conda in VS Code: 1) adding Anaconda to your PATH variables (as user, *not* as system/admin), and 2) setting up Anaconda to run on Git Bash. + +1. update the `PATH` variables: +* search for 'env' in the windows search to find the settings page. Anaconda doesn't recommend adding them to the system `PATH`, therefore when searching for the setting with the Windows search, so be careful to open **"edit environment variables for your account"**; do **not** "edit the system environment variables". +* Identify the box for adding variables for a specific user (the top box), not the system. +* select the line with 'Path' and click edit +* Add the `Anaconda3` and `Anaconda3/Scripts` directories to `PATH`. + +You will know if this worked because the `conda` commands below will not produce an error. You can also check by opening a Git Bash terminal and listing your Anaconda environments with `conda env list`. + +2. To setup Anaconda, open git bash and run the collowing commands (note, the first two commands to create `myenv` can be skipped if you already have an Anaconda environment; e.g., default is `base`): +* `conda create -n myenv python` +* `conda install -n myenv numpy` +* run `conda init bash` to link conda to Git Bash +* restart shell (type `exit` to close) +* run `conda activate myenv` +* restart shell + +Check if it worked by opening a notebook in an aribtrary `DIR` using the command `jupyter lab --notebook-dir DIR`. +```` +````{tab} MacOS +Usually, Anaconda works by default in the MacOS terminal. To check if it works for you, open a new terminal in VS Code by going clicking **View > Terminal** in the top menu bar. In the terminal window that appears, type `conda` and execute. If you get help page for `conda`, you're all set. If instead you get something like: + + zsh: command not found: conda + +then do the following: + +1. Find out where Anaconda is installed on your system. If you left everything as default during the installation, Anaconda is installed in `/Users//anaconda3` if you used the install script, or in `/Users//opt` if you used the graphical interface to install Anaconda. + +2. Execute the following line: + + source /bin/activate + +3. Then: + + conda init zsh + +4. Restart VS Code, then test by executing `conda` again. This time, it should give you the help page. You now have access to you Anaconda environments in the VS Code terminal. +```` +````{tab} Windows - Powershell + +```{note} +This is not the preferred installation method: Powershell has a different set of terminal commands that are not Unix-based, whereas Git Bash does. Since the vast majority of coding information and examples on the internet related to Jupyter and other open source projects uses Unix-based commands, we strongly recommend you always run Git Bash for your work in the terminal. +``` + On Windows, Anaconda usually does not work in the integrated VS Code terminal (Powershell). To check if it works for you, open a new terminal in VS Code by going clicking **View > Terminal** in the top menu bar. In the terminal window that appears, type `conda` and execute. If you get help page for `conda`, you're all set. If instead you get something like: conda : the term 'conda' is not recognized ... @@ -34,23 +90,4 @@ then do the following steps: 5. The command will ask you for some permissions. To do so, type `A`, then press enter. 6. Restart VS Code, then test by executing `conda` again. This time, it should give you the help page. You now have access to you Anaconda environments in the VS Code terminal. -``` -```{tab} MacOS -Usually, Anaconda works by default in the MacOS terminal. To check if it works for you, open a new terminal in VS Code by going clicking **View > Terminal** in the top menu bar. In the terminal window that appears, type `conda` and execute. If you get help page for `conda`, you're all set. If instead you get something like: - - zsh: command not found: conda - -then do the following: - -1. Find out where Anaconda is installed on your system. If you left everything as default during the installation, Anaconda is installed in `/Users//anaconda3` if you used the install script, or in `/Users//opt` if you used the graphical interface to install Anaconda. - -2. Execute the following line: - - source /bin/activate - -3. Then: - - conda init zsh - -4. Restart VS Code, then test by executing `conda` again. This time, it should give you the help page. You now have access to you Anaconda environments in the VS Code terminal. -``` \ No newline at end of file +```` \ No newline at end of file diff --git a/book/workflows/git-advanced.md b/book/workflows/git-advanced.md new file mode 100644 index 0000000..9691271 --- /dev/null +++ b/book/workflows/git-advanced.md @@ -0,0 +1,4 @@ +(git-advanced)= +# Advanced Git topics + +Under construction. \ No newline at end of file diff --git a/book/workflows/git-basic.md b/book/workflows/git-basic.md new file mode 100644 index 0000000..ccaa245 --- /dev/null +++ b/book/workflows/git-basic.md @@ -0,0 +1,23 @@ +# Working with Git in VS Code + +In this section you will learn how to perform basic Git actions in VS Code. Courses on Git usually make use of the command line interface to do these actions, but this introduces a steep learning curve. Instead, using Git from VS Code is much more visual (i.e., more "clicky"), and therefore easier to use for those inexperienced with the command line. + +## What is Git? +Git is a version control system (VCS). In short, a VCS is a tool that helps you keep track of changes to you codebase (e.g., a Jupyter Book project!) over time. It allows you to keep track of the history of changes made to your code, work collaboratively with others, and revert to a previous version of the code if needed. + +## What is GitLab? +GitLab is platform that hosts Git repositories, which can also be accessed through a web interface. Essentially, GitLab hosts the main repository that the members of a project interact with. Additionally, GitLab has plenty of other useful features such as issue tracking and code review. A selection of these features that are particularly useful for us as Jupyter Book developers will be highlighted in a {ref}`later section . + +## Terminology +This is a (non-exhaustive) list of terminology that will be used throughout this section: + +- Repository: a collection of files and folders, along with a history of their changes and who made them. +- Commit: a snapshot of the current state of the repository. +- Staging: preparation of files to be committed. During staging, we propose files to be committed. +- Branch: a development line. +- Local: on your own computer. +- Remote: on someone elses computer. +- Pushing: uploading new commits to the remote repository. +- Pulling: downloading new commits from the remote repository to your local repository. +- Tracked (files): files that Git knows about and keeps track of their history. +- Untracked (files): files that Git does not yet know about. \ No newline at end of file diff --git a/book/workflows/git-branch.md b/book/workflows/git-branch.md new file mode 100644 index 0000000..e14f894 --- /dev/null +++ b/book/workflows/git-branch.md @@ -0,0 +1,59 @@ +(git-branch)= +# Branches + +Branches are a very useful feature of Git. Branches allow you to work on multiple versions of your codebase simultaneously; you create a copy of your codebase, on which you can work independently of the main codebase. Some advantages of working with branches are: + +- Isolation: when you work on your own branch, your changes are isolated from the main codebase. Unfinished or unreviewed parts of your book are "hidden". +- Collaboration: branches make it easier for multiple project members to work on the same codebase simultaneously. If every member works on their own branch, they can make their changes without having to worry about interfering with another members' changes. + +## How to make a branch + +The act of making a new branch from an existing one is called *branching*. Usually, you want to branch from the `main` branch, but you can of course also choose to branch from another branch. The branch from which you create a new branch is called the *source* branch. + +1. On GitLab, navigate to the repository of your project. Under the project description, you should see a menu button with a "+" in it (see {numref}`new-branch-button`). Click this button, then click on "New branch". + +```{figure} ../images/gitlab-branch.png +:name: new-branch-button + +Click the highlighted button to make a new branch. +``` + +2. A new page opens ({numref}`new-branch-menu`), on which you can specify the name of your branch and the source branch. Try to think of a good name for your branch. For example, `chapter-2` is a suitable name if we're going to be working on Chapter 2 of our book. After you've made sure that the selected source branch is correct, click "Create branch". + +```{figure} ../images/gitlab-branch-name.png +:name: new-branch-menu + +We're making a new branch called `chapter-2` from the `main` branch. +``` + +3. You should be sent back to an overview page of your repository. In the branch selection menu (see {numref}`new-branch-repo`), it should now say `chapter-2` instead of `main`. + +```{figure} ../images/gitlab-new-branch-repo.png +:name: new-branch-repo + +The circled menu is the branch selection menu. +``` + +4. The final step is to *check out* our new branch in VS Code. *Checking out* means that we're changing our working branch (i.e., the one to which we will be committing our changes) from one to the other. In VS Code, open the branch menu from the bottom left (see {numref}`git-branch-menu-main`). You'll be be greeted with a menu that looks like the one in {numref}`git-branch-selection-menu`. In this menu, we get an overview of the branches of our repository. Notice that our new branch `chapter-2` appears as `origin/chapter-2`, and has a cloud icon in front of it. This means that the branch only exists on the remote repository. Check out the branch by simply clicking on it. Check that the branch is indeed checked out by looking at the branch menu button in the bottom left again; it should now say the name of your branch instead of `main` (in our case `chapter-2`). + +```{figure} ../images/git-branch-menu-main.png +:name: git-branch-menu-main + +Location of the branch menu. +``` + +```{figure} ../images/git-branch-selection-menu.png +:name: git-branch-selection-menu + +The branch selection menu, where our new branch has shown up. +``` + +5. Now that we've created and checked out our new branch, we can start {ref}`making and committing changes `. + +## Predefined branches + +If your repository is located in the [Interactive Textbooks CiTG](https://gitlab.tudelft.nl/interactivetextbooks-citg) group on GitLab, it is likely that it already has some predefined branches in addition to `main`, namely `publish` and `publish-html`. These branches serve the following purposes: + +- `main`: this is the main development branch on which new changes are collected. When the work on your own branch is done, you make a merge request to add your changes to `main`. This branch is protected, meaning that all changes to this branch have to be done via merge requests and you cannot push to it directly. +- `publish`: this is the version of the book that is visible to the public (i.e., the version that the students will see). Again, you cannot push directly to this branch. +- `publish-html`: this branch contains the HTML of the version of the book that lives on the `publish` branch. When a new version of the book is merged in the `publish` branch, the new book is built using the `jupyter book build` command and the HTML is uploaded to `publish-html`. The webserver is constantly fetching this branch and pulling any incoming changes. diff --git a/book/workflows/git-clone.md b/book/workflows/git-clone.md new file mode 100644 index 0000000..5ed959c --- /dev/null +++ b/book/workflows/git-clone.md @@ -0,0 +1,24 @@ +(git-clone)= +# Cloning a repository + +Before we can start working on our Jupyter Book, we of course need to have the repository on our own system. This is done by *cloning* the repository, and here's how you do it with VS Code: + +1. On GitLab, go to the repository of your book. Click on the blue button that says "Clone" ({numref}`gitlab-clone`), and copy the link under "Clone with SSH". + +```{figure} ../images/gitlab-clone.png +:name: gitlab-clone + +The cloning menu. +``` + +2. Open a new VS Code window. You should be greeted with a welcome page ({numref}`vscode-welcome`). On this page, click "Clone Git Repository". Alternatively, open the Source Control menu from the left side bar, and click on "Clone Repository". + +```{figure} ../images/vscode-welcome.png +:name: vscode-welcome + +The VS Code welcome screen. +``` + +3. On the top of your screen, you should get a prompt asking you for a URL. In this field, paste the link you copied from GitLab and press enter. It will ask you to choose the location of the repository. After you've chosen a location, a final prompt will show up asking if you want to open the repository. Click on "Open". + +4. You now have a local copy of the repository. Next, {ref}`make a new branch ` for your work. \ No newline at end of file diff --git a/book/workflows/git-merge-conflict.md b/book/workflows/git-merge-conflict.md new file mode 100644 index 0000000..f6fa5cb --- /dev/null +++ b/book/workflows/git-merge-conflict.md @@ -0,0 +1,4 @@ +(merge-conflicts)= +# Merge conflicts + +Under construction. \ No newline at end of file diff --git a/book/workflows/git-merge.md b/book/workflows/git-merge.md new file mode 100644 index 0000000..5402a75 --- /dev/null +++ b/book/workflows/git-merge.md @@ -0,0 +1,33 @@ +(git-merge)= +# Merging + +We're finished with our chapter, and now it's time to include all of our commits in the `main` branch. + +1. Make sure that you've committed all your new sections and changes, and that they're pushed to the remote repository. + +2. Go the repository overview on GitLab. In the left menu bar, go to "Merge requests". On the Merge requests page, click "New merge request". + +3. You can now choose a source branch and a target branch. Since we want to merge our changes from `chapter-2` *into* `main`, `chapter-2` is the source branch and `main` is the target branch ({numref}`gitlab-merge-request`) . Once you've selected the correct branches, click "Compare branches and continue". + +```{figure} ../images/gitlab-merge-request.png +:name: gitlab-merge-request + +Creating a new merge request to merge `chapter-2` into `main`. +``` + +4. On the next page, you can give your merge request a name, and provide a description of your changes ({numref}`gitlab-merge-request-description`). If somebody has to review your changes, you can add them in the "Reviewer menu". They will then be notified via email that they need to review your work. Finally, watch out for the option "Delete source branch when merge request is accepted". If you want to keep working on the same branch, even after your previous work has been merged, make sure you untick the box! Once everything is filled in, click "Create merge request". + +```{figure} ../images/gitlab-merge-request-description.png +:name: gitlab-merge-request-description + +Example of a filled-in merge request. +``` + +5. On the page of our merge request ({numref}`gitlab-finished-merge-request`), we're greeted with some more information. On top, you can read the description we just provided when creating the merge request. We can also see an "Approve" button, which you may or may not see, depending on your repository rights. Next, we can see a green checkmark, with a "Merge" button. This is good news: our changes do not conflict with existing files on the `main` branch, and we can merge our changes directly by clicking the button (if applicable: only after somebody else approved the request). If this isn't the case, you'll see a red cross instead of the green checkmark, and GitLab will tell that you've run into a *merge conflict*. You'll need to figure out which file(s) are causing this, and fix it. You can read more on this in the {ref}`section on merge conflicts `. + + +```{figure} ../images/gitlab-finished-merge-request.png +:name: gitlab-finished-merge-request + +The finished merge request. +``` \ No newline at end of file diff --git a/book/workflows/git-stage-commit-push.md b/book/workflows/git-stage-commit-push.md new file mode 100644 index 0000000..b15f130 --- /dev/null +++ b/book/workflows/git-stage-commit-push.md @@ -0,0 +1,44 @@ +(stage-commit-push)= +# Staging, committing and pushing + +Suppose we are writing a new chapter, or are updating an existing chapter for our Jupyter Book. We've created a new branch on which we are going to make the changes, and we've checked out this branch (make sure this is really the case by checking the name of the current branch in the bottom-left corner of your VS Code window!). + +## Staging +For demonstration purposes, we have created a new Markdown file called `new-chapter.md`, which we want to add to the book. Since this is a new file, Git has no knowledge about its existence yet. I.e., the file is *untracked*. To add new files to our repository, we need to *stage* and *commit* them. To do this, open the Source Control menu in the left side bar. Our new file should appear in this menu: + +```{figure} ../images/git-new-file.png + +The new file appears in the Source Control menu. +``` + +Notice that there is a green letter U placed besides the file, meaning that our file is untracked. To add a file to the *staging area*, click on the + that appears when you hover over the file. + +## Committing +Next, in the field that says "Message", write a short description of the changes you have made. This message is what is called the *Commit Message*. After we've done this, the menu should now look something like this: + +```{figure} ../images/git-file-added.png + +The new file has been staged, and we've written a Commit Message. +``` + +Now, we are ready to add the changes to the repository. This is done by *committing* the staged changes. To do this, simply press the big, blue "Commit" button in the Source Control menu. + +```{tip} +Unlike this example, you do not have to stage and commit every file individually. In a real case, a chapter may consist of multiple files (and images!), and it makes sense to add them in the same commit. +``` + +## Pushing the changes +Are we done now? No! Up to now, we have only made changes and committed to the *local* copy of the repository. We still need to *push* our changes to the *remote* repository (the one that is on the TU Delft GitLab), so that others can see our changes and interact with them. In the bottom left corner, next to the branch name, click the "Synchronize Changes" button (the one that looks like this: 🔄, highlighted in the image below) to push the changes to GitLab. + +```{figure} ../images/git-push.png + +After committing, click the 🔄 button to push our changes to the remote repository. +``` + +## Summary +In short, the Git workflow consists of the following steps: + +1. Staging new and modified files +2. Writing a commit message +3. Committing the changes +4. Pushing the new changes to GitLab \ No newline at end of file diff --git a/book/workflows/gitlab.md b/book/workflows/gitlab.md new file mode 100644 index 0000000..2a8bb8d --- /dev/null +++ b/book/workflows/gitlab.md @@ -0,0 +1,2 @@ +(gitlab)= +# Useful GitLab features \ No newline at end of file