Skip to content

Commit e7300e6

Browse files
parmentelatparmentelat
committed
add README and service file
1 parent 967f46d commit e7300e6

2 files changed

Lines changed: 238 additions & 0 deletions

File tree

‎.github/docker/README-setup-runner.md‎

Lines changed: 219 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,219 @@
1+
# self-hosted runner setup
2+
3+
## dockerhub
4+
5+
***WARNING*** need to cleanup the mess with 2 different names used here.
6+
we have
7+
8+
- python-node - to remove
9+
- python-nodejs - OK
10+
11+
### publish the image
12+
13+
```
14+
# as boxtops@root
15+
16+
cd
17+
# if needed
18+
git clone https://github.com/flotpython/jupyterlab-examples
19+
20+
cd ~/jupyterlab-examples/.github/docker
21+
22+
# will build & push the image to dockerhub.pl.sophia.inria.fr
23+
# the docker login as admin has been made once and for good as the root user there
24+
./rebuild.sh
25+
26+
# check available tags
27+
curl -u admin:$password https://dockerhub.pl.sophia.inria.fr/v2/_catalog
28+
curl -u admin:$password https://dockerhub.pl.sophia.inria.fr/v2/python-node/tags/list
29+
```
30+
31+
## general comments
32+
33+
### right in the host context
34+
35+
- the self-hosted runner is required to run as a regular user (not root), this
36+
is enforced by the GH runner code
37+
- as this needs a minimal amount of isolation between runs, we've tried using
38+
`conda` or `pixi` to provide that isolation; however, the bottom line is
39+
**it's not a good idea**
40+
- running right inside the host context is **doable**, and in a first stage
41+
we've used that for the `frontend` course on `bigjack`
42+
- however this is both **fragile** (not sure how several concurrent runs would
43+
behave) and also - very - **slow**
44+
45+
### in a container
46+
47+
- for that reason we have explored containerization options
48+
- ***BUT*** using **podman** turns out to be **overcomplex**, due to the need to
49+
run the runner as a non-root user - don't do that !
50+
- for that reason we have switched to using **regular docker**
51+
- btw: that is a bit unfortunate because we had `thermals` that could have done the
52+
job pretty well, but well, c'est la vie...
53+
- using `boxtops` then instead
54+
- also, it is important that the container itself must not run as root either,
55+
because `myst --execute` will then have its `jupyter-server` fail silently in
56+
that case (the logs would report that the `--allow-root` flag is missing)
57+
58+
### need to use a publicly available image
59+
60+
- the GH action that sets up the container insists on doing a `docker pull`
61+
first, there does not seem to be a way to skip that
62+
- hence the need to push the image to a publicly available docker registry
63+
- we have setup dockerhub on `dockerhub.pl.sophia.inria.fr` for that purpose
64+
it is requires login for publication but is configured to allow anonymous
65+
pulls
66+
67+
### the image
68+
69+
- this is expected to be run on boxtops
70+
does not really matter much, but it does require the `docker login` step
71+
- is published as `dockerhub.pl.sophia.inria.fr/python-nodejs:latest`
72+
- rebuild with
73+
```bash
74+
cd ~/jupyterlab-examples/.github/docker
75+
git pull
76+
./rebuild.sh
77+
```
78+
- this uses `Dockerfile.python-nodejs`
79+
- note that the user `github-actions` is created with uid/gid 1001 (see also
80+
below)
81+
82+
## setup steps on the GH side
83+
84+
### allow public repos to use self-hosted runners
85+
86+
- ***IMPORTANT*** - do it once for each orga !
87+
you also need to allow public repos to use self-hosted runners in the
88+
organization settings on GitHub
89+
90+
`github.com/$ORGA` -> Settings -> Actions -> Runner groups -> Default Group -> Allow public repositories
91+
92+
### how to check for current runners
93+
94+
useful later to see which runners are registered
95+
96+
```
97+
ORGA=flotpython
98+
gh api /orgs/$ORGA/actions/runners
99+
```
100+
101+
## setup on the runner host
102+
103+
***IMPORTANT***: make sure you run genuine docker and not podman
104+
105+
```bash
106+
dnf install -y docker-cli
107+
systemctl enable --now docker
108+
```
109+
110+
### runner cleanup steps
111+
112+
if needed: on the GH site, go to `github.com/$ORGA`
113+
> -> Settings -> Actions -> Runners
114+
115+
and remove the runners you don't need anymore
116+
117+
then on the host machine, as root:
118+
119+
```bash
120+
# spot running services
121+
systemctl | grep github-actions-runner
122+
123+
# stop them all - or be specific if only for one orga
124+
systemctl stop github-actions-runner@*
125+
126+
# disable them all - same
127+
systemctl disable github-actions-runner@*
128+
129+
# remove the service files - same
130+
rm /etc/systemd/system/github-actions-runner@*.service
131+
132+
# reload systemd
133+
systemctl daemon-reload
134+
135+
# remove the runner folders - if it's the last orga that had a runner on this box
136+
userdel -rf github-actions
137+
```
138+
139+
## runner setup steps
140+
141+
### once at least as root
142+
143+
it's important that `github-actions` user is created with **the same uid/gid** as the image !
144+
otherwise there will be permission issues and the container won't even start properly !
145+
146+
```bash
147+
148+
# create a user `github-actions` on the machine
149+
groupadd -g 1001 github-actions
150+
sudo useradd -u 1001 -g 1001 -d /containers/github-actions -s /bin/bash github-actions
151+
152+
# add it to the `docker` group
153+
usermod -aG docker github-actions
154+
```
155+
156+
### as github-actions - for one orga
157+
158+
```bash
159+
# login as `github-actions`
160+
su - github-actions
161+
162+
# create a folder for the orga runner
163+
ORGA=ue22-p25
164+
mkdir github-actions-runner-${ORGA}
165+
cd github-actions-runner-${ORGA}
166+
```
167+
168+
### create the runner
169+
170+
- go to github orga page
171+
-> Settings -> Actions -> Runners -> New self-hosted runner
172+
- select Linux / x64
173+
- Download:
174+
- copy-&-paste **but WITHOUT the creation of `actions-runner` folder**
175+
turns out in our setup this is replaced by `github-actions-runner-${ORGA}`
176+
- Configure:
177+
- you may add the 'docker' tag to the runner just in case
178+
179+
### back as root
180+
181+
- install the runner as a service
182+
183+
```bash
184+
# if needed
185+
cp ~/jupyterlab-examples/.github/scripts/github-actions-runner@.service /etc/systemd/system/github-actions-runner@.service
186+
187+
systemctl daemon-reload
188+
systemctl enable --now github-actions-runner@${ORGA}
189+
```
190+
191+
- double check the logs - esp. wrt to the extra folder level if you did create
192+
`actions-runner` folder
193+
in that case rename the folder as appropriate
194+
```bash
195+
journalctl -u github-actions-runner@${ORGA} -f
196+
```
197+
198+
## historical notes
199+
200+
### first successful build with frontend
201+
202+
- was done
203+
- with orga=ue22-p25
204+
- using host context (no container)
205+
- relies on conda for isolation
206+
- runner box is bigjack
207+
- takes in the 12 minutes
208+
- and would probably be fragile if several runs were to be launched concurrently
209+
210+
***NOTE*** on bigjack the service file is still hard-wired, it needs to be templatized like the one on bigjohn
211+
212+
### forget about pixi
213+
214+
too many issues with this idea of using pixi for isolation... just don't do that !
215+
the `pixi` branch in `flotpython/course` has some leftovers of that attempt
216+
217+
### remember to not try and run myst as root
218+
219+
one lesson though, it can't run as root (jupyter as spawned by myst would just fail)

‎.github/docker/github-actions-runner@.service‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
[Unit]
2+
Description=GitHub Actions Runner (%i)
3+
After=network.target
4+
5+
[Service]
6+
Type=simple
7+
User=github-actions
8+
# %i will be replaced by the name you provide (e.g., ue22-p25)
9+
WorkingDirectory=/containers/github-actions/github-actions-runner-%i
10+
ExecStart=/containers/github-actions/github-actions-runner-%i/run.sh
11+
Restart=always
12+
RestartSec=10
13+
14+
# Security hardening
15+
PrivateTmp=yes
16+
NoNewPrivileges=yes
17+
18+
[Install]
19+
WantedBy=multi-user.target

0 commit comments

Comments
 (0)