Farsight: Deep-Learning-Driven Prefetching for Far Memory

Farsight is a neural network based kernel level far memory prefetcher, it utilizes the far memory IO time to perform model computation and make prefetch decision. check out our blog and arxiv for more details

Required Packages and Configurations

No need to install OFED, we use RDMA drivers from kernel sources

kernel build packages

git build-essential kernel-package fakeroot libncurses5-dev libssl-dev libelf-dev ccache bison flex libpci-dev dwarves

ebpf build packages

libz-dev libelf-dev binutils-dev libreadline-dev clang llvm

model related packages

cmake python3-venv python3-pybind11

dynamorio

cmake g++ g++-multilib doxygen git zlib1g-dev libunwind-dev libsnappy-dev liblz4-dev

grub config

systemd.unified_cgroup_hierarchy=1 intel_pstate=passive transparent_hugepage=madvise

systemd.unified_cgroup_hierarchy=1  -> enable cgroup v2, which allow memory.high swap out
intel_pstate=passive                -> enable intel_cpufreq cpu frequency driver for custom cpu frequency

Trace a program and Train a model

1. run tracer with ./tracer/run.sh <output directory> <command> [args..]
2. parse the trace ./tracer/parse.sh <trace binary file> <output file>
3. build the dataset ./model_py/build_dataset.sh <target directory> <dict_size, e.g. 64> <train_file> <val_file>
4. train the model python3 ./model_py/train.py <dataset directory> --batch_size=#
5. model will be ready under ./model_py/model_<name>_layer#.pt

Farsight compile procedure and start

1. ./linux-5.14/build_kernel.sh build
2. ./linux-5.14/build_kernel.sh install
3. check if grub has intel_pstate=passive transparent_hugepage=madvise intel_iommu=off systemd.unified_cgroup_hierarchy=1
4. if updated, run sudo update-grub
5. reboot
6. go to ./client/ and make
7. on the server: sudo ./rdma_startup.sh server_start
8. sudo ./rdma_startup.sh client_start (server side is using huge page otherwise connection may timeout, if connection failed, may need to restart server to reset fragmentation)

applications run procedure

1. sudo python3 set_cpu_frequency.py 2100
2. echo 3 | sudo tee /proc/sys/vm/drop_caches to drop all page cache
3. sudo python3 app_launcher.py -c <thread count> --model <model path> -- /usr/bin/time -v python3 <app command>
4. use cat /sys/fs/cgroup/farsight/memory.stat to check some stats
5. sudo rmdir /sys/fs/cgroup/farsight/ before running another experiment