Tweak memory-based heuristics to pick default number of jobs

[giordano]

Default number of jobs is determined at https://github.com/vchuravy/ParallelTestRunner.jl/blob/26ec17954570cc0269b6ade76d573e0200e76952/src/ParallelTestRunner.jl#L289-L294 and it's based on a combination of number of cores and available memory. However, GitHub-hosted Apple Silicon runners have only 3 cores and 7 GB of memory, resulting this heuristic to choose only one runner. We may want to tweak the heuristics, to allow at least 2 workers out-of-the-box on Apple Silicon (and hope nothing crashes badly).

Ref: EnzymeAD/Enzyme.jl#2662 (comment).

[giordano]

The estimated memory use probably also needs to become configurable, as well as some additional flexibility to pick nworkers (because e.g. with CUDA.jl we estimate based on the available GPU memory).

Originally posted by @maleadt in #5 (comment)

[maleadt]

On my macOS laptop, free memory is even lower:

julia> Sys.free_memory()
0x0000000004344000

julia> Base.format_bytes(ans)
"67.266 MiB"

[giordano]

Yeah, I noticed that also on my Macbook yesterday I was getting 1 job by default, but I could easily run 4 parallel jobs in EnzymeAD/Enzyme.jl#2662 selecting them manually with --jobs=4.

[christiangnrd]

I looked into this because I've always found Apple's memory reporting confusing. Turns out macOS reports free memory as the fully free pages, but "inactive pages" are also available for the OS to use and should probably be included as available for any process wondering "how much memory can I use without causing problems".

I opened libuv/libuv#4908 and hopefully it's simple enough to get merged quickly.

Maybe as a solution in the meantime we can implement the available memory on our own?

Potential solution adapted from the last time Julia people encountered this issue: JuliaLang/libuv#34 (comment)

Code

mutable struct VmStatistics64
	free_count::UInt32
	active_count::UInt32
	inactive_count::UInt32
	wire_count::UInt32
	zero_fill_count::UInt64
	reactivations::UInt64
	pageins::UInt64
	pageouts::UInt64
	faults::UInt64
	cow_faults::UInt64
	lookups::UInt64
	hits::UInt64
	purges::UInt64
	purgeable_count::UInt32
	
	speculative_count::UInt32

	decompressions::UInt64
	compressions::UInt64
	swapins::UInt64
	swapouts::UInt64
	compressor_page_count::UInt32
	throttled_count::UInt32
	external_page_count::UInt32
	internal_page_count::UInt32
	total_uncompressed_pages_in_compressor::UInt64

	VmStatistics64() = new(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
end


function available_memory()
	vms = Ref{VmStatistics64}(VmStatistics64())
	mach_host_self = @ccall mach_host_self()::UInt32
	count = UInt32(sizeof(VmStatistics64) ÷ sizeof(Int32))
	ref_count = Ref(count)
	@ccall host_statistics64(mach_host_self::UInt32, 4::Int64, pointer_from_objref(vms[])::Ptr{Int64}, ref_count::Ref{UInt32})::Int64

	page_size = Int(@ccall sysconf(29::UInt32)::UInt32)

	return (Int(vms[].free_count) + Int(vms[].inactive_count)) * page_size
end