Parallel Map
Here we compare five executions of comp
, a "heavy" computation taking 2 seconds:
using YAActL, .Threads, Distributed
function comp(i) # a "heavy" computation
sleep(2)
return (id=i, thrd=threadid())
end
We can check how long it takes sequentially:
julia> @time map(comp, 1:5) # sequential
10.024817 seconds (36 allocations: 1.234 KiB)
5-element Array{NamedTuple{(:id, :thrd),Tuple{Int64,Int64}},1}:
(id = 1, thrd = 1)
(id = 2, thrd = 1)
(id = 3, thrd = 1)
(id = 4, thrd = 1)
(id = 5, thrd = 1)
now with Julia's Threads
:
julia> @time begin
t = map(x->(Threads.@spawn comp(x)), 1:5)
map(fetch, t)
end
2.074831 seconds (150.55 k allocations: 8.045 MiB)
5-element Array{NamedTuple{(:id, :thrd),Tuple{Int64,Int64}},1}:
(id = 1, thrd = 2)
(id = 2, thrd = 3)
(id = 3, thrd = 5)
(id = 4, thrd = 4)
(id = 5, thrd = 6)
now with YAActL
(we get the results in the USR
channel):
julia> A = map(i->Actor(parallel(), comp, i), 1:5); # start actors
julia> @time begin
foreach(a->call!(a, USR), A)
map(x->receive!(USR).y, 1:5)
end
2.042264 seconds (83.46 k allocations: 4.448 MiB, 0.53% gc time)
5-element Array{NamedTuple{(:id, :thrd),Tuple{Int64,Int64}},1}:
(id = 2, thrd = 3)
(id = 5, thrd = 6)
(id = 4, thrd = 4)
(id = 3, thrd = 2)
(id = 1, thrd = 5)
now with Distributed
:
julia> addprocs(4); # add processes
julia> @everywhere function comp(i) # a "heavy" computation
sleep(2)
return (id=i, prc=myid())
end
julia> @time begin
f = map(i->(@spawnat i comp(i)), 1:5)
map(fetch, f)
end
2.252625 seconds (185.30 k allocations: 9.740 MiB, 0.38% gc time)
5-element Array{NamedTuple{(:id, :prc),Tuple{Int64,Int64}},1}:
(id = 1, prc = 1)
(id = 2, prc = 2)
(id = 3, prc = 3)
(id = 4, prc = 4)
(id = 5, prc = 5)
YAActL
actors and Distributed
processes are persistent objects. Therefore we must allocate them before we do computations with them. If we work with remote actors on other processes or nodes, we have to ensure as in Distributed
that the code is available on each node.