Adisorn Owatsiriwong
Parallel programming in MATLAB enables us to perform multiple computations simultaneously by dividing tasks into subtasks that run on different cores or GPUs. This is particularly useful for speeding up time-consuming computations involving large datasets. MATLAB provides several tools for parallel computing, including parfor loops, distributed arrays, and parallel functions.
Fig. 1 shows the concept of traditional serial code. It consists of predefined instructions that must be executed in order. The total time to finish the loop in Fig. 1 is K*N/s where s is the speed of CPU core.
If we can distribute the execution to K workers (CPU cores or GPU), those serial instructions are executed in parallel and the total running time is N/s + Overhead, as we perform a single iteration in the serial code.
The parallelism works best when we subdivide the problem into sub-domains. We can assign each worker (CPU core or GPU) to each sub-domain. For the program like MATLAB, this can be done automatically via parfor loop. Look at this example
Example: Matrix multiplication using parfor
A = rand(1000);
B = rand(1000);
C = zeros(size(A));
parfor i = 1:1000
C(i,:) = A(i,:) * B;
end
In this example, parfor loops for each row of matrix and perform matrix A*B operation then store to C row by row. Each row of C is similar to sub-domain of A. From this example, it is intuitively to think that parfor is working like 1D array. The distribution to wrokers are done in one direction only (by row of A in this example). Nested-loop does not apply to parfor and won't benefit for parallelism due to overhead.
Considering the previous domain decomposition concept, although it looks like we are decomposing the analysis domain into 2D due to its perception, the way MATLAB working is shown in the following figure.
Example: Monte Carlo Simulation
N = 1e7; % Number of random points
inside = 0;
parfor i = 1:N
x = rand;
y = rand;
if x^2 + y^2 <= 1
inside = inside + 1;
end
end
pi_estimate = 4 * inside / N;
disp(['Estimated value of pi: ', num2str(pi_estimate)]);
In this example, parfor loops for N random points to check if the point is inside the unit circle and count. The inside variable is so-called reduction variable which is used to collect the out from each worker. Finally, the value of Pi can be computed.