index<--

Asynchronous Processes

In Bash you can call multiple commands or command groups asynchronously. That is when main thread start a sub-process but continoue on and can start another sub-process without wayting for the first to finish. Asynchronous processes can run in parallel on different core processors.

Bash is good for starting commands in parallel. There are two methods available: running commands in background and coprocesses. With coprocesses you have more control but you need to create a script for managing the workflow.

Forking

This is a terminology used sometimes for describing the workflow. A large job is divided in smaller parts and distribute the work in multiple threads to be executed in parallel. Most frequently the threads are doing exactly the same thing for equal load. Sometimes the threads may have different load and duration.

Asynchronous Processing

In this case the processes are relative independent. All threads start almost at the same time, and finish at different times. The main thread must wait for all processes to finish before doing something with the results. During the execution, the sub-processes do not communicate with the main thread.

forking.sh

#!/bin/bash
#start 4 asynchronous threads
echo "starting ..."
echo "1" && sleep 20 && echo "1" &
echo "2" && sleep 5  && echo "2" &
echo "3" && sleep 10 && echo "3" &
echo "4" && sleep 3  && echo "4" &

sleep 1
echo "ending ..."

wait
echo "done."

console

~/bash-repl$ bash forking.sh
starting ...
1 
2 
3 
4 
ending ...
4
2
3
1
done.

Notes:

Ending a chain of commands with "&" cause all gorup to run asynchronously in the background;
Sub-processes start almost immediately and the commands in the chain will be executed one by one;
After starting all necessary co-processes you must use "wait" to avoid exiting the script too early;
After waiting for all processes to finish you can print the results and prepare for exit;

Coprocess

You can use command: coproc to start a secondary process that can communicate with the main process. This is similar with forking with a single branch. The point is, the coprocess is running in parallel witht he main process.

Main process & Coprocess

example

#!/bin/bash
coproc (echo $(whoami))
echo "COPROC = ${COPROC[@]}"
echo "COPROC_PID = ${COPROC_PID}"

# read the output from pipeline
read -r user <&"${COPROC[0]}"
echo "Current user: $user"

console

~/bash-repl$ bash coprocess.sh 
COPROC = 63 60
COPROC_PID = 205
Current user: runner
~/bash-repl$

Notes:

co-processes are asynchronous, similar to commands that use "&" at the end;
co-processes have an advantage over normal jobs by providing a data pipeline;
co-processes can be used in producer-consumer design pattern.

Read next: External Commands