Concurrency, Parallelism and Related Terms

amrum-588135_1920 — Image credit-Pixabay

“Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once.”

Rob Pike
Co-author of Go and Limbo programming languages

Concurrency and Parallelism are two words that we can find and often get confused, when learning programming threads. These two may look same, but they are not.

These two concepts will form several other buzzwords as well. Some of them are mentioned below. I’ll describe each of them.

Concurrency
Parallelism
Process
Multitasking
Multiprocessing
Multiprogramming
Thread

Please note that I have obtained the definitions for this article by referring the websites mentioned at the end. You may visit them if additional information is required.

Concurrency

Concurrency is a property of a certain computational problem that has the ability to be decomposed in to parts. Then, we can solve each of these parts separately and the combination of the results will ultimately lead to the final solution.

How can we solve these parts? Suppose you decompose a problem in to three parts, A, B and C. You can start solving part A. What would you do thereafter?

1. Continue solving part A, finish solving it and then start part B. Thereafter C.

2. Pause A and start solving C. Later on, you pause C and start B. After that, continue A again by pausing B…..Likewise, you can change your attention between A and B and finally solve both of them.

3. Find a friend and give the part B to him. Both of you are now engaging with solving two parts. Meanwhile, you can pause solving A and engage with C as well.

Sequential Computing, Concurrent Computing and Parallel Computing

The 1st option is analogous to sequential computing, where you finish one part completely and start the next.
2nd and 3rd options are analogous to Concurrent computing, where you don’t need to finish the current computation to start another one.

3rd one is a special case of Concurrent computing. Here, you get the help of another brain to solve the part B of the problem. This is definitely faster than the previous 2 implementations. The method is analogous to Parallel computing.

Now, let’s see the definitions of sequential, concurrent and parallel computing.

Sequential computing

A consecutive and ordered execution of processes one after another.

Concurrent computing

Execution of several processes in overlapping time periods.

Parallel computing

Execution of more than one process at the same physical instant.

Process

I have used the word process frequently in above lines. Specially in concurrent computing definition, “execution of several processes in overlapping time periods”. So, what is a process?

A process is a similar word for the ‘task’ in some operating systems. It is an executing instance of a computer program.

A computer program is nothing but a collection of various instructions. Actual execution of these instructions is a process.

Suppose you are browsing Internet using Firefox. . With the Firefox e10s architecture, several processes are generated. One process handles the Firefox User Interface and the other processes handles the web-content.

Single-core vs Multi-core

Processors in computers can be divided in to two categories.

Single-core processors – Processors having only one Central Processing Unit (CPU).
Eg : intel pentium 4
AMD Athlon 64 FX-57
Multi-core processors – Processors with more than one core. That is many CPUs on a single chip.
Eg : intel Core i7 – 4/6/8 cores
AMD RYZEN 7 1700X – 8 cores

When you have more than one core, you can assign each core a process, so that ultimately, there would be several processes running at the same moment.

This means, multi-core processors are necessary for parallel computing. Remember, this is similar to the 3rd option in our previous example where we used two brains to solve math problem parts.

But in a single core processor, we can perform only one process at once. So, parallel computing is impossible in such processors.

Single-core processors can work similar to the ways of solving problems in 1st and 2nd options. It can do sequential programming and concurrent programming. But not parallel programming.

Processes are often run by sharing the CPU resources, specially in a single-core processor. That is, for example, if there are 10 apps running right now, there would be 10 individual processes that need to be maintained.
What a single-core processor would do is concurrent programming. Jump from one to another task very quickly so that we would feel all of them are running at the same time. It doesn’t actually need to finish one task completely. Pause it and jump. Come back again. This operation is very fast so we don’t see a difference.

Parallel programming could be done on multi-core processor so that each core is assigned fewer number of tasks. This would improve the performance. You would be able to open more and more processes. But if you do the same number of processes on a single-core, you would feel the computer is running slower.

Multitasking, Multiprocessing and Multiprogramming

Multitasking

It is the ability of computers to run several programs simultaneously or to make the user feel they run at the same time. Note that it is not compulsory to have simultaneous execution of these processes.

You can listen to music while opening a word document. This is multitasking and two processes are generated. But the two processes need not to run at the same physical time moment.

Remember, We could do this in our older single-core PCs existed before the introduction of multi-core processors.

That, is multitasking is possible from both single-core and multi-core processors.

Multiprocessing

Running two or more processes simultaneously in a computer with a multi-core processor.

Multiprogramming

Loading of one or more programs to main memory. After loading, the processes in the programs are executed with the available CPU resources.

Thread

We have discussed that a process is an execution of the code in a program. One process can be paused and another process can be started. Two processes can run simultaneously in a PC with a multi-core processor. But we didn’t discuss what’s inside a process.

A process can be made up of multiple components. Each of them is an individual code part called a Thread.

I have mentioned that when we open a Firefox window, several new processes are generated to handle it. Inside each one of these processes, there would be several threads.. Suppose you open a website with gifs. Several threads may get involved in loading a gif image. These threads belong to, hopefully, the GUI process. Likewise there could be many threads within a single process.

Two threads share the resources allocated for the parent process. For example, if one process allocates one core of a multi-core processor, threads in the process share this processor to execute time to time.
That is concurrent computing of threads. Two threads in a process can also execute at the same time in a multi-core processor, specially if a process shares two cores. At that time, we have parallel computing. Finally, we have another word, multi-threading.