Process vs Thread — [Notes]

· Process
∘ Interprocess communication
· Threads
∘ Thread Anatomy
∘ Contention
· Thread Pools
· Process vs Thread
· Reference

Process

• has separate memory space (a process not allowed to access other process’s memory space but using interprocess communication)
• Depending on the technology, your server may consist of one or more processes.
  → Java server — will see only one process.
  → NodeJS server — may see as many processes as you have cores.

• let’s say you have 10,000 users at the moment in your system and you want to cache them in the memory of your server. If you try to do something like that with technology that uses more than one process, you will consume a lot more memory than with the technology that uses only one process.

Interprocess communication

• File (also memory mapped file)

Let’s say we have those 4 NodeJS processes from before, and they decide that they don’t want to replicate all the user data all the time. Instead, they agreed that they will store all the data in a file called `users.tmp`, and whenever they need a user, they will open this file and look for it.

The solution is very simple to understand, but hard to get right.

• File may not exist, and then it’s hard to decide what to do. Will it be created later or not?
• if you have to update it, you need to make sure only one process does it at the same time.
• compared to storing data in memory, accessing a file is much, much slower. (solution: memory mapped file — It’s basically a portion of your RAM, that process thinks is a file.)

• Signal

→ A signal is just a number.
→ Some are predefined, but you can use numbers that are not taken to communicate arbitrary messages.
→ The downside of this approach is that you can send only one number, so it’s pretty limited.

receiving interrupt with signal to terminate

• Socket (also Unix Domain Socket)

→ two types of sockets, network sockets and Unix domain sockets.
→ Network sockets are for remote calls like your browser accessing your server. So although they are on the same machine, your browser doesn’t know that, and it will be a remote call.
→ Unix domain sockets — not limited to Unix only, but they are limited to the same machine, much faster than network sockets, but you cannot reach remote machines using those sockets.

• Pipe

• Shared Memory
• Message passing

Threads

• A thread is what executes the code.
• Every process will create at least one thread, but usually there will be more.

• JavaScript runtime, for example, is said to be single threaded, but it actually means that the developer is able to utilize a single thread per process. But there are still other threads for garbage collection and networking, for example.
• A thread is more lightweight than a process, and different threads can run on separate CPUs in parallel.

Thread Anatomy

• Every thread has a stack.
• Stack stores the local variables, as well as method parameters and the call chain. (If you ever wrote a recursive method that never terminates, you probably got StackOverflowException or something like “stack is too deep”. All those errors refer to the thread stack.)
• Every process has all its memory in something called a heap. If you ever introduced a memory leak in your program or created too many objects, that’s where the “Out of Memory” types of exceptions come from.
• The heap is shared between all threads, and this is what causes a lot of multithreaded problems.
• Threads overhead: let’s say you are asked to design a server that handles user requests and you need to handle 10K requests per second. You may be tempted to say that each time a user connects, you will create a new thread. This is called thread-per-request model.” The problem with that approach is that creating a thread has its costs.
  → First of all, it’s relatively heavy operation. Since your process needs to ask operating system to create a thread.
  → Then there may be a limit to how many threads you can have set by an operating system. If you reach that limit, you cannot create any more threads.
  → Then there is a memory limit. In JVM, for example, thread is allocated one megabyte of memory by default at start. So even before we speak about contention, you need to remember that you simply cannot create an infinite number of threads. Their numbers should be quite limited.

Contention

• Different threads compete among themselves for shared resources in the same process.
• Locks: Two threads can write different values to the same memory location at the same time. This is called a race, and it creates inconsistency in your system. You want to avoid those situations, so you use the locks. That means that your threads will now compete to get the lock. More threads, more competition.
• CPU Time: If you have 4 CPUs and 16 threads, they will compete. Who gets to run?
• Shared resources: a file, will create contention between threads.

Thread Pools

we mentioned that threads are great to run code in parallel, but you cannot create too many of them. How do you limit the number of threads? — Thread Pools

• It has a limit of how many threads it can create.
• If you didn’t reach the limit yet, it will create a new thread for you.
• But if the maximum number of threads was already created, you will have to wait until one of them is available again.
• Thread pools are a great example where we sacrifice availability, because someone had to wait, to make the system more stable.

Process vs Thread

• Process is an executable program/task for an OS (Operating System): physical resources, as well as memory, are allocated in separation to each Process operating concurrently.
• A Process can spawn many Threads that share the memory space, as well as the resources.
• Threads can easily communicate each other using the parent Process shared memory: potentially, Threads can operate concurrently on contended resources into the parent’s shared area.
• On the other hand, Processes can communicate each other using OS’s Sockets, Pipes and Shared Memories (allocated on the purpose, specific area of mapped memory that can be shared among Processes).
• Each Process’ Thread has a dedicated Stack, but shares any open resource (e.g. file descriptors, sockets, etc.) with the parent Process, and so potentially with other Threads.

• A Thread can be seen as a lightweight Process: as said memory and resources are shared with the parent Process, the only dedicated resource is the Stack.
• As intuitive, a Thread context switch is much lighter and quick than context switches between Processes (only a tiny Stack plus a few registers need to be saved).
• What really is a Thread/Process? Exemplifying, a Thread and/or Process can be seen as a well defined set of machine codes/operations executable on a CPU, created by a compilation process and alternated by conditional jumps which are determined only at run-time.
• And, what really is a Context Switch? Again, exemplifying, a Context Switch can be seen as a well defined set of machine codes/operations executable on a CPU, aiming at saving the current execution context before to release the CPU (and its registries) to a new Process or Thread.
• NOTE Hardware Thread is a different concept.
  → Normally, multiplexing techniques allow at hardware level to improve the single core utilization creating different execution flows.
  → Superscalar architectures are able to easily multiplex two separate and parallel execution paths onto a single core (the OS sees two separate cores) [6].
  → In the picture below and example of Hyper-Threading that improves the multi-tasking capability of a single physical core is provided: pipeline bubbles are reordered at Front End in order to increase as much as possible each core usage.

Reference

Scalable I/O: Events- Vs Multithreading-based