# Changeset 9c5aef9

Ignore:
Timestamp:
Mar 3, 2022, 1:37:17 PM (13 months ago)
Branches:
Children:
bf8b77e
Parents:
05ffb7b
Message:

Added heap structure and design choices to allocator chapter

Location:
doc/theses/mubeen_zulfiqar_MMath
Files:
2 edited

### Legend:

Unmodified
 r05ffb7b \chapter{Allocator} \noindent ==================== Writing Points: \begin{itemize} \item Objective of uHeapLmmm. \item Design philosophy. \item Background and previous design of uHeapLmmm. \item Distributed design of uHeapLmmm. ----- SHOULD WE GIVE IMPLEMENTATION DETAILS HERE? ----- \PAB{Maybe. There might be an Implementation chapter.} \item figure. \item Advantages of distributed design. \end{itemize} The new features added to uHeapLmmm (incl. @malloc_size@ routine) \CFA alloc interface with examples. \begin{itemize} \item Why did we need it? \item The added benefits. \end{itemize} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% uHeapLmmm Design %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Objective of uHeapLmmm} UHeapLmmm is a lightweight memory allocator. The objective behind uHeapLmmm is to design a minimal concurrent memory allocator that has new features and also fulfills GNU C Library requirements (FIX ME: cite requirements). \subsection{Design philosophy} The objective of uHeapLmmm's new design was to fulfill following requirements: \begin{itemize} \item It should be concurrent to be used in multi-threaded programs. \section{uHeap} uHeap is a lightweight memory allocator. The objective behind uHeap is to design a minimal concurrent memory allocator that has new features and also fulfills GNU C Library requirements (FIX ME: cite requirements). The objective of uHeap's new design was to fulfill following requirements: \begin{itemize} \item It should be concurrent and thread-safe for multi-threaded programs. \item It should avoid global locks, on resources shared across all threads, as much as possible. \item It's performance (FIX ME: cite performance benchmarks) should be comparable to the commonly used allocators (FIX ME: cite common allocators). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Background and previous design of uHeapLmmm} uHeapLmmm was originally designed by X in X (FIX ME: add original author after confirming with Peter). (FIX ME: make and add figure of previous design with description) %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Distributed design of uHeapLmmm} uHeapLmmm's design was reviewed and changed to fulfill new requirements (FIX ME: cite allocator philosophy). For this purpose, following two designs of uHeapLmm were proposed: \paragraph{Design 1: Decentralized} \section{Design choices for uHeap} uHeap's design was reviewed and changed to fulfill new requirements (FIX ME: cite allocator philosophy). For this purpose, following two designs of uHeapLmm were proposed: \paragraph{Design 1: Centralized} One heap, but lower bucket sizes are N-shared across KTs. This design leverages the fact that 95\% of allocation requests are less than 512 bytes and there are only 3--5 different request sizes. When KTs $\le$ N, the important bucket sizes are uncontented. When KTs $>$ N, the free buckets are contented. Therefore, threads are only contending for a small number of buckets, which are distributed among them to reduce contention. \begin{cquote} \centering \input{AllocDS2} \end{cquote} Problems: need to know when a kernel thread (KT) is created and destroyed to know when to assign a shared bucket-number. When no thread is assigned a bucket number, its free storage is unavailable. All KTs will be contended for one lock on sbrk for their initial allocations (before free-lists gets populated). \paragraph{Design 2: Decentralized N Heaps} Fixed number of heaps: shard the heap into N heaps each with a bump-area allocated from the @sbrk@ area. Kernel threads (KT) are assigned to the N heaps. Problems: need to know when a KT is created and destroyed to know when to assign/un-assign a heap to the KT. \paragraph{Design 2: Centralized} One heap, but lower bucket sizes are N-shared across KTs. This design leverages the fact that 95\% of allocation requests are less than 512 bytes and there are only 3--5 different request sizes. When KTs $\le$ N, the important bucket sizes are uncontented. When KTs $>$ N, the free buckets are contented. Therefore, threads are only contending for a small number of buckets, which are distributed among them to reduce contention. \begin{cquote} \paragraph{Design 3: Decentralized Per-thread Heaps} Design 3 is similar to design 2 but instead of having an M:N model, it uses a 1:1 model. So, instead of having N heaos and sharing them among M KTs, Design 3 has one heap for each KT. Dynamic number of heaps: create a thread-local heap for each kernel thread (KT) with a bump-area allocated from the @sbrk@ area. Each KT will have its own exclusive thread-local heap. Heap will be uncontended between KTs regardless how many KTs have been created. Operations on @sbrk@ area will still be protected by locks. %\begin{cquote} %\centering %\input{AllocDS3} FIXME add figs %\end{cquote} Problems: We cannot destroy the heap when a KT exits because our dynamic objects have ownership and they are returned to the heap that created them when the program frees a dynamic object. All dynamic objects point back to their owner heap. If a thread A creates an object O, passes it to another thread B, and A itself exits. When B will free object O, O should return to A's heap so A's heap should be preserved for the lifetime of the whole program as their might be objects in-use of other threads that were allocated by A. Also, we need to know when a KT is created and destroyed to know when to create/destroy a heap for the KT. \paragraph{Design 4: Decentralized Per-CPU Heaps} Design 4 is similar to Design 3 but instead of having a heap for each thread, it creates a heap for each CPU. Fixed number of heaps for a machine: create a heap for each CPU with a bump-area allocated from the @sbrk@ area. Each CPU will have its own CPU-local heap. When the program does a dynamic memory operation, it will be entertained by the heap of the CPU where the process is currently running on. Each CPU will have its own exclusive heap. Just like Design 3(FIXME cite), heap will be uncontended between KTs regardless how many KTs have been created. Operations on @sbrk@ area will still be protected by locks. To deal with preemtion during a dynamic memory operation, librseq(FIXME cite) will be used to make sure that the whole dynamic memory operation completes on one CPU. librseq's restartable sequences can make it possible to re-run a critical section and undo the current writes if a preemption happened during the critical section's execution. %\begin{cquote} %\centering %\input{AllocDS4} FIXME add figs %\end{cquote} Problems: This approach was slower than the per-thread model. Also, librseq does not provide such restartable sequences to detect preemtions in user-level threading system which is important to us as CFA(FIXME cite) has its own threading system that we want to support. Out of the four designs, Design 3 was chosen because of the following reasons. \begin{itemize} \item Decentralized designes are better in general as compared to centralized design because their concurrency is better across all bucket-sizes as design 1 shards a few buckets of selected sizes while other designs shards all the buckets. Decentralized designes shard the whole heap which has all the buckets with the addition of sharding sbrk area. So Design 1 was eliminated. \item Design 2 was eliminated because it has a possibility of contention in-case of KT > N while Design 3 and 4 have no contention in any scenerio. \item Design 4 was eliminated because it was slower than Design 3 and it provided no way to achieve user-threading safety using librseq. We had to use CFA interruption handling to achive user-threading safety which has some cost to it. Desing 4 was already slower than Design 3, adding cost of interruption handling on top of that would have made it even slower. \end{itemize} \subsection{Advantages of distributed design} The distributed design of uHeap is concurrent to work in multi-threaded applications. Some key benefits of the distributed design of uHeap are as follows: \begin{itemize} \item The bump allocation is concurrent as memory taken from sbrk is sharded across all heaps as bump allocation reserve. The call to sbrk will be protected using locks but bump allocation (on memory taken from sbrk) will not be contended once the sbrk call has returned. \item Low or almost no contention on heap resources. \item It is possible to use sharing and stealing techniques to share/find unused storage, when a free list is unused or empty. \item Distributed design avoids unnecassry locks on resources shared across all KTs. \end{itemize} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{uHeap Structure} As described in (FIXME cite 2.4) uHeap uses following features of multi-threaded memory allocators. \begin{itemize} \item uHeap has multiple heaps without a global heap and uses 1:1 model. (FIXME cite 2.5 1:1 model) \item uHeap uses object ownership. (FIXME cite 2.5.2) \item uHeap does not use object containers (FIXME cite 2.6) or any coalescing technique. Instead each dynamic object allocated by uHeap has a header than contains bookkeeping information. \item Each thread-local heap in uHeap has its own allocation buffer that is taken from the system using sbrk() call. (FIXME cite 2.7) \item Unless a heap is freeing an object that is owned by another thread's heap or heap is using sbrk() system call, uHeap is mostly lock-free which eliminates most of the contention on shared resources. (FIXME cite 2.8) \end{itemize} As uHeap uses a heap per-thread model to reduce contention on heap resources, we manage a list of heaps (heap-list) that can be used by threads. The list is empty at the start of the program. When a kernel thread (KT) is created, we check if heap-list is empty. If no then a heap is removed from the heap-list and is given to this new KT to use exclusively. If yes then a new heap object is created in dynamic memory and is given to this new KT to use exclusively. When a KT exits, its heap is not destroyed but instead its heap is put on the heap-list and is ready to be reused by new KTs. This reduces the memory footprint as the objects on free-lists of a KT that has exited can be reused by a new KT. Also, we preserve all the heaps that were created during the lifetime of the program till the end of the program. uHeap uses object ownership where an object is freed to the free-buckets of the heap that allocated it. Even after a KT A has exited, its heap has to be preserved as there might be objects in-use of other threads that were initially allocated by A and the passed to other threads. \begin{figure} \centering \input{AllocDS2} \end{cquote} Problems: need to know when a kernel thread (KT) is created and destroyed to know when to assign a shared bucket-number. When no thread is assigned a bucket number, its free storage is unavailable. All KTs will be contended for one lock on sbrk for their initial allocations (before free-lists gets populated). Out of the two designs, Design 1 was chosen because it's concurrency is better across all bucket-sizes as design-2 shards a few buckets of selected sizes while design-1 shards all the buckets. Design-2 shards the whole heap which has all the buckets with the addition of sharding sbrk area. \subsection{Advantages of distributed design} The distributed design of uHeapLmmm is concurrent to work in multi-threaded applications. Some key benefits of the distributed design of uHeapLmmm are as follows: \begin{itemize} \item The bump allocation is concurrent as memory taken from sbrk is sharded across all heaps as bump allocation reserve. The lock on bump allocation (on memory taken from sbrk) will only be contended if KTs $<$ N. The contention on sbrk area is less likely as it will only happen in the case if heaps assigned to two KTs get short of bump allocation reserve simultanously. \item N heaps are created at the start of the program and destroyed at the end of program. When a KT is created, we only assign it to one of the heaps. When a KT is destroyed, we only dissociate it from the assigned heap but we do not destroy that heap. That heap will go back to our pool-of-heaps, ready to be used by some new KT. And if that heap was shared among multiple KTs (like the case of KTs $<$ N) then, on deletion of one KT, that heap will be still in-use of the other KTs. This will prevent creation and deletion of heaps during run-time as heaps are re-usable which helps in keeping low-memory footprint. \item It is possible to use sharing and stealing techniques to share/find unused storage, when a free list is unused or empty. \item Distributed design avoids unnecassry locks on resources shared across all KTs. \end{itemize} FIX ME: Cite performance comparison of the two heap designs if required \includegraphics[width=0.65\textwidth]{figures/NewHeapStructure.eps} \caption{HeapStructure} \label{fig:heapStructureFig} \end{figure} Each heap uses seggregated free-buckets that have free objects of a specific size. Each free-bucket of a specific size has following 2 lists in it: \begin{itemize} \item Free list is used when a thread is freeing an object that is owned by its own heap so free list does not use any locks/atomic-operations as it is only used by the owner KT. \item Away list is used when a thread A is freeing an object that is owned by another KT B's heap. This object should be freed to the owner heap (B's heap) so A will place the object on the away list of B. Away list is lock protected as it is shared by all other threads. \end{itemize} When a dynamic object of a size S is requested. The thread-local heap will check if S is greater than or equal to the mmap threshhold. Any request larger than the mmap threshhold is fulfilled by allocating an mmap area of that size and such requests are not allocated on sbrk area. The value of this threshhold can be changed using mallopt routine but the new value should not be larger than our biggest free-bucket size. Algorithm~\ref{alg:heapObjectAlloc} briefly shows how an allocation request is fulfilled. \begin{algorithm} \caption{Dynamic object allocation of size S}\label{alg:heapObjectAlloc} \begin{algorithmic}[1] \State $\textit{O} \gets \text{NULL}$ \If {$S < \textit{mmap-threshhold}$} \State $\textit{B} \gets (\text{smallest free-bucket} \geq S)$ \If {$\textit{B's free-list is empty}$} \If {$\textit{B's away-list is empty}$} \If {$\textit{heap's allocation buffer} < S$} \State $\text{get allocation buffer using system call sbrk()}$ \EndIf \State $\textit{O} \gets \text{bump allocate an object of size S from allocation buffer}$ \Else \State $\textit{merge B's away-list into free-list}$ \State $\textit{O} \gets \text{pop an object from B's free-list}$ \EndIf \Else \State $\textit{O} \gets \text{pop an object from B's free-list}$ \EndIf \State $\textit{O's owner} \gets \text{B}$ \Else \State $\textit{O} \gets \text{allocate dynamic memory using system call mmap with size S}$ \EndIf \State $\Return \textit{ O}$ \end{algorithmic} \end{algorithm} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Added Features and Methods} To improve the UHeapLmmm allocator (FIX ME: cite uHeapLmmm) interface and make it more user friendly, we added a few more routines to the C allocator. Also, we built a \CFA (FIX ME: cite cforall) interface on top of C interface to increase the usability of the allocator. To improve the uHeap allocator (FIX ME: cite uHeap) interface and make it more user friendly, we added a few more routines to the C allocator. Also, we built a \CFA (FIX ME: cite cforall) interface on top of C interface to increase the usability of the allocator. \subsection{C Interface} @addr@: the address of the currently allocated dynamic object. \end{itemize} @malloc_alignment@ returns the alignment of the given dynamic object. On failure, it return the value of default alignment of the uHeapLmmm allocator. @malloc_alignment@ returns the alignment of the given dynamic object. On failure, it return the value of default alignment of the uHeap allocator. \subsection{\lstinline{bool malloc_zero_fill( void * addr )}} \subsection{\CFA Malloc Interface} We added some routines to the malloc interface of \CFA. These routines can only be used in \CFA and not in our standalone uHeapLmmm allocator as these routines use some features that are only provided by \CFA and not by C. It makes the allocator even more usable to the programmers. We added some routines to the malloc interface of \CFA. These routines can only be used in \CFA and not in our standalone uHeap allocator as these routines use some features that are only provided by \CFA and not by C. It makes the allocator even more usable to the programmers. \CFA provides the liberty to know the returned type of a call to the allocator. So, mainly in these added routines, we removed the object size parameter from the routine as allocator can calculate the size of the object from the returned type. \subsection{Alloc Interface} In addition to improve allocator interface both for \CFA and our standalone allocator uHeapLmmm in C. We also added a new alloc interface in \CFA that increases usability of dynamic memory allocation. In addition to improve allocator interface both for \CFA and our standalone allocator uHeap in C. We also added a new alloc interface in \CFA that increases usability of dynamic memory allocation. This interface helps programmers in three major ways.