source: doc/theses/lynn_tran_SE499/Chapters/Extensions.tex @ 432bffe

ADTast-experimentalenumforall-pointer-decaypthread-emulationqualifiedEnum
Last change on this file since 432bffe was 1b34b87, checked in by Peter A. Buhr <pabuhr@…>, 6 years ago

Lynn's GDB essay

  • Property mode set to 100644
File size: 19.5 KB
Line 
1\chapter{Extending GDB for \uC}
2
3\section{Introduction}
4A sequential program has a single call stack. A debugger knows about this call stack and provides
5commands to walk up/down the call frames to examine the values of local variables, as well as global
6variables. A concurrent program has multiple call stacks (for coroutines/tasks), so a debugger must
7be extended to locate these stacks for examination, similar to a sequential stack.  For example,
8when a concurrent program deadlocks, looking at the task's call stack can locate the resource and
9the blocking cycle that resulted in the deadlock. Hence, it is very useful to display the call stack
10of each task to know where it is executing and what values it is currently computing. Because each
11programming language's concurrency is different, GDB has to be specifically extended for \uC.
12
13\section{Design Constraints}
14As mentioned in Chapter \ref{GDB}, there are several ways to extend GDB. However, there are a few
15design constraints on the selected mechanism. All the functions implemented should maintain similar
16functionality to existing GDB commands. In addition to functional requirements, usability and
17flexibility are requirements for this project. These final requirements enable developers to be
18productive quickly and do more with the extensions. The extensions created for \uC are simple to use
19and versatile.
20
21The following new GDB command are all implemented through the Python API for GDB.  Python is a
22scripting language with built-in data structures and functions that enables the development of more
23complex operations and saves time on development.
24
25\section{\uC source-code example}
26Listing \ref{uC-src-code} shows a \uC program that implicitly creates two clusters, system and user,
27which implicitly have a processor (kernel thread) and processor task. The program explicitly creates
28three additional processors and ten tasks on the user cluster.
29
30\begin{figure}
31\begin{lstlisting}[numbers=left, xleftmargin=4.0ex, style=C++, caption={\uC source code used for GDB commands},label={uC-src-code}, basicstyle=\small]
32_Task T {
33    const int tid;
34    std::string name;
35
36    void f(int param) {
37        if ( param != 0 ) f( param - 1 );       // recursion
38        for ( volatile size_t i = 0; i < 100000000; i += 1 ); // delay
39        int x = 3;
40        std::string y = "example";
41    }                                           // breakpoint
42    void main() {
43        if ( tid != 0 )                         // T0 goes first
44            for ( volatile size_t i = 0; i < 1000000000; i += 1 ) // delay
45                if ( i % 10000000 == 0 ) yield(); // execute other tasks
46        f(3);
47    }
48  public:
49    T(const int tid) : tid( tid ) {
50        name = "T" + std::to_string(tid);
51        setName(name.c_str());
52    }
53};
54int main() {
55    uProcessor procs[3];                        // extra processors
56    const int numTasks = 10;
57    T * tasks[numTasks];                        // extra tasks
58    // allocate tasks with different names
59    for (int id = 0; id < numTasks; id += 1) {
60        tasks[id] = new T(id);
61    }
62    // deallocate tasks
63    for (int id = 0; id < numTasks; id += 1) {
64        delete tasks[id];
65    }
66}
67\end{lstlisting}
68\end{figure}
69
70\section{Existing User-defined GDB Commands}
71Listing \ref{uC-callstack} shows the GDB output at the base case of the recursion for one of the
72tasks created in the \uC program in listing \ref{uC-src-code}.  The task is stopped at line 10. The
73backtrace shows the three calls to function \verb|f|, started in the task's \verb|main|. The top two
74frames (5 and 6) are administrative frames from \uC. The values of the argument and local variables
75are printed.
76\begin{lstlisting}[caption={Call stack of function \texttt{a} in the \uC
77program from listing \ref{uC-src-code}}, label={uC-callstack}, basicstyle=\small\tt]
78(gdb) backtrace
79#0  T::f (this=0xa4f950, param=0) at test.cc:10
80#1  0x000000000041e509 in T::f (this=0xa4f950, param=1) at test.cc:6
81#2  0x000000000041e509 in T::f (this=0xa4f950, param=2) at test.cc:6
82#3  0x000000000041e509 in T::f (this=0xa4f950, param=3) at test.cc:6
83#4  0x000000000041e654 in T::main (this=0xa4f950) at test.cc:15
84#5  0x0000000000428de2 in ...::invokeTask (This=...) at ...
85#6  0x0000000000000000 in ?? ()
86(gdb) info args
87this = 0xa4f950
88param = 0
89(gdb) info locals
90x = 3
91y = "example"
92\end{lstlisting}
93
94\subsection{Listing all clusters in a \uC program}
95Listing \ref{clusters-command} shows the new command \verb|clusters| to list all program clusters
96along with their associated address.  The output shows the two \uC implicitly created clusters.
97\begin{lstlisting}[caption={clusters command}, label={clusters-command}, basicstyle=\small\tt]
98(gdb) clusters
99                Name           Address
100       systemCluster          0x65a300
101         userCluster          0x7ca300
102\end{lstlisting}
103
104\subsection{Listing all processors in a cluster}
105Listing \ref{cluster-procs} shows the new command \verb|processors|, which requires a cluster
106argument to show all the processors in that cluster. In particular, this example shows that there
107are four processors in the \verb|userCluster|, with their associated address, PID, preemption and
108spin.
109\begin{lstlisting}[caption={processors command}, label={cluster-procs}, basicstyle=\small\tt]
110(gdb) processors
111           Address                 PID          Preemption                Spin
112          0x7ccc30             8421504                  10                1000
113          0x8c9b50             9478272                  10                1000
114          0x8c9d10            10002560                  10                1000
115          0x8c9ed0            10530944                  10                1000
116\end{lstlisting}
117
118\subsection{Listing all tasks in all clusters}
119Listing \ref{tasks} shows the new command \verb|task| with the \verb|all| argument to list all the
120tasks in a \uC program at this point in the execution snapshot.  The internal \uC threads
121(implicitly created) are numbered with negative identifiers, while those created by the application
122are numbered with zero/positive. The \verb|*| indicates the \uC thread (\verb|T0|) that encountered
123the breakpoint at line 10. GDB stops all execution and the states of the other threads are ready,
124running, or blocked. If the argument \verb|all| is removed, only internal information about the
125\verb|userCluster| and its implicitly created threads is printed, which is sufficient for most
126applications.
127\begin{lstlisting}[caption={task command for displaying all tasks for all clusters}, label={tasks}, basicstyle=\footnotesize\tt]
128(gdb) task all
129        Cluster Name           Address
130       systemCluster          0x65a300
131  ID           Task Name           Address                    State
132  -1      uProcessorTask          0x6c99c0       uBaseTask::Blocked
133  -2         uSystemTask          0x789f40       uBaseTask::Blocked
134         userCluster          0x7ca300
135  ID           Task Name           Address                    State
136  -1      uProcessorTask          0x80ced0       uBaseTask::Blocked
137  -2           uBootTask          0x659dd0       uBaseTask::Blocked
138   0                main    0x7fffffffe490       uBaseTask::Blocked
139  -3      uProcessorTask          0x90e810       uBaseTask::Blocked
140  -4      uProcessorTask          0x98ee00       uBaseTask::Blocked
141  -5      uProcessorTask          0xa0f3f0       uBaseTask::Blocked
142 * 1                  T0          0xa4f950       uBaseTask::Running
143   2                  T1          0xa8fce0       uBaseTask::Running
144   3                  T2          0xad0070       uBaseTask::Running
145   4                  T3          0xb10400         uBaseTask::Ready
146   5                  T4          0xb50790         uBaseTask::Ready
147   6                  T5          0xb90b20         uBaseTask::Ready
148   7                  T6          0xbd0eb0         uBaseTask::Ready
149   8                  T7          0xc11240         uBaseTask::Ready
150   9                  T8          0xc515d0       uBaseTask::Running
151  10                  T9          0xc91960         uBaseTask::Ready
152\end{lstlisting}
153
154\subsection{Listing all tasks in a cluster}
155Listing \ref{cluster-tasks} shows the new command \verb|task| with a cluster argument to list only
156the names of the tasks on that cluster.  In this version of the command \verb|task|, the associated
157address for each task and its state is displayed.
158\begin{lstlisting}[caption={task command for displaying all tasks in a cluster}, label={cluster-tasks}, basicstyle=\small\tt]
159(gdb) task systemCluster
160  ID           Task Name           Address                    State
161  -1      uProcessorTask          0x6c99c0       uBaseTask::Blocked
162  -2         uSystemTask          0x789f40       uBaseTask::Blocked
163\end{lstlisting}
164
165\section{Changing Stacks}
166The next extension for displaying information is writing new commands that allow stepping from one
167\uC task to another. Each switching remembers the task tour in a LIFO way. This semantics means push
168and pop commands are needed. The push is performed by the \verb|task| command with a task argument.
169The pop is performed by the new command \verb|prevtask| or shorthand \verb|prev|.
170
171\subsection{Task Switching}
172The task argument for pushing is a relative id within a cluster or absolute address on any
173cluster. For instance, to switch from any task to task \verb|T2| seen in listing \ref{tasks}, the
174first command in listing \ref{task-addr-arguments} uses relative id (3) implicitly from the
175\verb|userCluster|, the second command uses an absolute address (\verb|0xad0070|), and the third
176command uses relative id (3) with the explicit \verb|userCluster|. Core functionality of these
177approaches is the same. Finally, the \verb|prevtask| command is used to unwind the stack until it is
178empty.
179\begin{lstlisting}[caption={task command arguments}, label={task-addr-arguments}, basicstyle=\small\tt]
180(gdb) task 3
181(gdb) task 0xad0070
182(gdb) task 3 userCluster
183(gdb) prevtask
184...
185(gdb) prev
186...
187(gdb) prev
188...
189(gdb) prev
190empty stack
191\end{lstlisting}
192
193
194\subsection{Switching Implementation}
195To implement the task tour, it is necessary to store the context information for every context
196switching. This requirement means the \verb|task| command needs to store this information every time
197it is invoked.
198
199\begin{figure}
200    \centering
201    \includegraphics[width=8cm]{uContext_stack}
202    \caption{Machine context (uMachContext) for each task}
203    \label{machine-context}
204
205\vspace*{0.5in}
206
207\begin{lstlisting}[style=Python, caption={Abridged \texttt{push\_task} source code}, label={pushtask-code}, basicstyle=\small\tt]
208# get GDB type of uContext_t *
209uContext_t_ptr_type = gdb.lookup_type('UPP::uMachContext::uContext_t').pointer()
210
211# retrieve the context object from a task and cast it to the type uContext_t *
212task_context = task['context'].cast(uContext_t_ptr_type)
213
214# the offset where sp would be starting from uSwitch function
215sp_address_offset = 48
216# lookup the value of stack pointer (sp), frame pointer (fp),
217# program counter (pc)
218xsp = task_context['SP'] + sp_address_offset
219xfp = task_context['FP']
220if not gdb.lookup_symbol('uSwitch'):
221    print('uSwitch symbol is unavailable')
222    return
223
224# This value is calculated here because we always here when the task is in
225# blocked state
226xpc = get_addr(gdb.parse_and_eval('uSwitch').address + 28)
227# must switch back to frame-0 to set 'pc' register with the value of xpc
228gdb.execute('select-frame 0')
229
230# retrieve register values and push sp, fp, pc into a global stack
231global STACK
232sp = gdb.parse_and_eval('$sp')
233fp = gdb.parse_and_eval('$fp')
234pc = gdb.parse_and_eval('$pc')
235stack_info = StackInfo(sp = sp, fp = fp, pc = pc)
236STACK.append(stack_info)
237
238# update registers for new task
239gdb.execute('set $rsp={}'.format(xsp))
240gdb.execute('set $rbp={}'.format(xfp))
241gdb.execute('set $pc={}'.format(xpc))
242\end{lstlisting}
243\end{figure}
244
245Figure \ref{machine-context} shows a task points to a structure containing a \verb|uContext_t| data
246structure, storing the stack and frame pointer, and the stack pointer. Listing \ref{pushtask-code}
247shows these pointers are copied into an instance of the Python tuple \verb|StackInfo| for every
248level of task switching. This tuple also stores information about the program counter that is
249calculated from the address of the \verb|uSwitch| assembly function because a task always stops in
250\verb|uSwitch| when its state is blocked.  Similarly, switching commands retrieve this context
251information but from the task that a user wants to switch to, and sets the equivalent registers to
252the new values.
253
254To push using the \verb|task| command, the value of the hardware stack pointer \verb|rsp| register,
255frame pointer \verb|rbp| register, and program counter register \verb|pc| are copied from the
256blocked task's save-area to the Python stack.  To pop using the \verb|prevtask| command, the three
257registers are moved from the Python stack to the appropriate hardware registers. Popping an empty
258stack prints a warning.
259
260Note, for tasks running when a breakpoint is encountered, the task's save-area is out-of-date; i.e.,
261the save area is only updated on a context switch, and a running task's stack represents the current
262unstored state for that task, which will be stored at the next context switch. Hence, to examine
263running tasks, it is necessary to use the GDB \verb|info threads| and \verb|thread| commands to
264examine and then step onto running tasks.
265
266Listing \ref{rr-tasks} shows how to examine ready and running tasks. Task \verb|T3| is ready (see
267Listing \ref{tasks}) because it was forced to context switch because of a time-slice preemption.
268Switching to \verb|T3|, which is relative id 4, and listing its the backtrace (stack frames) shows
269frames 0--6, which are the execution sequence for a time-slice preemption (and can be ignored), and
270frames 7--9, which are the frames at the point of preemption.  Frame 7 shows \verb|T3| is at line 14
271in the test program (see Listing \ref{uC-src-code}). Switching to running task \verb|T1|, which is
272relative id 2 (see Listing \ref{tasks}), and listing its backtrace shows a similar backtrace to
273ready task \verb|T3|. However, this backtrace contains stale information.  The GDB command
274\verb|info threads| shows the status of each kernel thread in the application, which represents the
275true location of each running thread. By observation, it can be seen that thread 2 is executing task
276\verb|0xa8fce0|, which is task \verb|T1|. Switching to kernel thread 2 via GDB command
277\verb|thread 2| and listing its backtrace show that task \verb|T1| is current executing at line 13
278in the test program.
279
280\begin{figure}
281\begin{lstlisting}[numbers=left, xleftmargin=3.0ex, caption={Examine ready/running tasks}, label={rr-tasks}, basicstyle=\footnotesize\tt]
282(gdb) task 4
283#0  T::f (this=0xa4f950, param=0) at test.cc:10
284(gdb) backtrace
285#0  uSwitch () at /u0/usystem/software/u++-7.0.0/src/kernel/uSwitch-x86_64.S:64
286#1  0x000000000042bd5c in uBaseCoroutine::taskCxtSw (this=0x8c9d28) ...
287#2  0x000000000042fff4 in UPP::uProcessorKernel::scheduleInternal ...
288#3  0x000000000042d4b6 in uBaseTask::uYieldInvoluntary ...
289#4  0x000000000042172f in uKernelModule::rollForward ...
290#5  0x000000000042f4fe in UPP::uSigHandlerModule::sigAlrmHandler ...
291#6  <signal handler called>
292#7  0x000000000041e620 in T::main (`this=0xb10400`) at test.cc:14
293#8  0x0000000000428de2 in UPP::uMachContext::invokeTask (This=...) ...
294#9  0x0000000000000000 in ?? ()
295(gdb) task 2
296(gdb) backtrace
297#0  uSwitch () at /u0/usystem/software/u++-7.0.0/src/kernel/uSwitch-x86_64.S:64
298#1  0x000000000042bd70 in uBaseCoroutine::taskCxtSw (this=0x8c9b68) ...
299#2  0x000000000042fff4 in UPP::uProcessorKernel::scheduleInternal ...
300#3  0x000000000042d4b6 in uBaseTask::uYieldInvoluntary ...
301#4  0x000000000042172f in uKernelModule::rollForward ...
302#5  0x000000000042f50c in UPP::uSigHandlerModule::sigAlrmHandler ...
303#6  <signal handler called>
304#7  0x000000000041e620 in T::main (`this=0xa8fce0`) at test.cc:14
305#8  0x0000000000428de2 in UPP::uMachContext::invokeTask (This=...) ...
306#9  0x0000000000000000 in ?? ()
307(gdb) info threads
308  Id Target Id                           Frame
309  1  Thread 0x7ffff7fc8780 (LWP 7425) "a.out" 0x00007ffff6d74826 in ...
310  2  Thread 0x808080 (LWP 7923) "a.out"  0x41e5fc in T::main `(this=0xa8fce0`) at test.cc:13
311* 3  Thread 0x90a080 (LWP 7926) "a.out"  uSwitch () ...
312  4  Thread 0x98a080 (LWP 7929) "a.out"  T::main (this=0xad0070) at test.cc:14
313  5  Thread 0xa0b080 (LWP 7931) "a.out"  0x41e629 in T::main (this=0xc515d0) at test.cc:14
314(gdb) thread 2
315#1  0x000000000041e509 in T::f (this=0xa4f950, param=1) at test.cc:6
3166               if ( param != 0 ) f( param - 1 );       // recursion
317[Switching to thread 2 (Thread 0x808080 (LWP 7923))]
318#0  0x000000000041e5fc in T::main (`this=0xa8fce0`) at test.cc:13
31913                  for ( volatile size_t i = 0; i < 1000000000; i += 1 ) // delay
320(gdb) backtrace
321#0  0x000000000041e5fc in T::main (`this=0xa8fce0`) at test.cc:13
322#1  0x0000000000428de2 in UPP::uMachContext::invokeTask (This=...) ...
323#2  0x0000000000000000 in ?? ()
324\end{lstlisting}
325\end{figure}
326
327\subsection{Continuing Implementation}
328When a breakpoint or error is encountered, all concurrent execution stops.  The state of the program
329can now be examined and changed; after which the program may be continued. Continuation must always
330occur from the top of the stack (current call) for each task, and at the specific task where GDB
331stopped execution.
332
333However, during a task tour, the new GDB commands change the notion of the task where execution
334stopped to make it possible to walk other stacks.  Hence, it is a requirement for continuation that
335the task walk always return to frame-0 of the original stopped task before any program continuation
336\cite{Reference11}.
337
338% For every new function call, a new stack frame is created and the values of all the registers are
339% changed for that frame. Therefore, in order to see the true value of hardware registers, innermost
340% frame that is frame-0 must be selected \cite{Reference11}. However, it is possible to not be in
341% frame-0, so prior to setting these values, the command must switch back to the innermost
342% (currently executing) frame first.
343
344To provide for this requirement, the original stop task is implicitly remembered, and there is a new
345\verb|reset| command that \emph{must} be explicitly executed before any continuation to restore the
346locate state.  To prevent errors from forgetting to call the \verb|reset| command, additional hooks
347are added to the existing built-in GDB continuation commands to implicitly call \verb|reset|. The
348following list of these commands results from GDB documentation \cite{Reference15} and similar work
349done for KOS \cite{Reference14}.
350\begin{lstlisting}[caption={Built-in GDB commands that allow continuation of a program}, label={continue-cmds}, basicstyle=\small\tt]
351continue,next,nexti,step,stepi,finish,advance,jump,signal,until,run,thread,
352reverse-next,reverse-step,reverse-stepi,reverse-continue,reverse-finish
353\end{lstlisting}
354
355% These hooks call a new command called \verb|reset| prior to executing the command to enable
356% continuation of a program to ensure that the program's context is automatically switched back to
357% the context of the task that initiates the first context switch. The \verb|reset| command behaves
358% as same as the command \verb|prevtask|, however, it goes back directly to where the task is when
359% the program last stops, which is the first task in the task tour.
360
361\section{Result}
362The current implementation successfully allows users to display a snapshot of \uC execution with
363respect to clusters, processors, and tasks. With this information it is possible to tour the call
364stacks of the tasks to see execution locations and data values. Additionally, users are allowed to
365continue the execution where the program last pauses assuming that the program has not crashed. The
366continuation of execution is done by automatically reversing the task walk from any existing GDB
367commands such as \verb|continue|, or a user can manually reverse the task walk using the command
368\verb|prevtask| and then continue.
Note: See TracBrowser for help on using the repository browser.