Changes in / [f93c50a:7e7a076]


Ignore:
Files:
8 deleted
24 edited

Legend:

Unmodified
Added
Removed

  • doc/theses/andrew_beach_MMath/Makefile

    rf93c50a r7e7a076  
    3131
    3232# The main rule, it does all the tex/latex processing.
    33 ${BUILD}/${BASE}.dvi: ${RAWSRC} ${FIGTEX} termhandle.pstex resumhandle.pstex Makefile | ${BUILD}
     33${BUILD}/${BASE}.dvi: ${RAWSRC} ${FIGTEX} Makefile | ${BUILD}
    3434        ${LATEX} ${BASE}
    3535        ${BIBTEX} ${BUILD}/${BASE}
    … …  
    4040${FIGTEX}: ${BUILD}/%.tex: %.fig | ${BUILD}
    4141        fig2dev -L eepic $< > $@
    42 
    43 %.pstex : %.fig | ${Build}
    44         fig2dev -L pstex $< > ${BUILD}/$@
    45         fig2dev -L pstex_t -p ${BUILD}/$@ $< > ${BUILD}/$@_t
    4642
    4743# Step through dvi & postscript to handle xfig specials.

  • doc/theses/andrew_beach_MMath/existing.tex

    rf93c50a r7e7a076  
    4949asterisk (@*@) is replaced with a ampersand (@&@);
    5050this includes cv-qualifiers (\snake{const} and \snake{volatile})
     51%\todo{Should I go into even more detail on cv-qualifiers.}
    5152and multiple levels of reference.
    5253

  • doc/theses/andrew_beach_MMath/features.tex

    rf93c50a r7e7a076  
    129129\section{Virtuals}
    130130\label{s:virtuals}
    131 A common feature in many programming languages is a tool to pair code
    132 (behaviour) with data.
    133 In \CFA this is done with the virtual system,
    134 which allow type information to be abstracted away, recovered and allow
    135 operations to be performed on the abstract objects.
    136 
     131%\todo{Maybe explain what "virtual" actually means.}
    137132Virtual types and casts are not part of \CFA's EHM nor are they required for
    138133an EHM.
    … …  
    493488Since it is so general, a more specific handler can be defined,
    494489overriding the default behaviour for the specific exception types.
    495 
    496 For example, consider an error reading a configuration file.
    497 This is most likely a problem with the configuration file (@config_error@),
    498 but the function could have been passed the wrong file name (@arg_error@).
    499 In this case the function could raise one exception and then, if it is
    500 unhandled, raise the other.
    501 This is not usual behaviour for either exception so changing the
    502 default handler will be done locally:
    503 \begin{cfa}
    504 {
    505         void defaultTerminationHandler(config_error &) {
    506                 throw (arg_error){arg_vt};
    507         }
    508         throw (config_error){config_vt};
    509 }
    510 \end{cfa}
     490%\todo{Examples?}
    511491
    512492\subsection{Resumption}
    … …  
    571551the just handled exception came from, and continues executing after it,
    572552not after the try statement.
    573 
    574 For instance, a resumption used to send messages to the logger may not
    575 need to be handled at all. Putting the following default handler
    576 at the global scope can make handling that exception optional by default.
    577 \begin{cfa}
    578 void defaultResumptionHandler(log_message &) {
    579     // Nothing, it is fine not to handle logging.
    580 }
    581 // ... No change at raise sites. ...
    582 throwResume (log_message){strlit_log, "Begin event processing."}
    583 \end{cfa}
     553%\todo{Examples?}
    584554
    585555\subsubsection{Resumption Marking}
    … …  
    908878After a coroutine stack is unwound, control returns to the @resume@ function
    909879that most recently resumed it. @resume@ reports a
    910 @CoroutineCancelled@ exception, which contains a reference to the cancelled
     880@CoroutineCancelled@ exception, which contains a references to the cancelled
    911881coroutine and the exception used to cancel it.
    912882The @resume@ function also takes the \defaultResumptionHandler{} from the

  • doc/theses/andrew_beach_MMath/implement.tex

    rf93c50a r7e7a076  
    5050The problem is that a type ID may appear in multiple TUs that compose a
    5151program (see \autoref{ss:VirtualTable}), so the initial solution would seem
    52 to be make it external in each translation unit. However, the type ID must
     52to be make it external in each translation unit. Hovever, the type ID must
    5353have a declaration in (exactly) one of the TUs to create the storage.
    5454No other declaration related to the virtual type has this property, so doing
    … …  
    167167\subsection{Virtual Table}
    168168\label{ss:VirtualTable}
     169%\todo{Clarify virtual table type vs. virtual table instance.}
    169170Each virtual type has a virtual table type that stores its type ID and
    170171virtual members.
    171 An instance of a virtual type is bound to a virtual table instance,
    172 which have the values of the virtual members.
    173 Both the layout of the fields (in the virtual table type)
    174 and their value (in the virtual table instance) are decided by the rules given
     172Each virtual type instance is bound to a table instance that is filled with
     173the values of virtual members.
     174Both the layout of the fields and their value are decided by the rules given
    175175below.
    176176
    … …  
    414414of a function's state with @setjmp@ and restoring that snapshot with
    415415@longjmp@. This approach bypasses the need to know stack details by simply
    416 resetting to a snapshot of an arbitrary but existing function frame on the
     416reseting to a snapshot of an arbitrary but existing function frame on the
    417417stack. It is up to the programmer to ensure the snapshot is valid when it is
    418418reset and that all required cleanup from the unwound stacks is performed.
    419 Because it does not automate or check any of this cleanup,
    420 it can be easy to make mistakes and always must be handled manually.
     419This approach is fragile and requires extra work in the surrounding code.
    421420
    422421With respect to the extra work in the surrounding code,
    … …  
    436435library that provides tools for stack walking, handler execution, and
    437436unwinding. What follows is an overview of all the relevant features of
    438 libunwind needed for this work.
    439 Following that is the description of the \CFA code that uses libunwind
    440 to implement termination.
     437libunwind needed for this work, and how \CFA uses them to implement exception
     438handling.
    441439
    442440\subsection{libunwind Usage}

  • doc/theses/andrew_beach_MMath/intro.tex

    rf93c50a r7e7a076  
    3939it returns control to that function.
    4040\begin{center}
    41 %\input{termination}
    42 %
    43 %\medskip
    44 \input{termhandle.pstex_t}
    45 % I hate these diagrams, but I can't access xfig to fix them and they are
    46 % better than the alternative.
     41\input{termination}
    4742\end{center}
     43%\todo{What does the right half of termination.fig mean?}
    4844
    4945Resumption exception handling searches the stack for a handler and then calls
    … …  
    5450that preformed the raise, usually starting after the raise.
    5551\begin{center}
    56 %\input{resumption}
    57 %
    58 %\medskip
    59 \input{resumhandle.pstex_t}
    60 % The other one.
     52\input{resumption}
    6153\end{center}
    6254
    … …  
    222214unwinding the stack like in termination exception
    223215handling.\cite{RustPanicMacro}\cite{RustPanicModule}
    224 Go's panic though is very similar to a termination, except it only supports
     216Go's panic through is very similar to a termination, except it only supports
    225217a catch-all by calling \code{Go}{recover()}, simplifying the interface at
    226218the cost of flexibility.\cite{Go:2021}
    … …  
    245237through multiple functions before it is addressed.
    246238
    247 Here is an example of the pattern in Bash, where commands can only  ``return"
    248 numbers and most output is done through streams of text.
    249 \begin{lstlisting}[language=bash,escapechar={}]
    250 # Immediately after running a command:
    251 case $? in
    252 0)
    253         # Success
    254         ;;
    255 1)
    256         # Error Code 1
    257         ;;
    258 2|3)
    259         # Error Code 2 or Error Code 3
    260         ;;
    261 # Add more cases as needed.
    262 asac
    263 \end{lstlisting}
    264 
    265239\item\emph{Special Return with Global Store}:
    266240Similar to the error codes pattern but the function itself only returns
    … …  
    272246
    273247This approach avoids the multiple results issue encountered with straight
    274 error codes as only a single error value has to be returned,
    275 but otherwise has the same disadvantages and more.
     248error codes but otherwise has the same disadvantages and more.
    276249Every function that reads or writes to the global store must agree on all
    277250possible errors and managing it becomes more complex with concurrency.
    278 
    279 This example shows some of what has to be done to robustly handle a C
    280 standard library function that reports errors this way.
    281 \begin{lstlisting}[language=C]
    282 // Make sure to clear the store.
    283 errno = 0;
    284 // Now a library function can set the error.
    285 int handle = open(path_name, flags);
    286 if (-1 == handle) {
    287         switch (errno) {
    288     case ENAMETOOLONG:
    289                 // path_name is a bad argument.
    290                 break;
    291         case ENFILE:
    292                 // A system resource has been exausted.
    293                 break;
    294         // And many more...
    295     }
    296 }
    297 \end{lstlisting}
    298 % cite open man page?
    299251
    300252\item\emph{Return Union}:
    … …  
    307259This pattern is very popular in any functional or semi-functional language
    308260with primitive support for tagged unions (or algebraic data types).
    309 Return unions can also be expressed as monads (evaluation in a context)
    310 and often are in languages with special syntax for monadic evaluation,
    311 such as Haskell's \code{haskell}{do} blocks.
    312 
     261% We need listing Rust/rust to format code snippets from it.
     262% Rust's \code{rust}{Result<T, E>}
    313263The main advantage is that an arbitrary object can be used to represent an
    314264error, so it can include a lot more information than a simple error code.
    … …  
    316266execution, and if there aren't primitive tagged unions proper, usage can be
    317267hard to enforce.
    318 % We need listing Rust/rust to format code snippets from it.
    319 % Rust's \code{rust}{Result<T, E>}
    320 
    321 This is a simple example of examining the result of a failing function in
    322 Haskell, using its \code{haskell}{Either} type.
    323 Examining \code{haskell}{error} further would likely involve more matching,
    324 but the type of \code{haskell}{error} is user defined so there are no
    325 general cases.
    326 \begin{lstlisting}[language=haskell]
    327 case failingFunction argA argB of
    328     Right value -> -- Use the successful computed value.
    329     Left error -> -- Handle the produced error.
    330 \end{lstlisting}
    331 
    332 Return unions as monads will result in the same code, but can hide most
    333 of the work to propagate errors in simple cases. The code to actually handle
    334 the errors, or to interact with other monads (a common case in these
    335 languages) still has to be written by hand.
    336 
    337 If \code{haskell}{failingFunction} is implemented with two helpers that
    338 use the same error type, then it can be implemented with a \code{haskell}{do}
    339 block.
    340 \begin{lstlisting}[language=haskell]
    341 failingFunction x y = do
    342         z <- helperOne x
    343         helperTwo y z
    344 \end{lstlisting}
    345268
    346269\item\emph{Handler Functions}:
    … …  
    363286function calls, but cheaper (constant time) to call,
    364287they are more suited to more frequent (less exceptional) situations.
    365 Although, in \Cpp and other languages that do not have checked exceptions,
    366 they can actually be enforced by the type system be more reliable.
    367 
    368 This is a more local example in \Cpp, using a function to provide
    369 a default value for a mapping.
    370 \begin{lstlisting}[language=C++]
    371 ValueT Map::key_or_default(KeyT key, ValueT(*make_default)(KeyT)) {
    372         ValueT * value = find_value(key);
    373         if (nullptr != value) {
    374                 return *value;
    375         } else {
    376                 return make_default(key);
    377         }
    378 }
    379 \end{lstlisting}
    380288\end{itemize}
    381289

  • doc/theses/andrew_beach_MMath/performance.tex

    rf93c50a r7e7a076  
    3737resumption exceptions. Even the older programming languages with resumption
    3838seem to be notable only for having resumption.
    39 On the other hand, the functional equivalents to resumption are too new.
    40 There does not seem to be any standard implementations in well-known
    41 languages, so far they seem confined to extensions and research languages.
    42 % There was some maybe interesting comparison to an OCaml extension
    43 % but I'm not sure how to get that working if it is interesting.
    4439Instead, resumption is compared to its simulation in other programming
    4540languages: fixup functions that are explicitly passed into a function.
    … …  
    317312\CFA, \Cpp and Java.
    318313% To be exact, the Match All and Match None cases.
    319 The most likely explination is that,
    320 the generally faster languages have made ``common cases fast" at the expense
    321 of the rarer cases. Since exceptions are considered rare, they are made
    322 expensive to help speed up common actions, such as entering and leaving try
    323 statements.
    324 Python on the other hand, while generally slower than the other languages,
    325 uses exceptions more and has not scarified their performance.
     314%\todo{Not true in Python.}
     315The most likely explanation is that, since exceptions
     316are rarely considered to be the common case, the more optimized languages
     317make that case expensive to improve other cases.
    326318In addition, languages with high-level representations have a much
    327319easier time scanning the stack as there is less to decode.

  • doc/theses/andrew_beach_MMath/uw-ethesis.bib

    rf93c50a r7e7a076  
    5050    author={The Rust Team},
    5151    key={Rust Panic Macro},
    52     howpublished={\href{https://doc.rust-lang.org/std/macro.panic.html}{https://\-doc.rust-lang.org/\-std/\-macro.panic.html}},
     52    howpublished={\href{https://doc.rust-lang.org/std/panic/index.html}{https://\-doc.rust-lang.org/\-std/\-panic/\-index.html}},
    5353    addendum={Accessed 2021-08-31},
    5454}

  • libcfa/src/concurrency/clib/cfathread.cfa

    rf93c50a r7e7a076  
    1313// Update Count     :
    1414//
    15 
    16 // #define EPOLL_FOR_SOCKETS
    1715
    1816#include "fstream.hfa"
    … …  
    2523#include "cfathread.h"
    2624
    27 extern "C" {
    28                 #include <string.h>
    29                 #include <errno.h>
    30 }
    31 
    3225extern void ?{}(processor &, const char[], cluster &, thread$ *);
    3326extern "C" {
    3427      extern void __cfactx_invoke_thread(void (*main)(void *), void * this);
    35         extern int accept4(int sockfd, struct sockaddr *addr, socklen_t *addrlen, int flags);
    3628}
    3729
    3830extern Time __kernel_get_time();
    39 extern unsigned register_proc_id( void );
    4031
    4132//================================================================================
    42 // Epoll support for sockets
    43 
    44 #if defined(EPOLL_FOR_SOCKETS)
    45         extern "C" {
    46                 #include <sys/epoll.h>
    47                 #include <sys/resource.h>
    48         }
    49 
    50         static pthread_t master_poller;
    51         static int master_epollfd = 0;
    52         static size_t poller_cnt = 0;
    53         static int * poller_fds = 0p;
    54         static struct leaf_poller * pollers = 0p;
    55 
    56         struct __attribute__((aligned)) fd_info_t {
    57                 int pollid;
    58                 size_t rearms;
    59         };
    60         rlim_t fd_limit = 0;
    61         static fd_info_t * volatile * fd_map = 0p;
    62 
    63         void * master_epoll( __attribute__((unused)) void * args ) {
    64                 unsigned id = register_proc_id();
    65 
    66                 enum { MAX_EVENTS = 5 };
    67                 struct epoll_event events[MAX_EVENTS];
    68                 for() {
    69                         int ret = epoll_wait(master_epollfd, events, MAX_EVENTS, -1);
    70                         if ( ret < 0 ) {
    71                                 abort | "Master epoll error: " | strerror(errno);
    72                         }
    73 
    74                         for(i; ret) {
    75                                 thread$ * thrd = (thread$ *)events[i].data.u64;
    76                                 unpark( thrd );
    77                         }
    78                 }
    79 
    80                 return 0p;
    81         }
    82 
    83         static inline int epoll_rearm(int epollfd, int fd, uint32_t event) {
    84                 struct epoll_event eevent;
    85                 eevent.events = event | EPOLLET | EPOLLONESHOT;
    86                 eevent.data.u64 = (uint64_t)active_thread();
    87 
    88                 if(0 != epoll_ctl(epollfd, EPOLL_CTL_MOD, fd, &eevent))
    89                 {
    90                         if(errno == ENOENT) return -1;
    91                         abort | acquire | "epoll" | epollfd | "ctl rearm" | fd | "error: " | errno | strerror(errno);
    92                 }
    93 
    94                 park();
    95                 return 0;
    96         }
    97 
    98         thread leaf_poller {
    99                 int epollfd;
    100         };
    101 
    102         void ?{}(leaf_poller & this, int fd) { this.epollfd = fd; }
    103 
    104         void main(leaf_poller & this) {
    105                 enum { MAX_EVENTS = 1024 };
    106                 struct epoll_event events[MAX_EVENTS];
    107                 const int max_retries = 5;
    108                 int retries = max_retries;
    109 
    110                 struct epoll_event event;
    111                 event.events = EPOLLIN | EPOLLET | EPOLLONESHOT;
    112                 event.data.u64 = (uint64_t)&(thread&)this;
    113 
    114                 if(0 != epoll_ctl(master_epollfd, EPOLL_CTL_ADD, this.epollfd, &event))
    115                 {
    116                         abort | "master epoll ctl add leaf: " | errno | strerror(errno);
    117                 }
    118 
    119                 park();
    120 
    121                 for() {
    122                         yield();
    123                         int ret = epoll_wait(this.epollfd, events, MAX_EVENTS, 0);
    124                         if ( ret < 0 ) {
    125                                 abort | "Leaf epoll error: " | errno | strerror(errno);
    126                         }
    127 
    128                         if(ret) {
    129                                 for(i; ret) {
    130                                         thread$ * thrd = (thread$ *)events[i].data.u64;
    131                                         unpark( thrd, UNPARK_REMOTE );
    132                                 }
    133                         }
    134                         else if(0 >= --retries) {
    135                                 epoll_rearm(master_epollfd, this.epollfd, EPOLLIN);
    136                         }
    137                 }
    138         }
    139 
    140         void setup_epoll( void ) __attribute__(( constructor ));
    141         void setup_epoll( void ) {
    142                 if(master_epollfd) abort | "Master epoll already setup";
    143 
    144                 master_epollfd = epoll_create1(0);
    145                 if(master_epollfd == -1) {
    146                         abort | "failed to create master epoll: " | errno | strerror(errno);
    147                 }
    148 
    149                 struct rlimit rlim;
    150                 if(int ret = getrlimit(RLIMIT_NOFILE, &rlim); 0 != ret) {
    151                         abort | "failed to get nofile limit: " | errno | strerror(errno);
    152                 }
    153 
    154                 fd_limit = rlim.rlim_cur;
    155                 fd_map = alloc(fd_limit);
    156                 for(i;fd_limit) {
    157                         fd_map[i] = 0p;
    158                 }
    159 
    160                 poller_cnt = 2;
    161                 poller_fds = alloc(poller_cnt);
    162                 pollers    = alloc(poller_cnt);
    163                 for(i; poller_cnt) {
    164                         poller_fds[i] = epoll_create1(0);
    165                         if(poller_fds[i] == -1) {
    166                                 abort | "failed to create leaf epoll [" | i | "]: " | errno | strerror(errno);
    167                         }
    168 
    169                         (pollers[i]){ poller_fds[i] };
    170                 }
    171 
    172                 pthread_attr_t attr;
    173                 if (int ret = pthread_attr_init(&attr); 0 != ret) {
    174                         abort | "failed to create master epoll thread attr: " | ret | strerror(ret);
    175                 }
    176 
    177                 if (int ret = pthread_create(&master_poller, &attr, master_epoll, 0p); 0 != ret) {
    178                         abort | "failed to create master epoll thread: " | ret | strerror(ret);
    179                 }
    180         }
    181 
    182         static inline int epoll_wait(int fd, uint32_t event) {
    183                 if(fd_map[fd] >= 1p) {
    184                         fd_map[fd]->rearms++;
    185                         epoll_rearm(poller_fds[fd_map[fd]->pollid], fd, event);
    186                         return 0;
    187                 }
    188 
    189                 for() {
    190                         fd_info_t * expected = 0p;
    191                         fd_info_t * sentinel = 1p;
    192                         if(__atomic_compare_exchange_n( &(fd_map[fd]), &expected, sentinel, true, __ATOMIC_SEQ_CST, __ATOMIC_RELAXED)) {
    193                                 struct epoll_event eevent;
    194                                 eevent.events = event | EPOLLET | EPOLLONESHOT;
    195                                 eevent.data.u64 = (uint64_t)active_thread();
    196 
    197                                 int id = thread_rand() % poller_cnt;
    198                                 if(0 != epoll_ctl(poller_fds[id], EPOLL_CTL_ADD, fd, &eevent))
    199                                 {
    200                                         abort | "epoll ctl add" | poller_fds[id] | fd | fd_map[fd] | expected | "error: " | errno | strerror(errno);
    201                                 }
    202 
    203                                 fd_info_t * ninfo = alloc();
    204                                 ninfo->pollid = id;
    205                                 ninfo->rearms = 0;
    206                                 __atomic_store_n( &fd_map[fd], ninfo, __ATOMIC_SEQ_CST);
    207 
    208                                 park();
    209                                 return 0;
    210                         }
    211 
    212                         if(expected >= 0) {
    213                                 fd_map[fd]->rearms++;
    214                                 epoll_rearm(poller_fds[fd_map[fd]->pollid], fd, event);
    215                                 return 0;
    216                         }
    217 
    218                         Pause();
    219                 }
    220         }
    221 #endif
    222 
    223 //================================================================================
    224 // Thread run by the C Interface
     33// Thread run y the C Interface
    22534
    22635struct cfathread_object {
    … …  
    436245        // Mutex
    437246        struct cfathread_mutex {
    438                 linear_backoff_then_block_lock impl;
     247                fast_lock impl;
    439248        };
    440249        int cfathread_mutex_init(cfathread_mutex_t *restrict mut, const cfathread_mutexattr_t *restrict) __attribute__((nonnull (1))) { *mut = new(); return 0; }
    … …  
    451260        // Condition
    452261        struct cfathread_condition {
    453                 condition_variable(linear_backoff_then_block_lock) impl;
     262                condition_variable(fast_lock) impl;
    454263        };
    455264        int cfathread_cond_init(cfathread_cond_t *restrict cond, const cfathread_condattr_t *restrict) __attribute__((nonnull (1))) { *cond = new(); return 0; }
    … …  
    479288        // IO operations
    480289        int cfathread_socket(int domain, int type, int protocol) {
    481                 return socket(domain, type
    482                 #if defined(EPOLL_FOR_SOCKETS)
    483                         | SOCK_NONBLOCK
    484                 #endif
    485                 , protocol);
     290                return socket(domain, type, protocol);
    486291        }
    487292        int cfathread_bind(int socket, const struct sockaddr *address, socklen_t address_len) {
    … …  
    494299
    495300        int cfathread_accept(int socket, struct sockaddr *restrict address, socklen_t *restrict address_len) {
    496                 #if defined(EPOLL_FOR_SOCKETS)
    497                         int ret;
    498                         for() {
    499                                 yield();
    500                                 ret = accept4(socket, address, address_len, SOCK_NONBLOCK);
    501                                 if(ret >= 0) break;
    502                                 if(errno != EAGAIN && errno != EWOULDBLOCK) break;
    503 
    504                                 epoll_wait(socket, EPOLLIN);
    505                         }
    506                         return ret;
    507                 #else
    508                         return cfa_accept4(socket, address, address_len, 0, CFA_IO_LAZY);
    509                 #endif
     301                return cfa_accept4(socket, address, address_len, 0, CFA_IO_LAZY);
    510302        }
    511303
    512304        int cfathread_connect(int socket, const struct sockaddr *address, socklen_t address_len) {
    513                 #if defined(EPOLL_FOR_SOCKETS)
    514                         int ret;
    515                         for() {
    516                                 ret = connect(socket, address, address_len);
    517                                 if(ret >= 0) break;
    518                                 if(errno != EAGAIN && errno != EWOULDBLOCK) break;
    519 
    520                                 epoll_wait(socket, EPOLLIN);
    521                         }
    522                         return ret;
    523                 #else
    524                         return cfa_connect(socket, address, address_len, CFA_IO_LAZY);
    525                 #endif
     305                return cfa_connect(socket, address, address_len, CFA_IO_LAZY);
    526306        }
    527307
    … …  
    535315
    536316        ssize_t cfathread_sendmsg(int socket, const struct msghdr *message, int flags) {
    537                 #if defined(EPOLL_FOR_SOCKETS)
    538                         ssize_t ret;
    539                         __STATS__( false, io.ops.sockwrite++; )
    540                         for() {
    541                                 ret = sendmsg(socket, message, flags);
    542                                 if(ret >= 0) break;
    543                                 if(errno != EAGAIN && errno != EWOULDBLOCK) break;
    544 
    545                                 __STATS__( false, io.ops.epllwrite++; )
    546                                 epoll_wait(socket, EPOLLOUT);
    547                         }
    548                 #else
    549                         ssize_t ret = cfa_sendmsg(socket, message, flags, CFA_IO_LAZY);
    550                 #endif
    551                 return ret;
     317                return cfa_sendmsg(socket, message, flags, CFA_IO_LAZY);
    552318        }
    553319
    554320        ssize_t cfathread_write(int fildes, const void *buf, size_t nbyte) {
    555321                // Use send rather then write for socket since it's faster
    556                 #if defined(EPOLL_FOR_SOCKETS)
    557                         ssize_t ret;
    558                         // __STATS__( false, io.ops.sockwrite++; )
    559                         for() {
    560                                 ret = send(fildes, buf, nbyte, 0);
    561                                 if(ret >= 0) break;
    562                                 if(errno != EAGAIN && errno != EWOULDBLOCK) break;
    563 
    564                                 // __STATS__( false, io.ops.epllwrite++; )
    565                                 epoll_wait(fildes, EPOLLOUT);
    566                         }
    567                 #else
    568                         ssize_t ret = cfa_send(fildes, buf, nbyte, 0, CFA_IO_LAZY);
    569                 #endif
    570                 return ret;
     322                return cfa_send(fildes, buf, nbyte, 0, CFA_IO_LAZY);
    571323        }
    572324
    … …  
    584336                msg.msg_controllen = 0;
    585337
    586                 #if defined(EPOLL_FOR_SOCKETS)
    587                         ssize_t ret;
    588                         yield();
    589                         for() {
    590                                 ret = recvmsg(socket, &msg, flags);
    591                                 if(ret >= 0) break;
    592                                 if(errno != EAGAIN && errno != EWOULDBLOCK) break;
    593 
    594                                 epoll_wait(socket, EPOLLIN);
    595                         }
    596                 #else
    597                         ssize_t ret = cfa_recvmsg(socket, &msg, flags, CFA_IO_LAZY);
    598                 #endif
     338                ssize_t ret = cfa_recvmsg(socket, &msg, flags, CFA_IO_LAZY);
    599339
    600340                if(address_len) *address_len = msg.msg_namelen;
    … …  
    604344        ssize_t cfathread_read(int fildes, void *buf, size_t nbyte) {
    605345                // Use recv rather then read for socket since it's faster
    606                 #if defined(EPOLL_FOR_SOCKETS)
    607                         ssize_t ret;
    608                         __STATS__( false, io.ops.sockread++; )
    609                         yield();
    610                         for() {
    611                                 ret = recv(fildes, buf, nbyte, 0);
    612                                 if(ret >= 0) break;
    613                                 if(errno != EAGAIN && errno != EWOULDBLOCK) break;
    614 
    615                                 __STATS__( false, io.ops.epllread++; )
    616                                 epoll_wait(fildes, EPOLLIN);
    617                         }
    618                 #else
    619                         ssize_t ret = cfa_recv(fildes, buf, nbyte, 0, CFA_IO_LAZY);
    620                 #endif
    621                 return ret;
    622         }
    623 
    624 }
     346                return cfa_recv(fildes, buf, nbyte, 0, CFA_IO_LAZY);
     347        }
     348
     349}

  • libcfa/src/concurrency/invoke.h

    rf93c50a r7e7a076  
    170170                bool corctx_flag;
    171171
     172                int last_cpu;
     173
    172174                //SKULLDUGGERY errno is not save in the thread data structure because returnToKernel appears to be the only function to require saving and restoring it
    173175
    … …  
    175177                struct cluster * curr_cluster;
    176178
    177                 // preferred ready-queue or CPU
     179                // preferred ready-queue
    178180                unsigned preferred;
    179181

  • libcfa/src/concurrency/io.cfa

    rf93c50a r7e7a076  
    9090        static inline unsigned __flush( struct $io_context & );
    9191        static inline __u32 __release_sqes( struct $io_context & );
    92         extern void __kernel_unpark( thread$ * thrd, unpark_hint );
     92        extern void __kernel_unpark( thread$ * thrd );
    9393
    9494        bool __cfa_io_drain( processor * proc ) {
    … …  
    118118                        __cfadbg_print_safe( io, "Kernel I/O : Syscall completed : cqe %p, result %d for %p\n", &cqe, cqe.res, future );
    119119
    120                         __kernel_unpark( fulfil( *future, cqe.res, false ), UNPARK_LOCAL );
     120                        __kernel_unpark( fulfil( *future, cqe.res, false ) );
    121121                }
    122122

  • libcfa/src/concurrency/kernel.cfa

    rf93c50a r7e7a076  
    341341                                }
    342342
    343                                 __STATS( if(this->print_halts) __cfaabi_bits_print_safe( STDOUT_FILENO, "PH:%d - %lld 0\n", this->unique_id, rdtscl()); )
     343                                        __STATS( if(this->print_halts) __cfaabi_bits_print_safe( STDOUT_FILENO, "PH:%d - %lld 0\n", this->unique_id, rdtscl()); )
    344344                                __cfadbg_print_safe(runtime_core, "Kernel : core %p waiting on eventfd %d\n", this, this->idle);
    345345
    346                                 {
    347                                         eventfd_t val;
    348                                         ssize_t ret = read( this->idle, &val, sizeof(val) );
    349                                         if(ret < 0) {
    350                                                 switch((int)errno) {
    351                                                 case EAGAIN:
    352                                                 #if EAGAIN != EWOULDBLOCK
    353                                                         case EWOULDBLOCK:
    354                                                 #endif
    355                                                 case EINTR:
    356                                                         // No need to do anything special here, just assume it's a legitimate wake-up
    357                                                         break;
    358                                                 default:
    359                                                         abort( "KERNEL : internal error, read failure on idle eventfd, error(%d) %s.", (int)errno, strerror( (int)errno ) );
    360                                                 }
    361                                         }
    362                                 }
     346                                // __disable_interrupts_hard();
     347                                eventfd_t val;
     348                                eventfd_read( this->idle, &val );
     349                                // __enable_interrupts_hard();
    363350
    364351                                        __STATS( if(this->print_halts) __cfaabi_bits_print_safe( STDOUT_FILENO, "PH:%d - %lld 1\n", this->unique_id, rdtscl()); )
    … …  
    422409        /* paranoid */ verifyf( thrd_dst->link.next == 0p, "Expected null got %p", thrd_dst->link.next );
    423410        __builtin_prefetch( thrd_dst->context.SP );
     411
     412        int curr = __kernel_getcpu();
     413        if(thrd_dst->last_cpu != curr) {
     414                int64_t l = thrd_dst->last_cpu;
     415                int64_t c = curr;
     416                int64_t v = (l << 32) | c;
     417                __push_stat( __tls_stats(), v, false, "Processor", this );
     418        }
     419
     420        thrd_dst->last_cpu = curr;
    424421
    425422        __cfadbg_print_safe(runtime_core, "Kernel : core %p running thread %p (%s)\n", this, thrd_dst, thrd_dst->self_cor.name);
    … …  
    476473                if(unlikely(thrd_dst->preempted != __NO_PREEMPTION)) {
    477474                        // The thread was preempted, reschedule it and reset the flag
    478                         schedule_thread$( thrd_dst, UNPARK_LOCAL );
     475                        schedule_thread$( thrd_dst );
    479476                        break RUNNING;
    480477                }
    … …  
    560557// Scheduler routines
    561558// KERNEL ONLY
    562 static void __schedule_thread( thread$ * thrd, unpark_hint hint ) {
     559static void __schedule_thread( thread$ * thrd ) {
    563560        /* paranoid */ verify( ! __preemption_enabled() );
    564561        /* paranoid */ verify( ready_schedule_islocked());
    … …  
    580577        // Dereference the thread now because once we push it, there is not guaranteed it's still valid.
    581578        struct cluster * cl = thrd->curr_cluster;
    582         __STATS(bool outside = hint == UNPARK_LOCAL && thrd->last_proc && thrd->last_proc != kernelTLS().this_processor; )
     579        __STATS(bool outside = thrd->last_proc && thrd->last_proc != kernelTLS().this_processor; )
    583580
    584581        // push the thread to the cluster ready-queue
    585         push( cl, thrd, hint );
     582        push( cl, thrd, local );
    586583
    587584        // variable thrd is no longer safe to use
    … …  
    608605}
    609606
    610 void schedule_thread$( thread$ * thrd, unpark_hint hint ) {
     607void schedule_thread$( thread$ * thrd ) {
    611608        ready_schedule_lock();
    612                 __schedule_thread( thrd, hint );
     609                __schedule_thread( thrd );
    613610        ready_schedule_unlock();
    614611}
    … …  
    661658}
    662659
    663 void __kernel_unpark( thread$ * thrd, unpark_hint hint ) {
     660void __kernel_unpark( thread$ * thrd ) {
    664661        /* paranoid */ verify( ! __preemption_enabled() );
    665662        /* paranoid */ verify( ready_schedule_islocked());
    … …  
    669666        if(__must_unpark(thrd)) {
    670667                // Wake lost the race,
    671                 __schedule_thread( thrd, hint );
     668                __schedule_thread( thrd );
    672669        }
    673670
    … …  
    676673}
    677674
    678 void unpark( thread$ * thrd, unpark_hint hint ) {
     675void unpark( thread$ * thrd ) {
    679676        if( !thrd ) return;
    680677
    … …  
    682679                disable_interrupts();
    683680                        // Wake lost the race,
    684                         schedule_thread$( thrd, hint );
     681                        schedule_thread$( thrd );
    685682                enable_interrupts(false);
    686683        }

  • libcfa/src/concurrency/kernel.hfa

    rf93c50a r7e7a076  
    151151struct __attribute__((aligned(128))) __timestamp_t {
    152152        volatile unsigned long long tv;
    153         volatile unsigned long long ma;
    154 };
    155 
    156 // Aligned timestamps which are used by the relaxed ready queue
    157 struct __attribute__((aligned(128))) __help_cnts_t {
    158         volatile unsigned long long src;
    159         volatile unsigned long long dst;
    160         volatile unsigned long long tri;
    161 };
    162 
    163 static inline void  ?{}(__timestamp_t & this) { this.tv = 0; this.ma = 0; }
     153};
     154
     155static inline void  ?{}(__timestamp_t & this) { this.tv = 0; }
    164156static inline void ^?{}(__timestamp_t & this) {}
    165157
    … …  
    177169                // Array of times
    178170                __timestamp_t * volatile tscs;
    179 
    180                 // Array of stats
    181                 __help_cnts_t * volatile help;
    182171
    183172                // Number of lanes (empty or not)

  • libcfa/src/concurrency/kernel/fwd.hfa

    rf93c50a r7e7a076  
    119119
    120120        extern "Cforall" {
    121                 enum unpark_hint { UNPARK_LOCAL, UNPARK_REMOTE };
    122 
    123121                extern void park( void );
    124                 extern void unpark( struct thread$ *, unpark_hint );
    125                 static inline void unpark( struct thread$ * thrd ) { unpark(thrd, UNPARK_LOCAL); }
     122                extern void unpark( struct thread$ * this );
    126123                static inline struct thread$ * active_thread () {
    127124                        struct thread$ * t = publicTLS_get( this_thread );

  • libcfa/src/concurrency/kernel/startup.cfa

    rf93c50a r7e7a076  
    200200        __cfadbg_print_safe(runtime_core, "Kernel : Main cluster ready\n");
    201201
    202         // Construct the processor context of the main processor
    203         void ?{}(processorCtx_t & this, processor * proc) {
    204                 (this.__cor){ "Processor" };
    205                 this.__cor.starter = 0p;
    206                 this.proc = proc;
    207         }
    208 
    209         void ?{}(processor & this) with( this ) {
    210                 ( this.terminated ){};
    211                 ( this.runner ){};
    212                 init( this, "Main Processor", *mainCluster, 0p );
    213                 kernel_thread = pthread_self();
    214 
    215                 runner{ &this };
    216                 __cfadbg_print_safe(runtime_core, "Kernel : constructed main processor context %p\n", &runner);
    217         }
    218 
    219         // Initialize the main processor and the main processor ctx
    220         // (the coroutine that contains the processing control flow)
    221         mainProcessor = (processor *)&storage_mainProcessor;
    222         (*mainProcessor){};
    223 
    224         register_tls( mainProcessor );
    225 
    226202        // Start by initializing the main thread
    227203        // SKULLDUGGERY: the mainThread steals the process main thread
    … …  
    234210        __cfadbg_print_safe(runtime_core, "Kernel : Main thread ready\n");
    235211
     212
     213
     214        // Construct the processor context of the main processor
     215        void ?{}(processorCtx_t & this, processor * proc) {
     216                (this.__cor){ "Processor" };
     217                this.__cor.starter = 0p;
     218                this.proc = proc;
     219        }
     220
     221        void ?{}(processor & this) with( this ) {
     222                ( this.terminated ){};
     223                ( this.runner ){};
     224                init( this, "Main Processor", *mainCluster, 0p );
     225                kernel_thread = pthread_self();
     226
     227                runner{ &this };
     228                __cfadbg_print_safe(runtime_core, "Kernel : constructed main processor context %p\n", &runner);
     229        }
     230
     231        // Initialize the main processor and the main processor ctx
     232        // (the coroutine that contains the processing control flow)
     233        mainProcessor = (processor *)&storage_mainProcessor;
     234        (*mainProcessor){};
     235
     236        register_tls( mainProcessor );
     237        mainThread->last_cpu = __kernel_getcpu();
     238
    236239        //initialize the global state variables
    237240        __cfaabi_tls.this_processor = mainProcessor;
    … …  
    249252        // Add the main thread to the ready queue
    250253        // once resume is called on mainProcessor->runner the mainThread needs to be scheduled like any normal thread
    251         schedule_thread$(mainThread, UNPARK_LOCAL);
     254        schedule_thread$(mainThread);
    252255
    253256        // SKULLDUGGERY: Force a context switch to the main processor to set the main thread's context to the current UNIX
    … …  
    483486        link.next = 0p;
    484487        link.ts   = -1llu;
    485         preferred = ready_queue_new_preferred();
     488        preferred = -1u;
    486489        last_proc = 0p;
    487490        #if defined( __CFA_WITH_VERIFY__ )

  • libcfa/src/concurrency/kernel_private.hfa

    rf93c50a r7e7a076  
    4646}
    4747
    48 void schedule_thread$( thread$ *, unpark_hint hint ) __attribute__((nonnull (1)));
     48void schedule_thread$( thread$ * ) __attribute__((nonnull (1)));
    4949
    5050extern bool __preemption_enabled();
    … …  
    300300// push thread onto a ready queue for a cluster
    301301// returns true if the list was previously empty, false otherwise
    302 __attribute__((hot)) void push(struct cluster * cltr, struct thread$ * thrd, unpark_hint hint);
     302__attribute__((hot)) void push(struct cluster * cltr, struct thread$ * thrd, bool local);
    303303
    304304//-----------------------------------------------------------------------
    … …  
    321321
    322322//-----------------------------------------------------------------------
    323 // get preferred ready for new thread
    324 unsigned ready_queue_new_preferred();
    325 
    326 //-----------------------------------------------------------------------
    327323// Increase the width of the ready queue (number of lanes) by 4
    328324void ready_queue_grow  (struct cluster * cltr);

  • libcfa/src/concurrency/ready_queue.cfa

    rf93c50a r7e7a076  
    100100        #define __kernel_rseq_unregister rseq_unregister_current_thread
    101101#elif defined(CFA_HAVE_LINUX_RSEQ_H)
    102         static void __kernel_raw_rseq_register  (void);
    103         static void __kernel_raw_rseq_unregister(void);
     102        void __kernel_raw_rseq_register  (void);
     103        void __kernel_raw_rseq_unregister(void);
    104104
    105105        #define __kernel_rseq_register __kernel_raw_rseq_register
    … …  
    246246// Cforall Ready Queue used for scheduling
    247247//=======================================================================
    248 unsigned long long moving_average(unsigned long long nval, unsigned long long oval) {
    249         const unsigned long long tw = 16;
    250         const unsigned long long nw = 4;
    251         const unsigned long long ow = tw - nw;
    252         return ((nw * nval) + (ow * oval)) / tw;
    253 }
    254 
    255248void ?{}(__ready_queue_t & this) with (this) {
    256249        #if defined(USE_CPU_WORK_STEALING)
    … …  
    258251                lanes.data = alloc( lanes.count );
    259252                lanes.tscs = alloc( lanes.count );
    260                 lanes.help = alloc( cpu_info.hthrd_count );
    261253
    262254                for( idx; (size_t)lanes.count ) {
    263255                        (lanes.data[idx]){};
    264256                        lanes.tscs[idx].tv = rdtscl();
    265                         lanes.tscs[idx].ma = rdtscl();
    266                 }
    267                 for( idx; (size_t)cpu_info.hthrd_count ) {
    268                         lanes.help[idx].src = 0;
    269                         lanes.help[idx].dst = 0;
    270                         lanes.help[idx].tri = 0;
    271257                }
    272258        #else
    273259                lanes.data  = 0p;
    274260                lanes.tscs  = 0p;
    275                 lanes.help  = 0p;
    276261                lanes.count = 0;
    277262        #endif
    … …  
    285270        free(lanes.data);
    286271        free(lanes.tscs);
    287         free(lanes.help);
    288272}
    289273
    290274//-----------------------------------------------------------------------
    291275#if defined(USE_CPU_WORK_STEALING)
    292         __attribute__((hot)) void push(struct cluster * cltr, struct thread$ * thrd, unpark_hint hint) with (cltr->ready_queue) {
     276        __attribute__((hot)) void push(struct cluster * cltr, struct thread$ * thrd, bool push_local) with (cltr->ready_queue) {
    293277                __cfadbg_print_safe(ready_queue, "Kernel : Pushing %p on cluster %p\n", thrd, cltr);
    294278
    295279                processor * const proc = kernelTLS().this_processor;
    296                 const bool external = (!proc) || (cltr != proc->cltr);
    297 
    298                 // Figure out the current cpu and make sure it is valid
     280                const bool external = !push_local || (!proc) || (cltr != proc->cltr);
     281
    299282                const int cpu = __kernel_getcpu();
    300283                /* paranoid */ verify(cpu >= 0);
    … …  
    302285                /* paranoid */ verify(cpu * READYQ_SHARD_FACTOR < lanes.count);
    303286
    304                 // Figure out where thread was last time and make sure it's
    305                 /* paranoid */ verify(thrd->preferred >= 0);
    306                 /* paranoid */ verify(thrd->preferred < cpu_info.hthrd_count);
    307                 /* paranoid */ verify(thrd->preferred * READYQ_SHARD_FACTOR < lanes.count);
    308                 const int prf = thrd->preferred * READYQ_SHARD_FACTOR;
    309 
    310                 const cpu_map_entry_t & map;
    311                 choose(hint) {
    312                         case UNPARK_LOCAL : &map = &cpu_info.llc_map[cpu];
    313                         case UNPARK_REMOTE: &map = &cpu_info.llc_map[prf];
    314                 }
     287                const cpu_map_entry_t & map = cpu_info.llc_map[cpu];
    315288                /* paranoid */ verify(map.start * READYQ_SHARD_FACTOR < lanes.count);
    316289                /* paranoid */ verify(map.self * READYQ_SHARD_FACTOR < lanes.count);
    … …  
    323296                        if(unlikely(external)) { r = __tls_rand(); }
    324297                        else { r = proc->rdq.its++; }
    325                         choose(hint) {
    326                                 case UNPARK_LOCAL : i = start + (r % READYQ_SHARD_FACTOR);
    327                                 case UNPARK_REMOTE: i = prf   + (r % READYQ_SHARD_FACTOR);
    328                         }
     298                        i = start + (r % READYQ_SHARD_FACTOR);
    329299                        // If we can't lock it retry
    330300                } while( !__atomic_try_acquire( &lanes.data[i].lock ) );
    … …  
    362332                processor * const proc = kernelTLS().this_processor;
    363333                const int start = map.self * READYQ_SHARD_FACTOR;
    364                 const unsigned long long ctsc = rdtscl();
    365334
    366335                // Did we already have a help target
    367336                if(proc->rdq.target == -1u) {
    368                         unsigned long long max = 0;
     337                        // if We don't have a
     338                        unsigned long long min = ts(lanes.data[start]);
    369339                        for(i; READYQ_SHARD_FACTOR) {
    370                                 unsigned long long tsc = moving_average(ctsc - ts(lanes.data[start + i]), lanes.tscs[start + i].ma);
    371                                 if(tsc > max) max = tsc;
    372                         }
    373                          proc->rdq.cutoff = (max + 2 * max) / 2;
     340                                unsigned long long tsc = ts(lanes.data[start + i]);
     341                                if(tsc < min) min = tsc;
     342                        }
     343                        proc->rdq.cutoff = min;
     344
    374345                        /* paranoid */ verify(lanes.count < 65536); // The following code assumes max 65536 cores.
    375346                        /* paranoid */ verify(map.count < 65536); // The following code assumes max 65536 cores.
    376347
    377                         if(0 == (__tls_rand() % 100)) {
     348                        if(0 == (__tls_rand() % 10_000)) {
    378349                                proc->rdq.target = __tls_rand() % lanes.count;
    379350                        } else {
    … …  
    387358                }
    388359                else {
    389                         unsigned long long max = 0;
    390                         for(i; READYQ_SHARD_FACTOR) {
    391                                 unsigned long long tsc = moving_average(ctsc - ts(lanes.data[start + i]), lanes.tscs[start + i].ma);
    392                                 if(tsc > max) max = tsc;
    393                         }
    394                         const unsigned long long cutoff = (max + 2 * max) / 2;
     360                        const unsigned long long bias = 0; //2_500_000_000;
     361                        const unsigned long long cutoff = proc->rdq.cutoff > bias ? proc->rdq.cutoff - bias : proc->rdq.cutoff;
    395362                        {
    396363                                unsigned target = proc->rdq.target;
    397364                                proc->rdq.target = -1u;
    398                                 lanes.help[target / READYQ_SHARD_FACTOR].tri++;
    399                                 if(moving_average(ctsc - lanes.tscs[target].tv, lanes.tscs[target].ma) > cutoff) {
     365                                if(lanes.tscs[target].tv < cutoff && ts(lanes.data[target]) < cutoff) {
    400366                                        thread$ * t = try_pop(cltr, target __STATS(, __tls_stats()->ready.pop.help));
    401367                                        proc->rdq.last = target;
    402368                                        if(t) return t;
    403                                         else proc->rdq.target = -1u;
    404369                                }
    405                                 else proc->rdq.target = -1u;
    406370                        }
    407371
    … …  
    464428        }
    465429
    466         __attribute__((hot)) void push(struct cluster * cltr, struct thread$ * thrd, unpark_hint hint) with (cltr->ready_queue) {
     430        __attribute__((hot)) void push(struct cluster * cltr, struct thread$ * thrd, bool push_local) with (cltr->ready_queue) {
    467431                __cfadbg_print_safe(ready_queue, "Kernel : Pushing %p on cluster %p\n", thrd, cltr);
    468432
    469                 const bool external = (hint != UNPARK_LOCAL) || (!kernelTLS().this_processor) || (cltr != kernelTLS().this_processor->cltr);
     433                const bool external = !push_local || (!kernelTLS().this_processor) || (cltr != kernelTLS().this_processor->cltr);
    470434                /* paranoid */ verify(external || kernelTLS().this_processor->rdq.id < lanes.count );
    471435
    … …  
    551515#endif
    552516#if defined(USE_WORK_STEALING)
    553         __attribute__((hot)) void push(struct cluster * cltr, struct thread$ * thrd, unpark_hint hint) with (cltr->ready_queue) {
     517        __attribute__((hot)) void push(struct cluster * cltr, struct thread$ * thrd, bool push_local) with (cltr->ready_queue) {
    554518                __cfadbg_print_safe(ready_queue, "Kernel : Pushing %p on cluster %p\n", thrd, cltr);
    555519
    556520                // #define USE_PREFERRED
    557521                #if !defined(USE_PREFERRED)
    558                 const bool external = (hint != UNPARK_LOCAL) || (!kernelTLS().this_processor) || (cltr != kernelTLS().this_processor->cltr);
     522                const bool external = !push_local || (!kernelTLS().this_processor) || (cltr != kernelTLS().this_processor->cltr);
    559523                /* paranoid */ verify(external || kernelTLS().this_processor->rdq.id < lanes.count );
    560524                #else
    561525                        unsigned preferred = thrd->preferred;
    562                         const bool external = (hint != UNPARK_LOCAL) || (!kernelTLS().this_processor) || preferred == -1u || thrd->curr_cluster != cltr;
     526                        const bool external = push_local || (!kernelTLS().this_processor) || preferred == -1u || thrd->curr_cluster != cltr;
    563527                        /* paranoid */ verifyf(external || preferred < lanes.count, "Invalid preferred queue %u for %u lanes", preferred, lanes.count );
    564528
    … …  
    681645        // Actually pop the list
    682646        struct thread$ * thrd;
    683         unsigned long long tsc_before = ts(lane);
    684647        unsigned long long tsv;
    685648        [thrd, tsv] = pop(lane);
    … …  
    695658        __STATS( stats.success++; )
    696659
    697         #if defined(USE_WORK_STEALING) || defined(USE_CPU_WORK_STEALING)
    698                 unsigned long long now = rdtscl();
     660        #if defined(USE_WORK_STEALING)
    699661                lanes.tscs[w].tv = tsv;
    700                 lanes.tscs[w].ma = moving_average(now > tsc_before ? now - tsc_before : 0, lanes.tscs[w].ma);
    701662        #endif
    702663
    703         #if defined(USE_CPU_WORK_STEALING)
    704                 thrd->preferred = w / READYQ_SHARD_FACTOR;
    705         #else
    706                 thrd->preferred = w;
    707         #endif
     664        thrd->preferred = w;
    708665
    709666        // return the popped thread
    … …  
    731688
    732689//-----------------------------------------------------------------------
    733 // get preferred ready for new thread
    734 unsigned ready_queue_new_preferred() {
    735         unsigned pref = 0;
    736         if(struct thread$ * thrd = publicTLS_get( this_thread )) {
    737                 pref = thrd->preferred;
    738         }
    739         else {
    740                 #if defined(USE_CPU_WORK_STEALING)
    741                         pref = __kernel_getcpu();
    742                 #endif
    743         }
    744 
    745         #if defined(USE_CPU_WORK_STEALING)
    746                 /* paranoid */ verify(pref >= 0);
    747                 /* paranoid */ verify(pref < cpu_info.hthrd_count);
    748         #endif
    749 
    750         return pref;
    751 }
    752 
    753 //-----------------------------------------------------------------------
    754690// Check that all the intrusive queues in the data structure are still consistent
    755691static void check( __ready_queue_t & q ) with (q) {
    … …  
    979915        extern void __enable_interrupts_hard();
    980916
    981         static void __kernel_raw_rseq_register  (void) {
     917        void __kernel_raw_rseq_register  (void) {
    982918                /* paranoid */ verify( __cfaabi_rseq.cpu_id == RSEQ_CPU_ID_UNINITIALIZED );
    983919
    … …  
    997933        }
    998934
    999         static void __kernel_raw_rseq_unregister(void) {
     935        void __kernel_raw_rseq_unregister(void) {
    1000936                /* paranoid */ verify( __cfaabi_rseq.cpu_id >= 0 );
    1001937

  • libcfa/src/concurrency/ready_subqueue.hfa

    rf93c50a r7e7a076  
    9898
    9999        // Get the relevant nodes locally
     100        unsigned long long ts = this.anchor.ts;
    100101        thread$ * node = this.anchor.next;
    101102        this.anchor.next = node->link.next;
    … …  
    115116        /* paranoid */ verify( node->link.ts   != 0  );
    116117        /* paranoid */ verify( this.anchor.ts  != 0  );
    117         return [node, this.anchor.ts];
     118        return [node, ts];
    118119}
    119120

  • libcfa/src/concurrency/stats.cfa

    rf93c50a r7e7a076  
    4848                        stats->io.calls.completed   = 0;
    4949                        stats->io.calls.errors.busy = 0;
    50                         stats->io.ops.sockread      = 0;
    51                         stats->io.ops.epllread      = 0;
    52                         stats->io.ops.sockwrite     = 0;
    53                         stats->io.ops.epllwrite     = 0;
    5450                #endif
    5551
    … …  
    108104                        tally_one( &cltr->io.calls.completed  , &proc->io.calls.completed   );
    109105                        tally_one( &cltr->io.calls.errors.busy, &proc->io.calls.errors.busy );
    110                         tally_one( &cltr->io.ops.sockread     , &proc->io.ops.sockread      );
    111                         tally_one( &cltr->io.ops.epllread     , &proc->io.ops.epllread      );
    112                         tally_one( &cltr->io.ops.sockwrite    , &proc->io.ops.sockwrite     );
    113                         tally_one( &cltr->io.ops.epllwrite    , &proc->io.ops.epllwrite     );
    114106                #endif
    115107        }
    … …  
    187179                                     | " - cmp " | eng3(io.calls.drain) | "/" | eng3(io.calls.completed) | "(" | ws(3, 3, avgcomp) | "/drain)"
    188180                                     | " - " | eng3(io.calls.errors.busy) | " EBUSY";
    189                                 sstr | "- ops blk: "
    190                                      |   " sk rd: " | eng3(io.ops.sockread)  | "epll: " | eng3(io.ops.epllread)
    191                                      |   " sk wr: " | eng3(io.ops.sockwrite) | "epll: " | eng3(io.ops.epllwrite);
    192181                                sstr | nl;
    193182                        }

  • libcfa/src/concurrency/stats.hfa

    rf93c50a r7e7a076  
    102102                                volatile uint64_t sleeps;
    103103                        } poller;
    104                         struct {
    105                                 volatile uint64_t sockread;
    106                                 volatile uint64_t epllread;
    107                                 volatile uint64_t sockwrite;
    108                                 volatile uint64_t epllwrite;
    109                         } ops;
    110104                };
    111105        #endif

  • libcfa/src/concurrency/thread.cfa

    rf93c50a r7e7a076  
    2525#include "invoke.h"
    2626
    27 uint64_t thread_rand();
    28 
    2927//-----------------------------------------------------------------------------
    3028// Thread ctors and dtors
    … …  
    3634        preempted = __NO_PREEMPTION;
    3735        corctx_flag = false;
     36        disable_interrupts();
     37        last_cpu = __kernel_getcpu();
     38        enable_interrupts();
    3839        curr_cor = &self_cor;
    3940        self_mon.owner = &this;
    … …  
    4344        link.next = 0p;
    4445        link.ts   = -1llu;
    45         preferred = ready_queue_new_preferred();
     46        preferred = -1u;
    4647        last_proc = 0p;
    4748        #if defined( __CFA_WITH_VERIFY__ )
    … …  
    140141        /* paranoid */ verify( this_thrd->context.SP );
    141142
    142         schedule_thread$( this_thrd, UNPARK_LOCAL );
     143        schedule_thread$( this_thrd );
    143144        enable_interrupts();
    144145}

  • libcfa/src/containers/string_res.cfa

    rf93c50a r7e7a076  
    2121
    2222
    23 
    24 
    25 
    26 
    27 
    28 
    29 // DON'T COMMIT:
    30 // #define VbyteDebug
    31 
    32 
    33 
    34 
    35 
    36 #ifdef VbyteDebug
    37 HandleNode *HeaderPtr;
     23#ifdef VbyteDebug
     24extern HandleNode *HeaderPtr;
    3825#endif // VbyteDebug
    3926
    … …  
    153140
    154141VbyteHeap HeapArea;
    155 
    156 VbyteHeap * DEBUG_string_heap = & HeapArea;
    157 
    158 size_t DEBUG_string_bytes_avail_until_gc( VbyteHeap * heap ) {
    159     return ((char*)heap->ExtVbyte) - heap->EndVbyte;
    160 }
    161 
    162 const char * DEBUG_string_heap_start( VbyteHeap * heap ) {
    163     return heap->StartVbyte;
    164 }
    165 
    166142
    167143// Returns the size of the string in bytes
    … …  
    249225void assign(string_res &this, const char* buffer, size_t bsize) {
    250226
    251     // traverse the incumbent share-edit set (SES) to recover the range of a base string to which `this` belongs
    252     string_res * shareEditSetStartPeer = & this;
    253     string_res * shareEditSetEndPeer = & this;
     227    char * afterBegin = this.Handle.s + this.Handle.lnth;
     228
     229    char * shareEditSetStart = this.Handle.s;
     230    char * shareEditSetEnd = afterBegin;
    254231    for (string_res * editPeer = this.shareEditSet_next; editPeer != &this; editPeer = editPeer->shareEditSet_next) {
    255         if ( editPeer->Handle.s < shareEditSetStartPeer->Handle.s ) {
    256             shareEditSetStartPeer = editPeer;
    257         }
    258         if ( shareEditSetEndPeer->Handle.s + shareEditSetEndPeer->Handle.lnth < editPeer->Handle.s + editPeer->Handle.lnth) {
    259             shareEditSetEndPeer = editPeer;
    260         }
    261     }
    262 
    263     // full string is from start of shareEditSetStartPeer thru end of shareEditSetEndPeer
    264     // `this` occurs in the middle of it, to be replaced
    265     // build up the new text in `pasting`
    266 
    267     string_res pasting = {
    268         shareEditSetStartPeer->Handle.s,                   // start of SES
    269         this.Handle.s - shareEditSetStartPeer->Handle.s }; // length of SES, before this
    270     append( pasting,
    271         buffer,                                            // start of replacement for this
    272         bsize );                                           // length of replacement for this
    273     append( pasting,
    274         this.Handle.s + this.Handle.lnth,                  // start of SES after this
    275         shareEditSetEndPeer->Handle.s + shareEditSetEndPeer->Handle.lnth -
    276         (this.Handle.s + this.Handle.lnth) );              // length of SES, after this
    277 
    278     // The above string building can trigger compaction.
    279     // The reference points (that are arguments of the string building) may move during that building.
    280     // From this point on, they are stable.
    281     // So now, capture their values for use in the overlap cases, below.
    282     // Do not factor these definitions with the arguments used above.
    283 
    284     char * beforeBegin = shareEditSetStartPeer->Handle.s;
    285     size_t beforeLen = this.Handle.s - beforeBegin;
    286 
    287     char * afterBegin = this.Handle.s + this.Handle.lnth;
    288     size_t afterLen = shareEditSetEndPeer->Handle.s + shareEditSetEndPeer->Handle.lnth - afterBegin;
     232        shareEditSetStart = min( shareEditSetStart, editPeer->Handle.s );
     233        shareEditSetEnd = max( shareEditSetStart, editPeer->Handle.s + editPeer->Handle.lnth);
     234    }
     235
     236    char * beforeBegin = shareEditSetStart;
     237    size_t beforeLen = this.Handle.s - shareEditSetStart;
     238    size_t afterLen = shareEditSetEnd - afterBegin;
     239
     240    string_res pasting = { beforeBegin, beforeLen };
     241    append(pasting, buffer, bsize);
     242    string_res after = { afterBegin, afterLen }; // juxtaposed with in-progress pasting
     243    pasting += after;                        // optimized case
    289244
    290245    size_t oldLnth = this.Handle.lnth;
    … …  
    298253    for (string_res * p = this.shareEditSet_next; p != &this; p = p->shareEditSet_next) {
    299254        assert (p->Handle.s >= beforeBegin);
    300         if ( p->Handle.s >= afterBegin ) {
    301             assert ( p->Handle.s <= afterBegin + afterLen );
    302             assert ( p->Handle.s + p->Handle.lnth <= afterBegin + afterLen );
    303             // p starts after the edit
    304             // take start and end as end-anchored
    305             size_t startOffsetFromEnd = afterBegin + afterLen - p->Handle.s;
    306             p->Handle.s = limit - startOffsetFromEnd;
    307             // p->Handle.lnth unaffected
    308         } else if ( p->Handle.s <= beforeBegin + beforeLen ) {
    309             // p starts before, or at the start of, the edit
    310             if ( p->Handle.s + p->Handle.lnth <= beforeBegin + beforeLen ) {
     255        if ( p->Handle.s < beforeBegin + beforeLen ) {
     256            // p starts before the edit
     257            if ( p->Handle.s + p->Handle.lnth < beforeBegin + beforeLen ) {
    311258                // p ends before the edit
    312259                // take end as start-anchored too
    313260                // p->Handle.lnth unaffected
    314261            } else if ( p->Handle.s + p->Handle.lnth < afterBegin ) {
    315                 // p ends during the edit; p does not include the last character replaced
     262                // p ends during the edit
    316263                // clip end of p to end at start of edit
    317264                p->Handle.lnth = beforeLen - ( p->Handle.s - beforeBegin );
    … …  
    327274            size_t startOffsetFromStart = p->Handle.s - beforeBegin;
    328275            p->Handle.s = pasting.Handle.s + startOffsetFromStart;
    329         } else {
    330             assert ( p->Handle.s < afterBegin );
     276        } else if ( p->Handle.s < afterBegin ) {
    331277            // p starts during the edit
    332278            assert( p->Handle.s + p->Handle.lnth >= beforeBegin + beforeLen );
    333279            if ( p->Handle.s + p->Handle.lnth < afterBegin ) {
    334                 // p ends during the edit; p does not include the last character replaced
     280                // p ends during the edit
    335281                // set p to empty string at start of edit
    336282                p->Handle.s = this.Handle.s;
    337283                p->Handle.lnth = 0;
    338284            } else {
    339                 // p includes the end of the edit
     285                // p ends after the edit
    340286                // clip start of p to start at end of edit
    341                 int charsToClip = afterBegin - p->Handle.s;
    342287                p->Handle.s = this.Handle.s + this.Handle.lnth;
    343                 p->Handle.lnth -= charsToClip;
     288                p->Handle.lnth += this.Handle.lnth;
     289                p->Handle.lnth -= oldLnth;
    344290            }
     291        } else {
     292            assert ( p->Handle.s <= afterBegin + afterLen );
     293            assert ( p->Handle.s + p->Handle.lnth <= afterBegin + afterLen );
     294            // p starts after the edit
     295            // take start and end as end-anchored
     296            size_t startOffsetFromEnd = afterBegin + afterLen - p->Handle.s;
     297            p->Handle.s = limit - startOffsetFromEnd;
     298            // p->Handle.lnth unaffected
    345299        }
    346300        MoveThisAfter( p->Handle, pasting.Handle );     // move substring handle to maintain sorted order by string position
    … …  
    687641    } // if
    688642#ifdef VbyteDebug
     643    serr | "exit:MoveThisAfter";
    689644    {
    690645        serr | "HandleList:";
    … …  
    695650                serr | n->s[i];
    696651            } // for
    697             serr | "\" flink:" | n->flink | " blink:" | n->blink | nl;
     652            serr | "\" flink:" | n->flink | " blink:" | n->blink;
    698653        } // for
    699654        serr | nlOn;
    700655    }
    701     serr | "exit:MoveThisAfter";
    702656#endif // VbyteDebug
    703657} // MoveThisAfter
    … …  
    708662
    709663//######################### VbyteHeap #########################
     664
     665#ifdef VbyteDebug
     666HandleNode *HeaderPtr = 0p;
     667#endif // VbyteDebug
    710668
    711669// Move characters from one location in the byte-string area to another. The routine handles the following situations:

  • libcfa/src/containers/string_res.hfa

    rf93c50a r7e7a076  
    3636void ?{}( HandleNode &, VbyteHeap & );          // constructor for nodes in the handle list
    3737void ^?{}( HandleNode & );                      // destructor for handle nodes
    38 
    39 extern VbyteHeap * DEBUG_string_heap;
    40 size_t DEBUG_string_bytes_avail_until_gc( VbyteHeap * heap );
    41 const char * DEBUG_string_heap_start( VbyteHeap * heap );
    4238
    4339

  • libcfa/src/fstream.cfa

    rf93c50a r7e7a076  
    1010// Created On       : Wed May 27 17:56:53 2015
    1111// Last Modified By : Peter A. Buhr
    12 // Last Modified On : Tue Sep 21 21:51:38 2021
    13 // Update Count     : 460
     12// Last Modified On : Thu Jul 29 22:34:10 2021
     13// Update Count     : 454
    1414//
    1515
    … …  
    2828#define IO_MSG "I/O error: "
    2929
    30 void ?{}( ofstream & os, void * file ) with(os) {
    31         file$ = file;
    32         sepDefault$ = true;
    33         sepOnOff$ = false;
    34         nlOnOff$ = true;
    35         prt$ = false;
    36         sawNL$ = false;
    37         acquired$ = false;
     30void ?{}( ofstream & os, void * file ) {
     31        os.file$ = file;
     32        os.sepDefault$ = true;
     33        os.sepOnOff$ = false;
     34        os.nlOnOff$ = true;
     35        os.prt$ = false;
     36        os.sawNL$ = false;
     37        os.acquired$ = false;
    3838        sepSetCur$( os, sepGet( os ) );
    3939        sepSet( os, " " );
    … …  
    124124void open( ofstream & os, const char name[], const char mode[] ) {
    125125        FILE * file = fopen( name, mode );
     126        // #ifdef __CFA_DEBUG__
    126127        if ( file == 0p ) {
    127128                throw (Open_Failure){ os };
    128129                // abort | IO_MSG "open output file \"" | name | "\"" | nl | strerror( errno );
    129130        } // if
    130         (os){ file };                                                                           // initialize
     131        // #endif // __CFA_DEBUG__
     132        (os){ file };
    131133} // open
    132134
    … …  
    135137} // open
    136138
    137 void close( ofstream & os ) with(os) {
    138   if ( (FILE *)(file$) == 0p ) return;
    139   if ( (FILE *)(file$) == (FILE *)stdout || (FILE *)(file$) == (FILE *)stderr ) return;
    140 
    141         if ( fclose( (FILE *)(file$) ) == EOF ) {
     139void close( ofstream & os ) {
     140  if ( (FILE *)(os.file$) == 0p ) return;
     141  if ( (FILE *)(os.file$) == (FILE *)stdout || (FILE *)(os.file$) == (FILE *)stderr ) return;
     142
     143        if ( fclose( (FILE *)(os.file$) ) == EOF ) {
    142144                throw (Close_Failure){ os };
    143145                // abort | IO_MSG "close output" | nl | strerror( errno );
    144146        } // if
    145         file$ = 0p;
     147        os.file$ = 0p;
    146148} // close
    147149
    … …  
    175177} // fmt
    176178
    177 inline void acquire( ofstream & os ) with(os) {
    178         lock( lock$ );                                                                          // may increase recursive lock
    179         if ( ! acquired$ ) acquired$ = true;                            // not locked ?
    180         else unlock( lock$ );                                                           // unwind recursive lock at start
     179inline void acquire( ofstream & os ) {
     180        lock( os.lock$ );
     181        if ( ! os.acquired$ ) os.acquired$ = true;
     182        else unlock( os.lock$ );
    181183} // acquire
    182184
    … …  
    185187} // release
    186188
    187 void ?{}( osacquire & acq, ofstream & os ) { lock( os.lock$ ); &acq.os = &os; }
     189inline void lock( ofstream & os ) { acquire( os ); }
     190inline void unlock( ofstream & os ) { release( os ); }
     191
     192void ?{}( osacquire & acq, ofstream & os ) { &acq.os = &os; lock( os.lock$ ); }
    188193void ^?{}( osacquire & acq ) { release( acq.os ); }
    189194
    … …  
    217222
    218223// private
    219 void ?{}( ifstream & is, void * file ) with(is) {
    220         file$ = file;
    221         nlOnOff$ = false;
    222         acquired$ = false;
     224void ?{}( ifstream & is, void * file ) {
     225        is.file$ = file;
     226        is.nlOnOff$ = false;
     227        is.acquired$ = false;
    223228} // ?{}
    224229
    … …  
    260265void open( ifstream & is, const char name[], const char mode[] ) {
    261266        FILE * file = fopen( name, mode );
     267        // #ifdef __CFA_DEBUG__
    262268        if ( file == 0p ) {
    263269                throw (Open_Failure){ is };
    264270                // abort | IO_MSG "open input file \"" | name | "\"" | nl | strerror( errno );
    265271        } // if
    266         (is){ file };                                                                           // initialize
     272        // #endif // __CFA_DEBUG__
     273        is.file$ = file;
    267274} // open
    268275
    … …  
    271278} // open
    272279
    273 void close( ifstream & is ) with(is) {
    274   if ( (FILE *)(file$) == 0p ) return;
    275   if ( (FILE *)(file$) == (FILE *)stdin ) return;
    276 
    277         if ( fclose( (FILE *)(file$) ) == EOF ) {
     280void close( ifstream & is ) {
     281  if ( (FILE *)(is.file$) == 0p ) return;
     282  if ( (FILE *)(is.file$) == (FILE *)stdin ) return;
     283
     284        if ( fclose( (FILE *)(is.file$) ) == EOF ) {
    278285                throw (Close_Failure){ is };
    279286                // abort | IO_MSG "close input" | nl | strerror( errno );
    280287        } // if
    281         file$ = 0p;
     288        is.file$ = 0p;
    282289} // close
    283290
    … …  
    320327} // fmt
    321328
    322 inline void acquire( ifstream & is ) with(is) {
    323         lock( lock$ );                                                                          // may increase recursive lock
    324         if ( ! acquired$ ) acquired$ = true;                            // not locked ?
    325         else unlock( lock$ );                                                           // unwind recursive lock at start
     329inline void acquire( ifstream & is ) {
     330        lock( is.lock$ );
     331        if ( ! is.acquired$ ) is.acquired$ = true;
     332        else unlock( is.lock$ );
    326333} // acquire
    327334
    … …  
    330337} // release
    331338
    332 void ?{}( isacquire & acq, ifstream & is ) { lock( is.lock$ ); &acq.is = &is; }
     339void ?{}( isacquire & acq, ifstream & is ) { &acq.is = &is; lock( is.lock$ ); }
    333340void ^?{}( isacquire & acq ) { release( acq.is ); }
    334341
    … …  
    343350
    344351// exception I/O constructors
    345 void ?{}( Open_Failure & ex, ofstream & ostream ) with(ex) {
    346         virtual_table = &Open_Failure_vt;
    347         ostream = &ostream;
    348         tag = 1;
    349 } // ?{}
    350 
    351 void ?{}( Open_Failure & ex, ifstream & istream ) with(ex) {
    352         virtual_table = &Open_Failure_vt;
    353         istream = &istream;
    354         tag = 0;
     352void ?{}( Open_Failure & this, ofstream & ostream ) {
     353        this.virtual_table = &Open_Failure_vt;
     354        this.ostream = &ostream;
     355        this.tag = 1;
     356} // ?{}
     357
     358void ?{}( Open_Failure & this, ifstream & istream ) {
     359        this.virtual_table = &Open_Failure_vt;
     360        this.istream = &istream;
     361        this.tag = 0;
    355362} // ?{}
    356363
    … …  
    359366
    360367// exception I/O constructors
    361 void ?{}( Close_Failure & ex, ofstream & ostream ) with(ex) {
    362         virtual_table = &Close_Failure_vt;
    363         ostream = &ostream;
    364         tag = 1;
    365 } // ?{}
    366 
    367 void ?{}( Close_Failure & ex, ifstream & istream ) with(ex) {
    368         virtual_table = &Close_Failure_vt;
    369         istream = &istream;
    370         tag = 0;
     368void ?{}( Close_Failure & this, ofstream & ostream ) {
     369        this.virtual_table = &Close_Failure_vt;
     370        this.ostream = &ostream;
     371        this.tag = 1;
     372} // ?{}
     373
     374void ?{}( Close_Failure & this, ifstream & istream ) {
     375        this.virtual_table = &Close_Failure_vt;
     376        this.istream = &istream;
     377        this.tag = 0;
    371378} // ?{}
    372379
    … …  
    375382
    376383// exception I/O constructors
    377 void ?{}( Write_Failure & ex, ofstream & ostream ) with(ex) {
    378         virtual_table = &Write_Failure_vt;
    379         ostream = &ostream;
    380         tag = 1;
    381 } // ?{}
    382 
    383 void ?{}( Write_Failure & ex, ifstream & istream ) with(ex) {
    384         virtual_table = &Write_Failure_vt;
    385         istream = &istream;
    386         tag = 0;
     384void ?{}( Write_Failure & this, ofstream & ostream ) {
     385        this.virtual_table = &Write_Failure_vt;
     386        this.ostream = &ostream;
     387        this.tag = 1;
     388} // ?{}
     389
     390void ?{}( Write_Failure & this, ifstream & istream ) {
     391        this.virtual_table = &Write_Failure_vt;
     392        this.istream = &istream;
     393        this.tag = 0;
    387394} // ?{}
    388395
    … …  
    391398
    392399// exception I/O constructors
    393 void ?{}( Read_Failure & ex, ofstream & ostream ) with(ex) {
    394         virtual_table = &Read_Failure_vt;
    395         ostream = &ostream;
    396         tag = 1;
    397 } // ?{}
    398 
    399 void ?{}( Read_Failure & ex, ifstream & istream ) with(ex) {
    400         virtual_table = &Read_Failure_vt;
    401         istream = &istream;
    402         tag = 0;
     400void ?{}( Read_Failure & this, ofstream & ostream ) {
     401        this.virtual_table = &Read_Failure_vt;
     402        this.ostream = &ostream;
     403        this.tag = 1;
     404} // ?{}
     405
     406void ?{}( Read_Failure & this, ifstream & istream ) {
     407        this.virtual_table = &Read_Failure_vt;
     408        this.istream = &istream;
     409        this.tag = 0;
    403410} // ?{}
    404411

  • tools/perf/process_stat_array.py

    rf93c50a r7e7a076  
    11#!/usr/bin/python3
    22
    3 import argparse, json, math, os, sys, re
    4 from PIL import Image
    5 import numpy as np
     3import argparse, os, sys, re
    64
    75def dir_path(string):
    … …  
    1311parser = argparse.ArgumentParser()
    1412parser.add_argument('--path', type=dir_path, default=".cfadata", help= 'paste path to biog.txt file')
    15 parser.add_argument('--out', type=argparse.FileType('w'), default=sys.stdout)
    1613
    1714try :
    … …  
    2623counters = {}
    2724
    28 max_cpu = 0
    29 min_cpu = 1000000
    30 max_tsc = 0
    31 min_tsc = 18446744073709551615
    32 
    3325#open the files
    3426for filename in filenames:
    … …  
    3931                with open(os.path.join(root, filename), 'r') as file:
    4032                        for line in file:
    41                                 raw = [int(x.strip()) for x in line.split(',')]
    42 
    43                                 ## from/to
    44                                 high = (raw[1] >> 32)
    45                                 low  = (raw[1] & 0xffffffff)
    46                                 data = [me, raw[0], high, low]
    47                                 max_cpu = max(max_cpu, high, low)
    48                                 min_cpu = min(min_cpu, high, low)
    49 
    50                                 ## number
    51                                 # high = (raw[1] >> 8)
    52                                 # low  = (raw[1] & 0xff)
    53                                 # data = [me, raw[0], high, low]
    54                                 # max_cpu = max(max_cpu, low)
    55                                 # min_cpu = min(min_cpu, low)
    56 
    57 
    58                                 max_tsc = max(max_tsc, raw[0])
    59                                 min_tsc = min(min_tsc, raw[0])
     33                                # data = [int(x.strip()) for x in line.split(',')]
     34                                data = [int(line.strip())]
     35                                data = [me, *data]
    6036                                merged.append(data)
    6137
    62         except Exception as e:
    63                 print(e)
     38        except:
    6439                pass
    6540
    66 
    67 print({"max-cpu": max_cpu, "min-cpu": min_cpu, "max-tsc": max_tsc, "min-tsc": min_tsc})
    6841
    6942# Sort by timestamp (the second element)
    … …  
    7447merged.sort(key=takeSecond)
    7548
    76 json.dump({"values":merged, "max-cpu": max_cpu, "min-cpu": min_cpu, "max-tsc": max_tsc, "min-tsc": min_tsc}, args.out)
     49# for m in merged:
     50#       print(m)
    7751
    78 # vmin = merged[ 0][1]
    79 # vmax = float(merged[-1][1] - vmin) / 2500000000.0
    80 # # print(vmax)
     52single = []
     53curr = 0
    8154
    82 # bins = []
    83 # for _ in range(0, int(math.ceil(vmax * 10))):
    84 #       bins.append([0] * (32 * 32))
     55# merge the data
     56# for (me, time, value) in merged:
     57for (me, value) in merged:
     58        # check now much this changes
     59        old = counters[me]
     60        change = value - old
     61        counters[me] = value
    8562
    86 # # print(len(bins))
    87 # bins = np.array(bins)
     63        # add change to the current
     64        curr = curr + change
     65        single.append( value )
    8866
    89 # rejected = 0
    90 # highest  = 0
     67        pass
    9168
    92 # for x in merged:
    93 #       b = int(float(x[1] - vmin) / 250000000.0)
    94 #       from_ = x[2]
    95 #       if from_ < 0 or from_ > 32:
    96 #               rejected += 1
    97 #               continue;
    98 #       to_   = x[3]
    99 #       if to_ < 0 or to_ > 32:
    100 #               rejected += 1
    101 #               continue;
    102 #       idx = (to_ * 32) + from_
    103 #       bins[b][idx] = bins[b][idx] + 1
    104 #       highest = max(highest, bins[b][idx])
    105 
    106 # bins = np.array(map(lambda x: np.int8(x * 255.0 / float(highest)), bins))
    107 
    108 # print([highest, rejected])
    109 # print(bins.shape)
    110 
    111 # im = Image.fromarray(bins)
    112 # im.save('test.png')
    113 
    114 # vmax = merged[-1][1]
    115 
    116 # diff = float(vmax - vmin) / 2500000000.0
    117 # print([vmin, vmax])
    118 # print([vmax - vmin, diff])
    119 
    120 # print(len(merged))
    121 
    122 # for b in bins:
    123 #       print(b)
    124 
    125 # single = []
    126 # curr = 0
    127 
    128 # # merge the data
    129 # # for (me, time, value) in merged:
    130 # for (me, value) in merged:
    131 #       # check now much this changes
    132 #       old = counters[me]
    133 #       change = value - old
    134 #       counters[me] = value
    135 
    136 #       # add change to the current
    137 #       curr = curr + change
    138 #       single.append( value )
    139 
    140 #       pass
    141 
    142 # print(single)
     69print(single)
    14370
    14471# single = sorted(single)[:len(single)-100]
Note: See TracChangeset for help on using the changeset viewer.