fluencelabs/musl
1
0
Fork 0
mirror of https://github.com/fluencelabs/musl synced 2025-04-26 15:52:13 +00:00
Code Issues Projects Releases Wiki Activity
musl/src/thread/pthread_create.c

307 lines
8.5 KiB
C
Raw Normal View History

use symbolic names for clone flags in pthread_create
2013-09-16 10:54:31 -04:00
#define _GNU_SOURCE
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
#include "pthread_impl.h"
add proper fuxed-based locking for stdio previously, stdio used spinlocks, which would be unacceptable if we ever add support for thread priorities, and which yielded pathologically bad performance if an application attempted to use flockfile on a key file as a major/primary locking mechanism. i had held off on making this change for fear that it would hurt performance in the non-threaded case, but actually support for recursive locking had already inflicted that cost. by having the internal locking functions store a flag indicating whether they need to perform unlocking, rather than using the actual recursive lock counter, i was able to combine the conditionals at unlock time, eliminating any additional cost, and also avoid a nasty corner case where a huge number of calls to ftrylockfile could cause deadlock later at the point of internal locking. this commit also fixes some issues with usage of pthread_self conflicting with __attribute__((const)) which resulted in crashes with some compiler versions/optimizations, mainly in flockfile prior to pthread_create.
2011-07-30 08:02:14 -04:00
#include "stdio_impl.h"
support configurable page size on mips, powerpc and microblaze PAGE_SIZE was hardcoded to 4096, which is historically what most systems use, but on several archs it is a kernel config parameter, user space can only know it at execution time from the aux vector. PAGE_SIZE and PAGESIZE are not defined on archs where page size is a runtime parameter, applications should use sysconf(_SC_PAGE_SIZE) to query it. Internally libc code defines PAGE_SIZE to libc.page_size, which is set to aux[AT_PAGESZ] in __init_libc and early in __dynlink as well. (Note that libc.page_size can be accessed without GOT, ie. before relocations are done) Some fpathconf settings are hardcoded to 4096, these should be actually queried from the filesystem using statfs.
2013-09-15 02:00:32 +00:00
#include "libc.h"
clean up sloppy nested inclusion from pthread_impl.h this mirrors the stdio_impl.h cleanup. one header which is not strictly needed, errno.h, is left in pthread_impl.h, because since pthread functions return their error codes rather than using errno, nearly every single pthread function needs the errno constants. in a few places, rather than bringing in string.h to use memset, the memset was replaced by direct assignment. this seems to generate much better code anyway, and makes many functions which were previously non-leaf functions into leaf functions (possibly eliminating a great deal of bloat on some platforms where non-leaf functions require ugly prologue and/or epilogue).
2012-11-08 17:04:20 -05:00
#include <sys/mman.h>
fix use of uninitialized memory with application-provided thread stacks the subsequent code in pthread_create and the code which copies TLS initialization images to the new thread's TLS space assume that the memory provided to them is zero-initialized, which is true when it's obtained by pthread_create using mmap. however, when the caller provides a stack using pthread_attr_setstack, pthread_create cannot make any assumptions about the contents. simply zero-filling the relevant memory in this case is the simplest and safest fix.
2014-08-22 14:05:10 -04:00
#include <string.h>
process robust list in pthread_exit to fix detached thread use-after-unmap the robust list head lies in the thread structure, which is unmapped before exit for detached threads. this leaves the kernel unable to process the exiting thread's robust list, and with a dangling pointer which may happen to point to new unrelated data at the time the kernel processes it. userspace processing of the robust list was already needed for non-pshared robust mutexes in order to perform private futex wakes rather than the shared ones the kernel would do, but it was conditional on linking pthread_mutexattr_setrobust and did not bother processing the pshared mutexes in the list, which requires additional logic for the robust list pending slot in case pthread_exit is interrupted by asynchronous process termination. the new robust list processing code is linked unconditionally (inlined in pthread_exit), handles both private and shared mutexes, and also removes the kernel's reference to the robust list before unmapping and exit if the exiting thread is detached.
2015-04-10 00:26:34 -04:00
#include <stddef.h>
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
use weak symbols for the POSIX functions that will be used by C threads The intent of this is to avoid name space pollution of the C threads implementation. This has two sides to it. First we have to provide symbols that wouldn't pollute the name space for the C threads implementation. Second we have to clean up some internal uses of POSIX functions such that they don't implicitly drag in such symbols.
2014-09-01 00:46:23 +02:00
void *__mmap(void *, size_t, int, int, int, off_t);
int __munmap(void *, size_t);
int __mprotect(void *, size_t, int);
process robust list in pthread_exit to fix detached thread use-after-unmap the robust list head lies in the thread structure, which is unmapped before exit for detached threads. this leaves the kernel unable to process the exiting thread's robust list, and with a dangling pointer which may happen to point to new unrelated data at the time the kernel processes it. userspace processing of the robust list was already needed for non-pshared robust mutexes in order to perform private futex wakes rather than the shared ones the kernel would do, but it was conditional on linking pthread_mutexattr_setrobust and did not bother processing the pshared mutexes in the list, which requires additional logic for the robust list pending slot in case pthread_exit is interrupted by asynchronous process termination. the new robust list processing code is linked unconditionally (inlined in pthread_exit), handles both private and shared mutexes, and also removes the kernel's reference to the robust list before unmapping and exit if the exiting thread is detached.
2015-04-10 00:26:34 -04:00
void __vm_lock_impl(int);
void __vm_unlock_impl(void);
use weak symbols for the POSIX functions that will be used by C threads The intent of this is to avoid name space pollution of the C threads implementation. This has two sides to it. First we have to provide symbols that wouldn't pollute the name space for the C threads implementation. Second we have to clean up some internal uses of POSIX functions such that they don't implicitly drag in such symbols.
2014-09-01 00:46:23 +02:00
move rsyscall out of pthread_create module this is something of a tradeoff, as now set*id() functions, rather than pthread_create, are what pull in the code overhead for dealing with linux's refusal to implement proper POSIX thread-vs-process semantics. my motivations are: 1. it's cleaner this way, especially cleaner to optimize out the rsyscall locking overhead from pthread_create when it's not needed. 2. it's expected that only a tiny number of core system programs will ever use set*id() functions, whereas many programs may want to use threads, and making thread overhead tiny is an incentive for "light" programs to try threads.
2011-04-06 20:27:07 -04:00
static void dummy_0()
{
}
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 11:51:50 -04:00
weak_alias(dummy_0, __acquire_ptc);
weak_alias(dummy_0, __release_ptc);
move some more code out of pthread_create.c this also de-uglifies the dummy function aliasing a bit.
2011-04-19 23:09:14 -04:00
weak_alias(dummy_0, __pthread_tsd_run_dtors);
fix false ownership of stdio FILEs due to tid reuse this is analogous commit fffc5cda10e0c5c910b40f7be0d4fa4e15bb3f48 which fixed the corresponding issue for mutexes. the robust list can't be used here because the locks do not share a common layout with mutexes. at some point it may make sense to simply incorporate a mutex object into the FILE structure and use it, but that would be a much more invasive change, and it doesn't mesh well with the current design that uses a simpler code path for internal locking and pulls in the recursive-mutex-like code when the flockfile API is used explicitly.
2014-08-23 23:35:10 -04:00
weak_alias(dummy_0, __do_orphaned_stdio_locks);
omit pthread tsd dtor code if tsd is not used
2011-04-03 02:33:50 -04:00
use weak symbols for the POSIX functions that will be used by C threads The intent of this is to avoid name space pollution of the C threads implementation. This has two sides to it. First we have to provide symbols that wouldn't pollute the name space for the C threads implementation. Second we have to clean up some internal uses of POSIX functions such that they don't implicitly drag in such symbols.
2014-09-01 00:46:23 +02:00
_Noreturn void __pthread_exit(void *result)
reorganize thread exit code, make pthread_exit call cancellation handlers (pt2)
2011-02-13 19:58:30 -05:00
{
replace all remaining internal uses of pthread_self with __pthread_self prior to version 1.1.0, the difference between pthread_self (the public function) and __pthread_self (the internal macro or inline function) was that the former would lazily initialize the thread pointer if it was not already initialized, whereas the latter would crash in this case. since lazy initialization is no longer supported, use of pthread_self no longer makes sense; it simply generates larger, slower code.
2014-06-10 04:02:40 -04:00
pthread_t self = __pthread_self();
make last thread's pthread_exit give exit(0) a consistent state the previous few commits ended up leaving the thread count and signal mask wrong for atexit handlers and stdio cleanup.
2013-04-26 16:16:04 -04:00
sigset_t set;
reorganize thread exit code, make pthread_exit call cancellation handlers (pt2)
2011-02-13 19:58:30 -05:00
make pthread_exit responsible for disabling cancellation this requirement is tucked away in XSH 2.9.5 Thread Cancellation under the heading Thread Cancellation Cleanup Handlers.
2015-02-16 22:25:50 -05:00
self->canceldisable = 1;
self->cancelasync = 0;
replace bad cancellation cleanup abi with a sane one the old abi was intended to duplicate glibc's abi at the expense of being ugly and slow, but it turns out glib was not even using that abi except on non-gcc-compatible compilers (which it doesn't even support) and was instead using an exceptions-in-c/unwind-based approach whose abi we could not duplicate anyway without nasty dwarf2/unwind integration. the new abi is copied from a very old glibc abi, which seems to still be supported/present in current glibc. it avoids all unwinding, whether by sjlj or exceptions, and merely maintains a linked list of cleanup functions to be called from the context of pthread_exit. i've made some care to ensure that longjmp out of a cleanup function should work, even though it is not required to. this change breaks abi compatibility with programs which were using pthread cancellation, which is unfortunate, but that's why i'm making the change now rather than later. considering that most pthread features have not been usable until recently anyway, i don't see it as a major issue at this point.
2012-02-09 02:33:08 -05:00
self->result = result;
while (self->cancelbuf) {
void (*f)(void *) = self->cancelbuf->__f;
void *x = self->cancelbuf->__x;
self->cancelbuf = self->cancelbuf->__next;
f(x);
fix pthread_exit from cancellation handler cancellation frames were not correctly popped, so this usage would not only loop, but also reuse discarded and invalid parts of the stack.
2011-04-17 17:06:05 -04:00
}
reorganize thread exit code, make pthread_exit call cancellation handlers (pt2)
2011-02-13 19:58:30 -05:00
move some more code out of pthread_create.c this also de-uglifies the dummy function aliasing a bit.
2011-04-19 23:09:14 -04:00
__pthread_tsd_run_dtors();
reorganize thread exit code, make pthread_exit call cancellation handlers (pt2)
2011-02-13 19:58:30 -05:00
fix several locks that weren't updated right for new futex-based __lock these could have caused memory corruption due to invalid accesses to the next field. all should be fixed now; I found the errors with fgrep -r '__lock(&', which is bogus since the argument should be an array.
2012-07-12 11:23:43 -04:00
__lock(self->exitlock);
run dtors before taking the exit-lock in pthread exit previously a long-running dtor could cause pthread_detach to block.
2011-06-14 01:25:17 -04:00
optimize pthread termination in the non-detached case we can avoid blocking signals by simply using a flag to mark that the thread has exited and prevent it from getting counted in the rsyscall signal-pingpong. this restores the original pthread create/join throughput from before the sigprocmask call was added.
2011-03-10 18:31:37 -05:00
/* Mark this thread dead before decrementing count */
fix several locks that weren't updated right for new futex-based __lock these could have caused memory corruption due to invalid accesses to the next field. all should be fixed now; I found the errors with fgrep -r '__lock(&', which is bogus since the argument should be an array.
2012-07-12 11:23:43 -04:00
__lock(self->killlock);
optimize pthread termination in the non-detached case we can avoid blocking signals by simply using a flag to mark that the thread has exited and prevent it from getting counted in the rsyscall signal-pingpong. this restores the original pthread create/join throughput from before the sigprocmask call was added.
2011-03-10 18:31:37 -05:00
self->dead = 1;
race condition fix: block all signals before decrementing thread count the existence of a (kernelspace) thread must never have observable effects after the thread count is decremented. if signals are not blocked, it could end up handling the signal for rsyscall and contributing towards the count of threads which have changed ids, causing a thread to be missed. this could lead to one thread retaining unwanted privilege level. this change may also address other subtle race conditions in application code that uses signals.
2011-02-19 11:04:36 -05:00
add comments on some of the pthread_exit logic
2013-04-26 16:04:30 -04:00
/* Block all signals before decrementing the live thread count.
* This is important to ensure that dynamically allocated TLS
* is not under-allocated/over-committed, and possibly for other
* reasons as well. */
transition to using functions for internal signal blocking/restoring there are several reasons for this change. one is getting rid of the repetition of the syscall signature all over the place. another is sharing the constant masks without costly GOT accesses in PIC. the main motivation, however, is accurately representing whether we want to block signals that might be handled by the application, or all signals.
2013-04-26 19:48:01 -04:00
__block_all_sigs(&set);
always block signals in pthread_exit before decrementing thread count the thread count (1+libc.threads_minus_1) must always be greater than or equal to the number of threads which could have application code running, even in an async-signal-safe sense. there is at least one dangerous race condition if this invariant fails to hold: dlopen could allocate too little TLS for existing threads, and a signal handler running in the exiting thread could claim the allocated TLS for itself (via __tls_get_addr), leaving too little for the other threads it was allocated for and thereby causing out-of-bounds access. there may be other situations where it's dangerous for the thread count to be too low, particularly in the case where only one thread should be left, in which case locking may be omitted. however, all such code paths seem to arise from undefined behavior, since async-signal-unsafe functions are not permitted to be called from a signal handler that interrupts pthread_exit (which is itself async-signal-unsafe). this change may also simplify logic in __synccall and improve the chances of making __synccall async-signal-safe.
2013-04-26 15:47:44 -04:00
prevent code from running under a thread id which already gave ESRCH
2013-04-26 17:51:22 -04:00
/* Wait to unlock the kill lock, which governs functions like
* pthread_kill which target a thread id, until signals have
* been blocked. This precludes observation of the thread id
* as a live thread (with application code running in it) after
* the thread was reported dead by ESRCH being returned. */
__unlock(self->killlock);
make last thread's pthread_exit give exit(0) a consistent state the previous few commits ended up leaving the thread count and signal mask wrong for atexit handlers and stdio cleanup.
2013-04-26 16:16:04 -04:00
/* It's impossible to determine whether this is "the last thread"
* until performing the atomic decrement, since multiple threads
* could exit at the same time. For the last thread, revert the
* decrement and unblock signals to give the atexit handlers and
* stdio cleanup code a consistent state. */
if (a_fetch_add(&libc.threads_minus_1, -1)==0) {
libc.threads_minus_1 = 0;
transition to using functions for internal signal blocking/restoring there are several reasons for this change. one is getting rid of the repetition of the syscall signature all over the place. another is sharing the constant masks without costly GOT accesses in PIC. the main motivation, however, is accurately representing whether we want to block signals that might be handled by the application, or all signals.
2013-04-26 19:48:01 -04:00
__restore_sigs(&set);
make last thread's pthread_exit give exit(0) a consistent state the previous few commits ended up leaving the thread count and signal mask wrong for atexit handlers and stdio cleanup.
2013-04-26 16:16:04 -04:00
exit(0);
}
make pthread_exit run dtors for last thread, wait to decrement thread count
2011-02-19 10:38:57 -05:00
add locale framework this commit adds non-stub implementations of setlocale, duplocale, newlocale, and uselocale, along with the data structures and minimal code needed for representing the active locale on a per-thread basis and optimizing the common case where thread-local locale settings are not in use. at this point, the data structures only contain what is necessary to represent LC_CTYPE (a single flag) and LC_MESSAGES (a name for use in finding message translation files). representation for the other categories will be added later; the expectation is that a single pointer will suffice for each. for LC_CTYPE, the strings "C" and "POSIX" are treated as special; any other string is accepted and treated as "C.UTF-8". for other categories, any string is accepted after being truncated to a maximum supported length (currently 15 bytes). for LC_MESSAGES, the name is kept regardless of whether libc itself can use such a message translation locale, since applications using catgets or gettext should be able to use message locales libc is not aware of. for other categories, names which are not successfully loaded as locales (which, at present, means all names) are treated as aliases for "C". setlocale never fails. locale settings are not yet used anywhere, so this commit should have no visible effects except for the contents of the string returned by setlocale.
2014-07-02 19:33:19 -04:00
if (self->locale != &libc.global_locale) {
a_dec(&libc.uselocale_cnt);
if (self->locale->ctype_utf8)
a_dec(&libc.bytelocale_cnt_minus_1);
}
process robust list in pthread_exit to fix detached thread use-after-unmap the robust list head lies in the thread structure, which is unmapped before exit for detached threads. this leaves the kernel unable to process the exiting thread's robust list, and with a dangling pointer which may happen to point to new unrelated data at the time the kernel processes it. userspace processing of the robust list was already needed for non-pshared robust mutexes in order to perform private futex wakes rather than the shared ones the kernel would do, but it was conditional on linking pthread_mutexattr_setrobust and did not bother processing the pshared mutexes in the list, which requires additional logic for the robust list pending slot in case pthread_exit is interrupted by asynchronous process termination. the new robust list processing code is linked unconditionally (inlined in pthread_exit), handles both private and shared mutexes, and also removes the kernel's reference to the robust list before unmapping and exit if the exiting thread is detached.
2015-04-10 00:26:34 -04:00
/* Process robust list in userspace to handle non-pshared mutexes
* and the detached thread case where the robust list head will
* be invalid when the kernel would process it. */
__vm_lock_impl(+1);
volatile void *volatile *rp;
while ((rp=self->robust_list.head) && rp != &self->robust_list.head) {
pthread_mutex_t *m = (void *)((char *)rp
- offsetof(pthread_mutex_t, _m_next));
int waiters = m->_m_waiters;
int priv = (m->_m_type & 128) ^ 128;
self->robust_list.pending = rp;
self->robust_list.head = *rp;
int cont = a_swap(&m->_m_lock, self->tid|0x40000000);
self->robust_list.pending = 0;
if (cont < 0 || waiters)
__wake(&m->_m_lock, 1, priv);
}
__vm_unlock_impl();
fix false ownership of stdio FILEs due to tid reuse this is analogous commit fffc5cda10e0c5c910b40f7be0d4fa4e15bb3f48 which fixed the corresponding issue for mutexes. the robust list can't be used here because the locks do not share a common layout with mutexes. at some point it may make sense to simply incorporate a mutex object into the FILE structure and use it, but that would be a much more invasive change, and it doesn't mesh well with the current design that uses a simpler code path for internal locking and pulls in the recursive-mutex-like code when the flockfile API is used explicitly.
2014-08-23 23:35:10 -04:00
__do_orphaned_stdio_locks();
enable private futex for process-local robust mutexes the kernel always uses non-private wake when walking the robust list when a thread or process exits, so it's not able to wake waiters listening with the private futex flag. this problem is solved by doing the equivalent in userspace as the last step of pthread_exit. care is taken to remove mutexes from the robust list before unlocking them so that the kernel will not attempt to access them again, possibly after another thread locks them. this removal code can treat the list as singly-linked, since no further code which would add or remove items is able to run at this point. moreover, the pending pointer is not needed since the mutexes being unlocked are all process-local; in the case of asynchronous process termination, they all cease to exist. since a process-local robust mutex cannot come into existence without a call to pthread_mutexattr_setrobust in the same process, the code for userspace robust list processing is put in that source file, and a weak alias to a dummy function is used to avoid pulling in this bloat as part of pthread_exit in static-linked programs.
2014-08-16 02:28:34 -04:00
optimize pthread termination in the non-detached case we can avoid blocking signals by simply using a flag to mark that the thread has exited and prevent it from getting counted in the rsyscall signal-pingpong. this restores the original pthread create/join throughput from before the sigprocmask call was added.
2011-03-10 18:31:37 -05:00
if (self->detached && self->map_base) {
add comments on some of the pthread_exit logic
2013-04-26 16:04:30 -04:00
/* Detached threads must avoid the kernel clear_child_tid
* feature, since the virtual address will have been
* unmapped and possibly already reused by a new mapping
* at the time the kernel would perform the write. In
* the case of threads that started out detached, the
* initial clone flags are correct, but if the thread was
* detached later (== 2), we need to clear it here. */
if (self->detached == 2) __syscall(SYS_set_tid_address, 0);
process robust list in pthread_exit to fix detached thread use-after-unmap the robust list head lies in the thread structure, which is unmapped before exit for detached threads. this leaves the kernel unable to process the exiting thread's robust list, and with a dangling pointer which may happen to point to new unrelated data at the time the kernel processes it. userspace processing of the robust list was already needed for non-pshared robust mutexes in order to perform private futex wakes rather than the shared ones the kernel would do, but it was conditional on linking pthread_mutexattr_setrobust and did not bother processing the pshared mutexes in the list, which requires additional logic for the robust list pending slot in case pthread_exit is interrupted by asynchronous process termination. the new robust list processing code is linked unconditionally (inlined in pthread_exit), handles both private and shared mutexes, and also removes the kernel's reference to the robust list before unmapping and exit if the exiting thread is detached.
2015-04-10 00:26:34 -04:00
/* Robust list will no longer be valid, and was already
* processed above, so unregister it with the kernel. */
if (self->robust_list.off)
__syscall(SYS_set_robust_list, 0, 3*sizeof(long));
add comments on some of the pthread_exit logic
2013-04-26 16:04:30 -04:00
/* The following call unmaps the thread's stack mapping
* and then exits without touching the stack. */
reorganize thread exit code, make pthread_exit call cancellation handlers (pt2)
2011-02-13 19:58:30 -05:00
__unmapself(self->map_base, self->map_size);
optimize pthread termination in the non-detached case we can avoid blocking signals by simply using a flag to mark that the thread has exited and prevent it from getting counted in the rsyscall signal-pingpong. this restores the original pthread create/join throughput from before the sigprocmask call was added.
2011-03-10 18:31:37 -05:00
}
reorganize thread exit code, make pthread_exit call cancellation handlers (pt2)
2011-02-13 19:58:30 -05:00
further use of _Noreturn, for non-plain-C functions note that POSIX does not specify these functions as _Noreturn, because POSIX is aligned with C99, not the new C11 standard. when POSIX is eventually updated to C11, it will almost surely give these functions the _Noreturn attribute. for now, the actual _Noreturn keyword is not used anyway when compiling with a c99 compiler, which is what POSIX requires; the GCC __attribute__ is used instead if it's available, however. in a few places, I've added infinite for loops at the end of _Noreturn functions to silence compiler warnings. presumably __buildin_unreachable could achieve the same thing, but it would only work on newer GCCs and would not be portable. the loops should have near-zero code size cost anyway. like the previous _Noreturn commit, this one is based on patches contributed by philomath.
2012-09-06 23:34:10 -04:00
for (;;) __syscall(SYS_exit, 0);
reorganize thread exit code, make pthread_exit call cancellation handlers (pt2)
2011-02-13 19:58:30 -05:00
}
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
simplify cancellation push/pop slightly no need to pass unnecessary extra arguments on to the core code in pthread_create.c. this just wastes cycles and code bloat.
2012-05-23 14:13:54 -04:00
void __do_cleanup_push(struct __ptcb *cb)
further debloat cancellation handlers cleanup push and pop are also no-ops if pthread_exit is not reachable. this can make a big difference for library code which needs to protect itself against cancellation, but which is unlikely to actually be used in programs with threads/cancellation.
2011-08-03 19:57:46 -04:00
{
always initialize thread pointer at program start this is the first step in an overhaul aimed at greatly simplifying and optimizing everything dealing with thread-local state. previously, the thread pointer was initialized lazily on first access, or at program startup if stack protector was in use, or at certain random places where inconsistent state could be reached if it were not initialized early. while believed to be fully correct, the logic was fragile and non-obvious. in the first phase of the thread pointer overhaul, support is retained (and in some cases improved) for systems/situation where loading the thread pointer fails, e.g. old kernels. some notes on specific changes: - the confusing use of libc.main_thread as an indicator that the thread pointer is initialized is eliminated in favor of an explicit has_thread_pointer predicate. - sigaction no longer needs to ensure that the thread pointer is initialized before installing a signal handler (this was needed to prevent a situation where the signal handler caused the thread pointer to be initialized and the subsequent sigreturn cleared it again) but it still needs to ensure that implementation-internal thread-related signals are not blocked. - pthread tsd initialization for the main thread is deferred in a new manner to minimize bloat in the static-linked __init_tp code. - pthread_setcancelstate no longer needs special handling for the situation before the thread pointer is initialized. it simply fails on systems that cannot support a thread pointer, which are non-conforming anyway. - pthread_cleanup_push/pop now check for missing thread pointer and nop themselves out in this case, so stdio no longer needs to avoid the cancellable path when the thread pointer is not available. a number of cases remain where certain interfaces may crash if the system does not support a thread pointer. at this point, these should be limited to pthread interfaces, and the number of such cases should be fewer than before.
2014-03-24 16:57:11 -04:00
if (!libc.has_thread_pointer) return;
replace all remaining internal uses of pthread_self with __pthread_self prior to version 1.1.0, the difference between pthread_self (the public function) and __pthread_self (the internal macro or inline function) was that the former would lazily initialize the thread pointer if it was not already initialized, whereas the latter would crash in this case. since lazy initialization is no longer supported, use of pthread_self no longer makes sense; it simply generates larger, slower code.
2014-06-10 04:02:40 -04:00
struct pthread *self = __pthread_self();
further debloat cancellation handlers cleanup push and pop are also no-ops if pthread_exit is not reachable. this can make a big difference for library code which needs to protect itself against cancellation, but which is unlikely to actually be used in programs with threads/cancellation.
2011-08-03 19:57:46 -04:00
cb->__next = self->cancelbuf;
self->cancelbuf = cb;
}
simplify cancellation push/pop slightly no need to pass unnecessary extra arguments on to the core code in pthread_create.c. this just wastes cycles and code bloat.
2012-05-23 14:13:54 -04:00
void __do_cleanup_pop(struct __ptcb *cb)
further debloat cancellation handlers cleanup push and pop are also no-ops if pthread_exit is not reachable. this can make a big difference for library code which needs to protect itself against cancellation, but which is unlikely to actually be used in programs with threads/cancellation.
2011-08-03 19:57:46 -04:00
{
always initialize thread pointer at program start this is the first step in an overhaul aimed at greatly simplifying and optimizing everything dealing with thread-local state. previously, the thread pointer was initialized lazily on first access, or at program startup if stack protector was in use, or at certain random places where inconsistent state could be reached if it were not initialized early. while believed to be fully correct, the logic was fragile and non-obvious. in the first phase of the thread pointer overhaul, support is retained (and in some cases improved) for systems/situation where loading the thread pointer fails, e.g. old kernels. some notes on specific changes: - the confusing use of libc.main_thread as an indicator that the thread pointer is initialized is eliminated in favor of an explicit has_thread_pointer predicate. - sigaction no longer needs to ensure that the thread pointer is initialized before installing a signal handler (this was needed to prevent a situation where the signal handler caused the thread pointer to be initialized and the subsequent sigreturn cleared it again) but it still needs to ensure that implementation-internal thread-related signals are not blocked. - pthread tsd initialization for the main thread is deferred in a new manner to minimize bloat in the static-linked __init_tp code. - pthread_setcancelstate no longer needs special handling for the situation before the thread pointer is initialized. it simply fails on systems that cannot support a thread pointer, which are non-conforming anyway. - pthread_cleanup_push/pop now check for missing thread pointer and nop themselves out in this case, so stdio no longer needs to avoid the cancellable path when the thread pointer is not available. a number of cases remain where certain interfaces may crash if the system does not support a thread pointer. at this point, these should be limited to pthread interfaces, and the number of such cases should be fewer than before.
2014-03-24 16:57:11 -04:00
if (!libc.has_thread_pointer) return;
replace bad cancellation cleanup abi with a sane one the old abi was intended to duplicate glibc's abi at the expense of being ugly and slow, but it turns out glib was not even using that abi except on non-gcc-compatible compilers (which it doesn't even support) and was instead using an exceptions-in-c/unwind-based approach whose abi we could not duplicate anyway without nasty dwarf2/unwind integration. the new abi is copied from a very old glibc abi, which seems to still be supported/present in current glibc. it avoids all unwinding, whether by sjlj or exceptions, and merely maintains a linked list of cleanup functions to be called from the context of pthread_exit. i've made some care to ensure that longjmp out of a cleanup function should work, even though it is not required to. this change breaks abi compatibility with programs which were using pthread cancellation, which is unfortunate, but that's why i'm making the change now rather than later. considering that most pthread features have not been usable until recently anyway, i don't see it as a major issue at this point.
2012-02-09 02:33:08 -05:00
__pthread_self()->cancelbuf = cb->__next;
further debloat cancellation handlers cleanup push and pop are also no-ops if pthread_exit is not reachable. this can make a big difference for library code which needs to protect itself against cancellation, but which is unlikely to actually be used in programs with threads/cancellation.
2011-08-03 19:57:46 -04:00
}
overhaul clone syscall wrapping several things are changed. first, i have removed the old __uniclone function signature and replaced it with the "standard" linux __clone/clone signature. this was necessary to expose clone to applications anyway, and it makes it easier to port __clone to new archs, since it's now testable independently of pthread_create. secondly, i have removed all references to the ugly ldt descriptor structure (i386 only) from the c code and pthread structure. in places where it is needed, it is now created on the stack just when it's needed, in assembly code. thus, the i386 __clone function takes the desired thread pointer as its argument, rather than an ldt descriptor pointer, just like on all other sane archs. this should not affect applications since there is really no way an application can use clone with threads/tls in a way that doesn't horribly conflict with and clobber the underlying implementation's use. applications are expected to use clone only for creating actual processes, possibly with new namespace features and whatnot.
2011-09-18 10:14:37 -04:00
static int start(void *p)
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
{
overhaul clone syscall wrapping several things are changed. first, i have removed the old __uniclone function signature and replaced it with the "standard" linux __clone/clone signature. this was necessary to expose clone to applications anyway, and it makes it easier to port __clone to new archs, since it's now testable independently of pthread_create. secondly, i have removed all references to the ugly ldt descriptor structure (i386 only) from the c code and pthread structure. in places where it is needed, it is now created on the stack just when it's needed, in assembly code. thus, the i386 __clone function takes the desired thread pointer as its argument, rather than an ldt descriptor pointer, just like on all other sane archs. this should not affect applications since there is really no way an application can use clone with threads/tls in a way that doesn't horribly conflict with and clobber the underlying implementation's use. applications are expected to use clone only for creating actual processes, possibly with new namespace features and whatnot.
2011-09-18 10:14:37 -04:00
pthread_t self = p;
add support for thread scheduling (POSIX TPS option) linux's sched_* syscalls actually implement the TPS (thread scheduling) functionality, not the PS (process scheduling) functionality which the sched_* functions are supposed to have. omitting support for the PS option (and having the sched_* interfaces fail with ENOSYS rather than omitting them, since some broken software assumes they exist) seems to be the only conforming way to do this on linux.
2012-11-11 15:38:04 -05:00
if (self->startlock[0]) {
__wait(self->startlock, 0, 1, 1);
if (self->startlock[0]) {
self->detached = 2;
pthread_exit(0);
}
transition to using functions for internal signal blocking/restoring there are several reasons for this change. one is getting rid of the repetition of the syscall signature all over the place. another is sharing the constant masks without costly GOT accesses in PIC. the main motivation, however, is accurately representing whether we want to block signals that might be handled by the application, or all signals.
2013-04-26 19:48:01 -04:00
__restore_sigs(self->sigmask);
add support for thread scheduling (POSIX TPS option) linux's sched_* syscalls actually implement the TPS (thread scheduling) functionality, not the PS (process scheduling) functionality which the sched_* functions are supposed to have. omitting support for the PS option (and having the sched_* interfaces fail with ENOSYS rather than omitting them, since some broken software assumes they exist) seems to be the only conforming way to do this on linux.
2012-11-11 15:38:04 -05:00
}
overhaul implementation-internal signal protections the new approach relies on the fact that the only ways to create sigset_t objects without invoking UB are to use the sig*set() functions, or from the masks returned by sigprocmask, sigaction, etc. or in the ucontext_t argument to a signal handler. thus, as long as sigfillset and sigaddset avoid adding the "protected" signals, there is no way the application will ever obtain a sigset_t including these bits, and thus no need to add the overhead of checking/clearing them when sigprocmask or sigaction is called. note that the old code actually *failed* to remove the bits from sa_mask when sigaction was called. the new implementations are also significantly smaller, simpler, and faster due to ignoring the useless "GNU HURD signals" 65-1024, which are not used and, if there's any sanity in the world, never will be used.
2011-05-07 23:23:58 -04:00
if (self->unblock_cancel)
fix (hopefully) all hard-coded 8's for kernel sigset_t size some minor changes to how hard-coded sets for thread-related purposes are handled were also needed, since the old object sizes were not necessarily sufficient. things have gotten a bit ugly in this area, and i think a cleanup is in order at some point, but for now the goal is just to get the code working on all supported archs including mips, which was badly broken by linux rejecting syscalls with the wrong sigset_t size.
2012-08-09 22:52:13 -04:00
__syscall(SYS_rt_sigprocmask, SIG_UNBLOCK,
remove __SYSCALL_SSLEN arch macro in favor of using public _NSIG the issue at hand is that many syscalls require as an argument the kernel-ABI size of sigset_t, intended to allow the kernel to switch to a larger sigset_t in the future. previously, each arch was defining this size in syscall_arch.h, which was redundant with the definition of _NSIG in bits/signal.h. as it's used in some not-quite-portable application code as well, _NSIG is much more likely to be recognized and understood immediately by someone reading the code, and it's also shorter and less cluttered. note that _NSIG is actually 65/129, not 64/128, but the division takes care of throwing away the off-by-one part.
2013-03-26 23:07:31 -04:00
SIGPT_SET, 0, _NSIG/8);
add C11 thread creation and related thread functions based on patch by Jens Gustedt. the main difficulty here is handling the difference between start function signatures and thread return types for C11 threads versus POSIX threads. pointers to void are assumed to be able to represent faithfully all values of int. the function pointer for the thread start function is cast to an incorrect type for passing through pthread_create, but is cast back to its correct type before calling so that the behavior of the call is well-defined. changes to the existing threads implementation were kept minimal to reduce the risk of regressions, and duplication of code that carries implementation-specific assumptions was avoided for ease and safety of future maintenance.
2014-09-07 10:28:08 -04:00
__pthread_exit(self->start(self->start_arg));
return 0;
}
static int start_c11(void *p)
{
pthread_t self = p;
int (*start)(void*) = (int(*)(void*)) self->start;
__pthread_exit((void *)(uintptr_t)start(self->start_arg));
overhaul clone syscall wrapping several things are changed. first, i have removed the old __uniclone function signature and replaced it with the "standard" linux __clone/clone signature. this was necessary to expose clone to applications anyway, and it makes it easier to port __clone to new archs, since it's now testable independently of pthread_create. secondly, i have removed all references to the ugly ldt descriptor structure (i386 only) from the c code and pthread structure. in places where it is needed, it is now created on the stack just when it's needed, in assembly code. thus, the i386 __clone function takes the desired thread pointer as its argument, rather than an ldt descriptor pointer, just like on all other sane archs. this should not affect applications since there is really no way an application can use clone with threads/tls in a way that doesn't horribly conflict with and clobber the underlying implementation's use. applications are expected to use clone only for creating actual processes, possibly with new namespace features and whatnot.
2011-09-18 10:14:37 -04:00
return 0;
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
}
#define ROUND(x) (((x)+PAGE_SIZE-1)&-PAGE_SIZE)
/* pthread_key_create.c overrides this */
work around constant folding bug 61144 in gcc 4.9.0 and 4.9.1 previously we detected this bug in configure and issued advice for a workaround, but this turned out not to work. since then gcc 4.9.0 has appeared in several distributions, and now 4.9.1 has been released without a fix despite this being a wrong code generation bug which is supposed to be a release-blocker, per gcc policy. since the scope of the bug seems to affect only data objects (rather than functions) whose definitions are overridable, and there are only a very small number of these in musl, I am just changing them from const to volatile for the time being. simply removing the const would be sufficient to make gcc 4.9.1 work (the non-const case was inadvertently fixed as part of another change in gcc), and this would also be sufficient with 4.9.0 if we forced -O0 on the affected files or on the whole build. however it's cleaner to just remove all the broken compiler detection and use volatile, which will ensure that they are never constant-folded. the quality of a non-broken compiler's output should not be affected except for the fact that these objects are no longer const and thus possibly add a few bytes to data/bss. this change can be reconsidered and possibly reverted at some point in the future when the broken gcc versions are no longer relevant.
2014-07-16 21:32:06 -04:00
static volatile size_t dummy = 0;
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
weak_alias(dummy, __pthread_tsd_size);
work around constant folding bug 61144 in gcc 4.9.0 and 4.9.1 previously we detected this bug in configure and issued advice for a workaround, but this turned out not to work. since then gcc 4.9.0 has appeared in several distributions, and now 4.9.1 has been released without a fix despite this being a wrong code generation bug which is supposed to be a release-blocker, per gcc policy. since the scope of the bug seems to affect only data objects (rather than functions) whose definitions are overridable, and there are only a very small number of these in musl, I am just changing them from const to volatile for the time being. simply removing the const would be sufficient to make gcc 4.9.1 work (the non-const case was inadvertently fixed as part of another change in gcc), and this would also be sufficient with 4.9.0 if we forced -O0 on the affected files or on the whole build. however it's cleaner to just remove all the broken compiler detection and use volatile, which will ensure that they are never constant-folded. the quality of a non-broken compiler's output should not be affected except for the fact that these objects are no longer const and thus possibly add a few bytes to data/bss. this change can be reconsidered and possibly reverted at some point in the future when the broken gcc versions are no longer relevant.
2014-07-16 21:32:06 -04:00
static void *dummy_tsd[1] = { 0 };
always initialize thread pointer at program start this is the first step in an overhaul aimed at greatly simplifying and optimizing everything dealing with thread-local state. previously, the thread pointer was initialized lazily on first access, or at program startup if stack protector was in use, or at certain random places where inconsistent state could be reached if it were not initialized early. while believed to be fully correct, the logic was fragile and non-obvious. in the first phase of the thread pointer overhaul, support is retained (and in some cases improved) for systems/situation where loading the thread pointer fails, e.g. old kernels. some notes on specific changes: - the confusing use of libc.main_thread as an indicator that the thread pointer is initialized is eliminated in favor of an explicit has_thread_pointer predicate. - sigaction no longer needs to ensure that the thread pointer is initialized before installing a signal handler (this was needed to prevent a situation where the signal handler caused the thread pointer to be initialized and the subsequent sigreturn cleared it again) but it still needs to ensure that implementation-internal thread-related signals are not blocked. - pthread tsd initialization for the main thread is deferred in a new manner to minimize bloat in the static-linked __init_tp code. - pthread_setcancelstate no longer needs special handling for the situation before the thread pointer is initialized. it simply fails on systems that cannot support a thread pointer, which are non-conforming anyway. - pthread_cleanup_push/pop now check for missing thread pointer and nop themselves out in this case, so stdio no longer needs to avoid the cancellable path when the thread pointer is not available. a number of cases remain where certain interfaces may crash if the system does not support a thread pointer. at this point, these should be limited to pthread interfaces, and the number of such cases should be fewer than before.
2014-03-24 16:57:11 -04:00
weak_alias(dummy_tsd, __pthread_tsd_main);
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
overhaul __synccall and fix AS-safety and other issues in set*id multi-threaded set*id and setrlimit use the internal __synccall function to work around the kernel's wrongful treatment of these process properties as thread-local. the old implementation of __synccall failed to be AS-safe, despite POSIX requiring setuid and setgid to be AS-safe, and was not rigorous in assuring that all threads were caught. in a worst case, threads late in the process of exiting could retain permissions after setuid reported success, in which case attacks to regain dropped permissions may have been possible under the right conditions. the new implementation of __synccall depends on the presence of /proc/self/task and will fail if it can't be opened, but is able to determine that it has caught all threads, and does not use any locks except its own. it thereby achieves AS-safety simply by blocking signals to preclude re-entry in the same thread. with this commit, all known conformance and safety issues in set*id functions should be fixed.
2015-01-15 23:17:38 -05:00
volatile int __block_new_threads = 0;
work around constant folding bug 61144 in gcc 4.9.0 and 4.9.1 previously we detected this bug in configure and issued advice for a workaround, but this turned out not to work. since then gcc 4.9.0 has appeared in several distributions, and now 4.9.1 has been released without a fix despite this being a wrong code generation bug which is supposed to be a release-blocker, per gcc policy. since the scope of the bug seems to affect only data objects (rather than functions) whose definitions are overridable, and there are only a very small number of these in musl, I am just changing them from const to volatile for the time being. simply removing the const would be sufficient to make gcc 4.9.1 work (the non-const case was inadvertently fixed as part of another change in gcc), and this would also be sufficient with 4.9.0 if we forced -O0 on the affected files or on the whole build. however it's cleaner to just remove all the broken compiler detection and use volatile, which will ensure that they are never constant-folded. the quality of a non-broken compiler's output should not be affected except for the fact that these objects are no longer const and thus possibly add a few bytes to data/bss. this change can be reconsidered and possibly reverted at some point in the future when the broken gcc versions are no longer relevant.
2014-07-16 21:32:06 -04:00
static FILE *volatile dummy_file = 0;
add proper fuxed-based locking for stdio previously, stdio used spinlocks, which would be unacceptable if we ever add support for thread priorities, and which yielded pathologically bad performance if an application attempted to use flockfile on a key file as a major/primary locking mechanism. i had held off on making this change for fear that it would hurt performance in the non-threaded case, but actually support for recursive locking had already inflicted that cost. by having the internal locking functions store a flag indicating whether they need to perform unlocking, rather than using the actual recursive lock counter, i was able to combine the conditionals at unlock time, eliminating any additional cost, and also avoid a nasty corner case where a huge number of calls to ftrylockfile could cause deadlock later at the point of internal locking. this commit also fixes some issues with usage of pthread_self conflicting with __attribute__((const)) which resulted in crashes with some compiler versions/optimizations, mainly in flockfile prior to pthread_create.
2011-07-30 08:02:14 -04:00
weak_alias(dummy_file, __stdin_used);
weak_alias(dummy_file, __stdout_used);
weak_alias(dummy_file, __stderr_used);
static void init_file_lock(FILE *f)
{
if (f && f->lock<0) f->lock = 0;
}
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 11:51:50 -04:00
void *__copy_tls(unsigned char *);
TLS (GNU/C11 thread-local storage) support for static-linked programs the design for TLS in dynamic-linked programs is mostly complete too, but I have not yet implemented it. cost is nonzero but still low for programs which do not use TLS and/or do not use threads (a few hundred bytes of new code, plus dependency on memcpy). i believe it can be made smaller at some point by merging __init_tls and __init_security into __libc_start_main and avoiding duplicate auxv-parsing code. at the same time, I've also slightly changed the logic pthread_create uses to allocate guard pages to ensure that guard pages are not counted towards commit charge.
2012-10-04 16:35:46 -04:00
use weak symbols for the POSIX functions that will be used by C threads The intent of this is to avoid name space pollution of the C threads implementation. This has two sides to it. First we have to provide symbols that wouldn't pollute the name space for the C threads implementation. Second we have to clean up some internal uses of POSIX functions such that they don't implicitly drag in such symbols.
2014-09-01 00:46:23 +02:00
int __pthread_create(pthread_t *restrict res, const pthread_attr_t *restrict attrp, void *(*entry)(void *), void *restrict arg)
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
{
add C11 thread creation and related thread functions based on patch by Jens Gustedt. the main difficulty here is handling the difference between start function signatures and thread return types for C11 threads versus POSIX threads. pointers to void are assumed to be able to represent faithfully all values of int. the function pointer for the thread start function is cast to an incorrect type for passing through pthread_create, but is cast back to its correct type before calling so that the behavior of the call is well-defined. changes to the existing threads implementation were kept minimal to reduce the risk of regressions, and duplication of code that carries implementation-specific assumptions was avoided for ease and safety of future maintenance.
2014-09-07 10:28:08 -04:00
int ret, c11 = (attrp == __ATTRP_C11_THREAD);
pthread stack treatment overhaul for application-provided stacks, etc. the main goal of these changes is to address the case where an application provides a stack of size N, but TLS has size M that's a significant portion of the size N (or even larger than N), thus giving the application less stack space than it expected or no stack at all! the new strategy pthread_create now uses is to only put TLS on the application-provided stack if TLS is smaller than 1/8 of the stack size or 2k, whichever is smaller. this ensures that the application always has "close enough" to what it requested, and the threshold is chosen heuristically to make sure "sane" amounts of TLS still end up in the application-provided stack. if TLS does not fit the above criteria, pthread_create uses mmap to obtain space for TLS, but still uses the application-provided stack for actual call frame stack. this is to avoid wasting memory, and for the sake of supporting ugly hacks like garbage collection based on assumptions that the implementation will use the provided stack range. in order for the above heuristics to ever succeed, the amount of TLS space wasted on POSIX TSD (pthread_key_create based) needed to be reduced. otherwise, these changes would preclude any use of pthread_create without mmap, which would have serious memory usage and performance costs for applications trying to create huge numbers of threads using pre-allocated stack space. the new value of PTHREAD_KEYS_MAX is the minimum allowed by POSIX, 128. this should still be plenty more than real-world applications need, especially now that C11/gcc-style TLS is now supported in musl, and most apps and libraries choose to use that instead of POSIX TSD when available. at the same time, PTHREAD_STACK_MIN has been decreased. it was originally set to PAGE_SIZE back when there was no support for TLS or application-provided stacks, and requests smaller than a whole page did not make sense. now, there are two good reasons to support requests smaller than a page: (1) applications could provide pre-allocated stacks smaller than a page, and (2) with smaller stack sizes, stack+TLS+TSD can all fit in one page, making it possible for applications which need huge numbers of threads with minimal stack needs to allocate exactly one page per thread. the new value of PTHREAD_STACK_MIN, 2k, is aligned with the minimum size for sigaltstack.
2013-02-01 22:10:40 -05:00
size_t size, guard;
always initialize thread pointer at program start this is the first step in an overhaul aimed at greatly simplifying and optimizing everything dealing with thread-local state. previously, the thread pointer was initialized lazily on first access, or at program startup if stack protector was in use, or at certain random places where inconsistent state could be reached if it were not initialized early. while believed to be fully correct, the logic was fragile and non-obvious. in the first phase of the thread pointer overhaul, support is retained (and in some cases improved) for systems/situation where loading the thread pointer fails, e.g. old kernels. some notes on specific changes: - the confusing use of libc.main_thread as an indicator that the thread pointer is initialized is eliminated in favor of an explicit has_thread_pointer predicate. - sigaction no longer needs to ensure that the thread pointer is initialized before installing a signal handler (this was needed to prevent a situation where the signal handler caused the thread pointer to be initialized and the subsequent sigreturn cleared it again) but it still needs to ensure that implementation-internal thread-related signals are not blocked. - pthread tsd initialization for the main thread is deferred in a new manner to minimize bloat in the static-linked __init_tp code. - pthread_setcancelstate no longer needs special handling for the situation before the thread pointer is initialized. it simply fails on systems that cannot support a thread pointer, which are non-conforming anyway. - pthread_cleanup_push/pop now check for missing thread pointer and nop themselves out in this case, so stdio no longer needs to avoid the cancellable path when the thread pointer is not available. a number of cases remain where certain interfaces may crash if the system does not support a thread pointer. at this point, these should be limited to pthread interfaces, and the number of such cases should be fewer than before.
2014-03-24 16:57:11 -04:00
struct pthread *self, *new;
implement pthread_getattr_np this function is mainly (purely?) for obtaining stack address information, but we also provide the detach state since it's easy to do anyway.
2013-03-31 23:25:55 -04:00
unsigned char *map = 0, *stack = 0, *tsd = 0, *stack_limit;
use symbolic names for clone flags in pthread_create
2013-09-16 10:54:31 -04:00
unsigned flags = CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_SIGHAND
omit CLONE_PARENT flag to clone in pthread_create CLONE_PARENT is not necessary (CLONE_THREAD provides all the useful parts of it) and Linux treats CLONE_PARENT as an error in certain situations, without noticing that it would be a no-op due to CLONE_THREAD. this error case prevents, for example, use of a multi-threaded init process and certain usages with containers.
2013-09-16 10:56:01 -04:00
| CLONE_THREAD | CLONE_SYSVSEM | CLONE_SETTLS
use symbolic names for clone flags in pthread_create
2013-09-16 10:54:31 -04:00
| CLONE_PARENT_SETTID | CLONE_CHILD_CLEARTID | CLONE_DETACHED;
add support for thread scheduling (POSIX TPS option) linux's sched_* syscalls actually implement the TPS (thread scheduling) functionality, not the PS (process scheduling) functionality which the sched_* functions are supposed to have. omitting support for the PS option (and having the sched_* interfaces fail with ENOSYS rather than omitting them, since some broken software assumes they exist) seems to be the only conforming way to do this on linux.
2012-11-11 15:38:04 -05:00
int do_sched = 0;
pthread stack treatment overhaul for application-provided stacks, etc. the main goal of these changes is to address the case where an application provides a stack of size N, but TLS has size M that's a significant portion of the size N (or even larger than N), thus giving the application less stack space than it expected or no stack at all! the new strategy pthread_create now uses is to only put TLS on the application-provided stack if TLS is smaller than 1/8 of the stack size or 2k, whichever is smaller. this ensures that the application always has "close enough" to what it requested, and the threshold is chosen heuristically to make sure "sane" amounts of TLS still end up in the application-provided stack. if TLS does not fit the above criteria, pthread_create uses mmap to obtain space for TLS, but still uses the application-provided stack for actual call frame stack. this is to avoid wasting memory, and for the sake of supporting ugly hacks like garbage collection based on assumptions that the implementation will use the provided stack range. in order for the above heuristics to ever succeed, the amount of TLS space wasted on POSIX TSD (pthread_key_create based) needed to be reduced. otherwise, these changes would preclude any use of pthread_create without mmap, which would have serious memory usage and performance costs for applications trying to create huge numbers of threads using pre-allocated stack space. the new value of PTHREAD_KEYS_MAX is the minimum allowed by POSIX, 128. this should still be plenty more than real-world applications need, especially now that C11/gcc-style TLS is now supported in musl, and most apps and libraries choose to use that instead of POSIX TSD when available. at the same time, PTHREAD_STACK_MIN has been decreased. it was originally set to PAGE_SIZE back when there was no support for TLS or application-provided stacks, and requests smaller than a whole page did not make sense. now, there are two good reasons to support requests smaller than a page: (1) applications could provide pre-allocated stacks smaller than a page, and (2) with smaller stack sizes, stack+TLS+TSD can all fit in one page, making it possible for applications which need huge numbers of threads with minimal stack needs to allocate exactly one page per thread. the new value of PTHREAD_STACK_MIN, 2k, is aligned with the minimum size for sigaltstack.
2013-02-01 22:10:40 -05:00
pthread_attr_t attr = {0};
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
always initialize thread pointer at program start this is the first step in an overhaul aimed at greatly simplifying and optimizing everything dealing with thread-local state. previously, the thread pointer was initialized lazily on first access, or at program startup if stack protector was in use, or at certain random places where inconsistent state could be reached if it were not initialized early. while believed to be fully correct, the logic was fragile and non-obvious. in the first phase of the thread pointer overhaul, support is retained (and in some cases improved) for systems/situation where loading the thread pointer fails, e.g. old kernels. some notes on specific changes: - the confusing use of libc.main_thread as an indicator that the thread pointer is initialized is eliminated in favor of an explicit has_thread_pointer predicate. - sigaction no longer needs to ensure that the thread pointer is initialized before installing a signal handler (this was needed to prevent a situation where the signal handler caused the thread pointer to be initialized and the subsequent sigreturn cleared it again) but it still needs to ensure that implementation-internal thread-related signals are not blocked. - pthread tsd initialization for the main thread is deferred in a new manner to minimize bloat in the static-linked __init_tp code. - pthread_setcancelstate no longer needs special handling for the situation before the thread pointer is initialized. it simply fails on systems that cannot support a thread pointer, which are non-conforming anyway. - pthread_cleanup_push/pop now check for missing thread pointer and nop themselves out in this case, so stdio no longer needs to avoid the cancellable path when the thread pointer is not available. a number of cases remain where certain interfaces may crash if the system does not support a thread pointer. at this point, these should be limited to pthread interfaces, and the number of such cases should be fewer than before.
2014-03-24 16:57:11 -04:00
if (!libc.can_do_threads) return ENOSYS;
self = __pthread_self();
clean up handling of thread/nothread mode, locking
2011-04-17 16:53:54 -04:00
if (!libc.threaded) {
add proper fuxed-based locking for stdio previously, stdio used spinlocks, which would be unacceptable if we ever add support for thread priorities, and which yielded pathologically bad performance if an application attempted to use flockfile on a key file as a major/primary locking mechanism. i had held off on making this change for fear that it would hurt performance in the non-threaded case, but actually support for recursive locking had already inflicted that cost. by having the internal locking functions store a flag indicating whether they need to perform unlocking, rather than using the actual recursive lock counter, i was able to combine the conditionals at unlock time, eliminating any additional cost, and also avoid a nasty corner case where a huge number of calls to ftrylockfile could cause deadlock later at the point of internal locking. this commit also fixes some issues with usage of pthread_self conflicting with __attribute__((const)) which resulted in crashes with some compiler versions/optimizations, mainly in flockfile prior to pthread_create.
2011-07-30 08:02:14 -04:00
for (FILE *f=libc.ofl_head; f; f=f->next)
init_file_lock(f);
init_file_lock(__stdin_used);
init_file_lock(__stdout_used);
init_file_lock(__stderr_used);
always initialize thread pointer at program start this is the first step in an overhaul aimed at greatly simplifying and optimizing everything dealing with thread-local state. previously, the thread pointer was initialized lazily on first access, or at program startup if stack protector was in use, or at certain random places where inconsistent state could be reached if it were not initialized early. while believed to be fully correct, the logic was fragile and non-obvious. in the first phase of the thread pointer overhaul, support is retained (and in some cases improved) for systems/situation where loading the thread pointer fails, e.g. old kernels. some notes on specific changes: - the confusing use of libc.main_thread as an indicator that the thread pointer is initialized is eliminated in favor of an explicit has_thread_pointer predicate. - sigaction no longer needs to ensure that the thread pointer is initialized before installing a signal handler (this was needed to prevent a situation where the signal handler caused the thread pointer to be initialized and the subsequent sigreturn cleared it again) but it still needs to ensure that implementation-internal thread-related signals are not blocked. - pthread tsd initialization for the main thread is deferred in a new manner to minimize bloat in the static-linked __init_tp code. - pthread_setcancelstate no longer needs special handling for the situation before the thread pointer is initialized. it simply fails on systems that cannot support a thread pointer, which are non-conforming anyway. - pthread_cleanup_push/pop now check for missing thread pointer and nop themselves out in this case, so stdio no longer needs to avoid the cancellable path when the thread pointer is not available. a number of cases remain where certain interfaces may crash if the system does not support a thread pointer. at this point, these should be limited to pthread interfaces, and the number of such cases should be fewer than before.
2014-03-24 16:57:11 -04:00
__syscall(SYS_rt_sigprocmask, SIG_UNBLOCK, SIGPT_SET, 0, _NSIG/8);
fix pointer type mismatch and misplacement of const
2014-03-24 17:39:08 -04:00
self->tsd = (void **)__pthread_tsd_main;
clean up handling of thread/nothread mode, locking
2011-04-17 16:53:54 -04:00
libc.threaded = 1;
}
add C11 thread creation and related thread functions based on patch by Jens Gustedt. the main difficulty here is handling the difference between start function signatures and thread return types for C11 threads versus POSIX threads. pointers to void are assumed to be able to represent faithfully all values of int. the function pointer for the thread start function is cast to an incorrect type for passing through pthread_create, but is cast back to its correct type before calling so that the behavior of the call is well-defined. changes to the existing threads implementation were kept minimal to reduce the risk of regressions, and duplication of code that carries implementation-specific assumptions was avoided for ease and safety of future maintenance.
2014-09-07 10:28:08 -04:00
if (attrp && !c11) attr = *attrp;
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 11:51:50 -04:00
__acquire_ptc();
overhaul __synccall and fix AS-safety and other issues in set*id multi-threaded set*id and setrlimit use the internal __synccall function to work around the kernel's wrongful treatment of these process properties as thread-local. the old implementation of __synccall failed to be AS-safe, despite POSIX requiring setuid and setgid to be AS-safe, and was not rigorous in assuring that all threads were caught. in a worst case, threads late in the process of exiting could retain permissions after setuid reported success, in which case attacks to regain dropped permissions may have been possible under the right conditions. the new implementation of __synccall depends on the presence of /proc/self/task and will fail if it can't be opened, but is able to determine that it has caught all threads, and does not use any locks except its own. it thereby achieves AS-safety simply by blocking signals to preclude re-entry in the same thread. with this commit, all known conformance and safety issues in set*id functions should be fixed.
2015-01-15 23:17:38 -05:00
if (__block_new_threads) __wait(&__block_new_threads, 0, 1, 1);
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 11:51:50 -04:00
pthread stack treatment overhaul for application-provided stacks, etc. the main goal of these changes is to address the case where an application provides a stack of size N, but TLS has size M that's a significant portion of the size N (or even larger than N), thus giving the application less stack space than it expected or no stack at all! the new strategy pthread_create now uses is to only put TLS on the application-provided stack if TLS is smaller than 1/8 of the stack size or 2k, whichever is smaller. this ensures that the application always has "close enough" to what it requested, and the threshold is chosen heuristically to make sure "sane" amounts of TLS still end up in the application-provided stack. if TLS does not fit the above criteria, pthread_create uses mmap to obtain space for TLS, but still uses the application-provided stack for actual call frame stack. this is to avoid wasting memory, and for the sake of supporting ugly hacks like garbage collection based on assumptions that the implementation will use the provided stack range. in order for the above heuristics to ever succeed, the amount of TLS space wasted on POSIX TSD (pthread_key_create based) needed to be reduced. otherwise, these changes would preclude any use of pthread_create without mmap, which would have serious memory usage and performance costs for applications trying to create huge numbers of threads using pre-allocated stack space. the new value of PTHREAD_KEYS_MAX is the minimum allowed by POSIX, 128. this should still be plenty more than real-world applications need, especially now that C11/gcc-style TLS is now supported in musl, and most apps and libraries choose to use that instead of POSIX TSD when available. at the same time, PTHREAD_STACK_MIN has been decreased. it was originally set to PAGE_SIZE back when there was no support for TLS or application-provided stacks, and requests smaller than a whole page did not make sense. now, there are two good reasons to support requests smaller than a page: (1) applications could provide pre-allocated stacks smaller than a page, and (2) with smaller stack sizes, stack+TLS+TSD can all fit in one page, making it possible for applications which need huge numbers of threads with minimal stack needs to allocate exactly one page per thread. the new value of PTHREAD_STACK_MIN, 2k, is aligned with the minimum size for sigaltstack.
2013-02-01 22:10:40 -05:00
if (attr._a_stackaddr) {
size_t need = libc.tls_size + __pthread_tsd_size;
size = attr._a_stacksize + DEFAULT_STACK_SIZE;
stack = (void *)(attr._a_stackaddr & -16);
fix type error in pthread_create, introduced with pthread_getattr_np
2013-04-06 01:15:08 -04:00
stack_limit = (void *)(attr._a_stackaddr - size);
pthread stack treatment overhaul for application-provided stacks, etc. the main goal of these changes is to address the case where an application provides a stack of size N, but TLS has size M that's a significant portion of the size N (or even larger than N), thus giving the application less stack space than it expected or no stack at all! the new strategy pthread_create now uses is to only put TLS on the application-provided stack if TLS is smaller than 1/8 of the stack size or 2k, whichever is smaller. this ensures that the application always has "close enough" to what it requested, and the threshold is chosen heuristically to make sure "sane" amounts of TLS still end up in the application-provided stack. if TLS does not fit the above criteria, pthread_create uses mmap to obtain space for TLS, but still uses the application-provided stack for actual call frame stack. this is to avoid wasting memory, and for the sake of supporting ugly hacks like garbage collection based on assumptions that the implementation will use the provided stack range. in order for the above heuristics to ever succeed, the amount of TLS space wasted on POSIX TSD (pthread_key_create based) needed to be reduced. otherwise, these changes would preclude any use of pthread_create without mmap, which would have serious memory usage and performance costs for applications trying to create huge numbers of threads using pre-allocated stack space. the new value of PTHREAD_KEYS_MAX is the minimum allowed by POSIX, 128. this should still be plenty more than real-world applications need, especially now that C11/gcc-style TLS is now supported in musl, and most apps and libraries choose to use that instead of POSIX TSD when available. at the same time, PTHREAD_STACK_MIN has been decreased. it was originally set to PAGE_SIZE back when there was no support for TLS or application-provided stacks, and requests smaller than a whole page did not make sense. now, there are two good reasons to support requests smaller than a page: (1) applications could provide pre-allocated stacks smaller than a page, and (2) with smaller stack sizes, stack+TLS+TSD can all fit in one page, making it possible for applications which need huge numbers of threads with minimal stack needs to allocate exactly one page per thread. the new value of PTHREAD_STACK_MIN, 2k, is aligned with the minimum size for sigaltstack.
2013-02-01 22:10:40 -05:00
/* Use application-provided stack for TLS only when
* it does not take more than ~12% or 2k of the
* application's stack space. */
if (need < size/8 && need < 2048) {
tsd = stack - __pthread_tsd_size;
stack = tsd - libc.tls_size;
fix use of uninitialized memory with application-provided thread stacks the subsequent code in pthread_create and the code which copies TLS initialization images to the new thread's TLS space assume that the memory provided to them is zero-initialized, which is true when it's obtained by pthread_create using mmap. however, when the caller provides a stack using pthread_attr_setstack, pthread_create cannot make any assumptions about the contents. simply zero-filling the relevant memory in this case is the simplest and safest fix.
2014-08-22 14:05:10 -04:00
memset(stack, 0, need);
pthread stack treatment overhaul for application-provided stacks, etc. the main goal of these changes is to address the case where an application provides a stack of size N, but TLS has size M that's a significant portion of the size N (or even larger than N), thus giving the application less stack space than it expected or no stack at all! the new strategy pthread_create now uses is to only put TLS on the application-provided stack if TLS is smaller than 1/8 of the stack size or 2k, whichever is smaller. this ensures that the application always has "close enough" to what it requested, and the threshold is chosen heuristically to make sure "sane" amounts of TLS still end up in the application-provided stack. if TLS does not fit the above criteria, pthread_create uses mmap to obtain space for TLS, but still uses the application-provided stack for actual call frame stack. this is to avoid wasting memory, and for the sake of supporting ugly hacks like garbage collection based on assumptions that the implementation will use the provided stack range. in order for the above heuristics to ever succeed, the amount of TLS space wasted on POSIX TSD (pthread_key_create based) needed to be reduced. otherwise, these changes would preclude any use of pthread_create without mmap, which would have serious memory usage and performance costs for applications trying to create huge numbers of threads using pre-allocated stack space. the new value of PTHREAD_KEYS_MAX is the minimum allowed by POSIX, 128. this should still be plenty more than real-world applications need, especially now that C11/gcc-style TLS is now supported in musl, and most apps and libraries choose to use that instead of POSIX TSD when available. at the same time, PTHREAD_STACK_MIN has been decreased. it was originally set to PAGE_SIZE back when there was no support for TLS or application-provided stacks, and requests smaller than a whole page did not make sense. now, there are two good reasons to support requests smaller than a page: (1) applications could provide pre-allocated stacks smaller than a page, and (2) with smaller stack sizes, stack+TLS+TSD can all fit in one page, making it possible for applications which need huge numbers of threads with minimal stack needs to allocate exactly one page per thread. the new value of PTHREAD_STACK_MIN, 2k, is aligned with the minimum size for sigaltstack.
2013-02-01 22:10:40 -05:00
} else {
size = ROUND(need);
guard = 0;
add pthread_attr_setstack interface (and get) i originally omitted these (optional, per POSIX) interfaces because i considered them backwards implementation details. however, someone later brought to my attention a fairly legitimate use case: allocating thread stacks in memory that's setup for sharing and/or fast transfer between CPU and GPU so that the thread can move data to a GPU directly from automatic-storage buffers without having to go through additional buffer copies. perhaps there are other situations in which these interfaces are useful too.
2012-06-09 19:53:29 -04:00
}
pthread stack treatment overhaul for application-provided stacks, etc. the main goal of these changes is to address the case where an application provides a stack of size N, but TLS has size M that's a significant portion of the size N (or even larger than N), thus giving the application less stack space than it expected or no stack at all! the new strategy pthread_create now uses is to only put TLS on the application-provided stack if TLS is smaller than 1/8 of the stack size or 2k, whichever is smaller. this ensures that the application always has "close enough" to what it requested, and the threshold is chosen heuristically to make sure "sane" amounts of TLS still end up in the application-provided stack. if TLS does not fit the above criteria, pthread_create uses mmap to obtain space for TLS, but still uses the application-provided stack for actual call frame stack. this is to avoid wasting memory, and for the sake of supporting ugly hacks like garbage collection based on assumptions that the implementation will use the provided stack range. in order for the above heuristics to ever succeed, the amount of TLS space wasted on POSIX TSD (pthread_key_create based) needed to be reduced. otherwise, these changes would preclude any use of pthread_create without mmap, which would have serious memory usage and performance costs for applications trying to create huge numbers of threads using pre-allocated stack space. the new value of PTHREAD_KEYS_MAX is the minimum allowed by POSIX, 128. this should still be plenty more than real-world applications need, especially now that C11/gcc-style TLS is now supported in musl, and most apps and libraries choose to use that instead of POSIX TSD when available. at the same time, PTHREAD_STACK_MIN has been decreased. it was originally set to PAGE_SIZE back when there was no support for TLS or application-provided stacks, and requests smaller than a whole page did not make sense. now, there are two good reasons to support requests smaller than a page: (1) applications could provide pre-allocated stacks smaller than a page, and (2) with smaller stack sizes, stack+TLS+TSD can all fit in one page, making it possible for applications which need huge numbers of threads with minimal stack needs to allocate exactly one page per thread. the new value of PTHREAD_STACK_MIN, 2k, is aligned with the minimum size for sigaltstack.
2013-02-01 22:10:40 -05:00
} else {
guard = ROUND(DEFAULT_GUARD_SIZE + attr._a_guardsize);
size = guard + ROUND(DEFAULT_STACK_SIZE + attr._a_stacksize
+ libc.tls_size + __pthread_tsd_size);
}
if (!tsd) {
TLS (GNU/C11 thread-local storage) support for static-linked programs the design for TLS in dynamic-linked programs is mostly complete too, but I have not yet implemented it. cost is nonzero but still low for programs which do not use TLS and/or do not use threads (a few hundred bytes of new code, plus dependency on memcpy). i believe it can be made smaller at some point by merging __init_tls and __init_security into __libc_start_main and avoiding duplicate auxv-parsing code. at the same time, I've also slightly changed the logic pthread_create uses to allocate guard pages to ensure that guard pages are not counted towards commit charge.
2012-10-04 16:35:46 -04:00
if (guard) {
use weak symbols for the POSIX functions that will be used by C threads The intent of this is to avoid name space pollution of the C threads implementation. This has two sides to it. First we have to provide symbols that wouldn't pollute the name space for the C threads implementation. Second we have to clean up some internal uses of POSIX functions such that they don't implicitly drag in such symbols.
2014-09-01 00:46:23 +02:00
map = __mmap(0, size, PROT_NONE, MAP_PRIVATE|MAP_ANON, -1, 0);
fix stale locks left behind when pthread_create fails this bug seems to have been around a long time.
2013-02-01 22:25:19 -05:00
if (map == MAP_FAILED) goto fail;
use weak symbols for the POSIX functions that will be used by C threads The intent of this is to avoid name space pollution of the C threads implementation. This has two sides to it. First we have to provide symbols that wouldn't pollute the name space for the C threads implementation. Second we have to clean up some internal uses of POSIX functions such that they don't implicitly drag in such symbols.
2014-09-01 00:46:23 +02:00
if (__mprotect(map+guard, size-guard, PROT_READ|PROT_WRITE)) {
__munmap(map, size);
fix stale locks left behind when pthread_create fails this bug seems to have been around a long time.
2013-02-01 22:25:19 -05:00
goto fail;
TLS (GNU/C11 thread-local storage) support for static-linked programs the design for TLS in dynamic-linked programs is mostly complete too, but I have not yet implemented it. cost is nonzero but still low for programs which do not use TLS and/or do not use threads (a few hundred bytes of new code, plus dependency on memcpy). i believe it can be made smaller at some point by merging __init_tls and __init_security into __libc_start_main and avoiding duplicate auxv-parsing code. at the same time, I've also slightly changed the logic pthread_create uses to allocate guard pages to ensure that guard pages are not counted towards commit charge.
2012-10-04 16:35:46 -04:00
}
} else {
use weak symbols for the POSIX functions that will be used by C threads The intent of this is to avoid name space pollution of the C threads implementation. This has two sides to it. First we have to provide symbols that wouldn't pollute the name space for the C threads implementation. Second we have to clean up some internal uses of POSIX functions such that they don't implicitly drag in such symbols.
2014-09-01 00:46:23 +02:00
map = __mmap(0, size, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANON, -1, 0);
fix stale locks left behind when pthread_create fails this bug seems to have been around a long time.
2013-02-01 22:25:19 -05:00
if (map == MAP_FAILED) goto fail;
TLS (GNU/C11 thread-local storage) support for static-linked programs the design for TLS in dynamic-linked programs is mostly complete too, but I have not yet implemented it. cost is nonzero but still low for programs which do not use TLS and/or do not use threads (a few hundred bytes of new code, plus dependency on memcpy). i believe it can be made smaller at some point by merging __init_tls and __init_security into __libc_start_main and avoiding duplicate auxv-parsing code. at the same time, I've also slightly changed the logic pthread_create uses to allocate guard pages to ensure that guard pages are not counted towards commit charge.
2012-10-04 16:35:46 -04:00
}
add pthread_attr_setstack interface (and get) i originally omitted these (optional, per POSIX) interfaces because i considered them backwards implementation details. however, someone later brought to my attention a fairly legitimate use case: allocating thread stacks in memory that's setup for sharing and/or fast transfer between CPU and GPU so that the thread can move data to a GPU directly from automatic-storage buffers without having to go through additional buffer copies. perhaps there are other situations in which these interfaces are useful too.
2012-06-09 19:53:29 -04:00
tsd = map + size - __pthread_tsd_size;
implement pthread_getattr_np this function is mainly (purely?) for obtaining stack address information, but we also provide the detach state since it's easy to do anyway.
2013-03-31 23:25:55 -04:00
if (!stack) {
stack = tsd - libc.tls_size;
stack_limit = map + guard;
}
optimize out useless default-attribute object in pthread_create
2011-05-07 23:39:48 -04:00
}
pthread stack treatment overhaul for application-provided stacks, etc. the main goal of these changes is to address the case where an application provides a stack of size N, but TLS has size M that's a significant portion of the size N (or even larger than N), thus giving the application less stack space than it expected or no stack at all! the new strategy pthread_create now uses is to only put TLS on the application-provided stack if TLS is smaller than 1/8 of the stack size or 2k, whichever is smaller. this ensures that the application always has "close enough" to what it requested, and the threshold is chosen heuristically to make sure "sane" amounts of TLS still end up in the application-provided stack. if TLS does not fit the above criteria, pthread_create uses mmap to obtain space for TLS, but still uses the application-provided stack for actual call frame stack. this is to avoid wasting memory, and for the sake of supporting ugly hacks like garbage collection based on assumptions that the implementation will use the provided stack range. in order for the above heuristics to ever succeed, the amount of TLS space wasted on POSIX TSD (pthread_key_create based) needed to be reduced. otherwise, these changes would preclude any use of pthread_create without mmap, which would have serious memory usage and performance costs for applications trying to create huge numbers of threads using pre-allocated stack space. the new value of PTHREAD_KEYS_MAX is the minimum allowed by POSIX, 128. this should still be plenty more than real-world applications need, especially now that C11/gcc-style TLS is now supported in musl, and most apps and libraries choose to use that instead of POSIX TSD when available. at the same time, PTHREAD_STACK_MIN has been decreased. it was originally set to PAGE_SIZE back when there was no support for TLS or application-provided stacks, and requests smaller than a whole page did not make sense. now, there are two good reasons to support requests smaller than a page: (1) applications could provide pre-allocated stacks smaller than a page, and (2) with smaller stack sizes, stack+TLS+TSD can all fit in one page, making it possible for applications which need huge numbers of threads with minimal stack needs to allocate exactly one page per thread. the new value of PTHREAD_STACK_MIN, 2k, is aligned with the minimum size for sigaltstack.
2013-02-01 22:10:40 -05:00
new = __copy_tls(tsd - libc.tls_size);
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
new->map_base = map;
new->map_size = size;
implement pthread_getattr_np this function is mainly (purely?) for obtaining stack address information, but we also provide the detach state since it's easy to do anyway.
2013-03-31 23:25:55 -04:00
new->stack = stack;
new->stack_size = stack - stack_limit;
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
new->start = entry;
new->start_arg = arg;
new->self = new;
new->tsd = (void *)tsd;
add locale framework this commit adds non-stub implementations of setlocale, duplocale, newlocale, and uselocale, along with the data structures and minimal code needed for representing the active locale on a per-thread basis and optimizing the common case where thread-local locale settings are not in use. at this point, the data structures only contain what is necessary to represent LC_CTYPE (a single flag) and LC_MESSAGES (a name for use in finding message translation files). representation for the other categories will be added later; the expectation is that a single pointer will suffice for each. for LC_CTYPE, the strings "C" and "POSIX" are treated as special; any other string is accepted and treated as "C.UTF-8". for other categories, any string is accepted after being truncated to a maximum supported length (currently 15 bytes). for LC_MESSAGES, the name is kept regardless of whether libc itself can use such a message translation locale, since applications using catgets or gettext should be able to use message locales libc is not aware of. for other categories, names which are not successfully loaded as locales (which, at present, means all names) are treated as aliases for "C". setlocale never fails. locale settings are not yet used anywhere, so this commit should have no visible effects except for the contents of the string returned by setlocale.
2014-07-02 19:33:19 -04:00
new->locale = &libc.global_locale;
pthread stack treatment overhaul for application-provided stacks, etc. the main goal of these changes is to address the case where an application provides a stack of size N, but TLS has size M that's a significant portion of the size N (or even larger than N), thus giving the application less stack space than it expected or no stack at all! the new strategy pthread_create now uses is to only put TLS on the application-provided stack if TLS is smaller than 1/8 of the stack size or 2k, whichever is smaller. this ensures that the application always has "close enough" to what it requested, and the threshold is chosen heuristically to make sure "sane" amounts of TLS still end up in the application-provided stack. if TLS does not fit the above criteria, pthread_create uses mmap to obtain space for TLS, but still uses the application-provided stack for actual call frame stack. this is to avoid wasting memory, and for the sake of supporting ugly hacks like garbage collection based on assumptions that the implementation will use the provided stack range. in order for the above heuristics to ever succeed, the amount of TLS space wasted on POSIX TSD (pthread_key_create based) needed to be reduced. otherwise, these changes would preclude any use of pthread_create without mmap, which would have serious memory usage and performance costs for applications trying to create huge numbers of threads using pre-allocated stack space. the new value of PTHREAD_KEYS_MAX is the minimum allowed by POSIX, 128. this should still be plenty more than real-world applications need, especially now that C11/gcc-style TLS is now supported in musl, and most apps and libraries choose to use that instead of POSIX TSD when available. at the same time, PTHREAD_STACK_MIN has been decreased. it was originally set to PAGE_SIZE back when there was no support for TLS or application-provided stacks, and requests smaller than a whole page did not make sense. now, there are two good reasons to support requests smaller than a page: (1) applications could provide pre-allocated stacks smaller than a page, and (2) with smaller stack sizes, stack+TLS+TSD can all fit in one page, making it possible for applications which need huge numbers of threads with minimal stack needs to allocate exactly one page per thread. the new value of PTHREAD_STACK_MIN, 2k, is aligned with the minimum size for sigaltstack.
2013-02-01 22:10:40 -05:00
if (attr._a_detach) {
fix potential race condition in detached threads after the thread unmaps its own stack/thread structure, the kernel, performing child tid clear and futex wake, could clobber a new mapping made at the same location as the just-removed thread's tid field. disable kernel clearing of child tid to prevent this.
2012-07-11 23:36:46 -04:00
new->detached = 1;
use symbolic names for clone flags in pthread_create
2013-09-16 10:54:31 -04:00
flags -= CLONE_CHILD_CLEARTID;
fix potential race condition in detached threads after the thread unmaps its own stack/thread structure, the kernel, performing child tid clear and futex wake, could clobber a new mapping made at the same location as the just-removed thread's tid field. disable kernel clearing of child tid to prevent this.
2012-07-11 23:36:46 -04:00
}
pthread stack treatment overhaul for application-provided stacks, etc. the main goal of these changes is to address the case where an application provides a stack of size N, but TLS has size M that's a significant portion of the size N (or even larger than N), thus giving the application less stack space than it expected or no stack at all! the new strategy pthread_create now uses is to only put TLS on the application-provided stack if TLS is smaller than 1/8 of the stack size or 2k, whichever is smaller. this ensures that the application always has "close enough" to what it requested, and the threshold is chosen heuristically to make sure "sane" amounts of TLS still end up in the application-provided stack. if TLS does not fit the above criteria, pthread_create uses mmap to obtain space for TLS, but still uses the application-provided stack for actual call frame stack. this is to avoid wasting memory, and for the sake of supporting ugly hacks like garbage collection based on assumptions that the implementation will use the provided stack range. in order for the above heuristics to ever succeed, the amount of TLS space wasted on POSIX TSD (pthread_key_create based) needed to be reduced. otherwise, these changes would preclude any use of pthread_create without mmap, which would have serious memory usage and performance costs for applications trying to create huge numbers of threads using pre-allocated stack space. the new value of PTHREAD_KEYS_MAX is the minimum allowed by POSIX, 128. this should still be plenty more than real-world applications need, especially now that C11/gcc-style TLS is now supported in musl, and most apps and libraries choose to use that instead of POSIX TSD when available. at the same time, PTHREAD_STACK_MIN has been decreased. it was originally set to PAGE_SIZE back when there was no support for TLS or application-provided stacks, and requests smaller than a whole page did not make sense. now, there are two good reasons to support requests smaller than a page: (1) applications could provide pre-allocated stacks smaller than a page, and (2) with smaller stack sizes, stack+TLS+TSD can all fit in one page, making it possible for applications which need huge numbers of threads with minimal stack needs to allocate exactly one page per thread. the new value of PTHREAD_STACK_MIN, 2k, is aligned with the minimum size for sigaltstack.
2013-02-01 22:10:40 -05:00
if (attr._a_sched) {
add support for thread scheduling (POSIX TPS option) linux's sched_* syscalls actually implement the TPS (thread scheduling) functionality, not the PS (process scheduling) functionality which the sched_* functions are supposed to have. omitting support for the PS option (and having the sched_* interfaces fail with ENOSYS rather than omitting them, since some broken software assumes they exist) seems to be the only conforming way to do this on linux.
2012-11-11 15:38:04 -05:00
do_sched = new->startlock[0] = 1;
transition to using functions for internal signal blocking/restoring there are several reasons for this change. one is getting rid of the repetition of the syscall signature all over the place. another is sharing the constant masks without costly GOT accesses in PIC. the main motivation, however, is accurately representing whether we want to block signals that might be handled by the application, or all signals.
2013-04-26 19:48:01 -04:00
__block_app_sigs(new->sigmask);
add support for thread scheduling (POSIX TPS option) linux's sched_* syscalls actually implement the TPS (thread scheduling) functionality, not the PS (process scheduling) functionality which the sched_* functions are supposed to have. omitting support for the PS option (and having the sched_* interfaces fail with ENOSYS rather than omitting them, since some broken software assumes they exist) seems to be the only conforming way to do this on linux.
2012-11-11 15:38:04 -05:00
}
optimize out setting up robust list with kernel when not needed as a result of commit 12e1e324683a1d381b7f15dd36c99b37dd44d940, kernel processing of the robust list is only needed for process-shared mutexes. previously the first attempt to lock any owner-tracked mutex resulted in robust list initialization and a set_robust_list syscall. this is no longer necessary, and since the kernel's record of the robust list must now be cleared at thread exit time for detached threads, optimizing it out is more worthwhile than before too.
2015-04-10 00:54:48 -04:00
new->robust_list.head = &new->robust_list.head;
major improvements to cancellation handling - there is no longer any risk of spoofing cancellation requests, since the cancel flag is set in pthread_cancel rather than in the signal handler. - cancellation signal is no longer unblocked when running the cancellation handlers. instead, pthread_create will cause any new threads created from a cancellation handler to unblock their own cancellation signal. - various tweaks in preparation for POSIX timer support.
2011-03-29 12:58:22 -04:00
new->unblock_cancel = self->cancel;
clean up and refactor program initialization the code in __libc_start_main is now responsible for parsing auxv, rather than duplicating the parsing all over the place. this should shave off a few cycles and some code size. __init_libc is left as an external-linkage function despite the fact that it could be static, to prevent it from being inlined and permanently wasting stack space when main is called. a few other minor changes are included, like eliminating per-thread ssp canaries (they were likely broken when combined with certain dlopen usages, and completely unnecessary) and some other unnecessary checks. since this code gets linked into every program, it should be as small and simple as possible.
2012-10-07 21:43:46 -04:00
new->canary = self->canary;
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
a_inc(&libc.threads_minus_1);
add C11 thread creation and related thread functions based on patch by Jens Gustedt. the main difficulty here is handling the difference between start function signatures and thread return types for C11 threads versus POSIX threads. pointers to void are assumed to be able to represent faithfully all values of int. the function pointer for the thread start function is cast to an incorrect type for passing through pthread_create, but is cast back to its correct type before calling so that the behavior of the call is well-defined. changes to the existing threads implementation were kept minimal to reduce the risk of regressions, and duplication of code that carries implementation-specific assumptions was avoided for ease and safety of future maintenance.
2014-09-07 10:28:08 -04:00
ret = __clone((c11 ? start_c11 : start), stack, flags, new, &new->tid, TP_ADJ(new), &new->tid);
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
support for TLS in dynamic-loaded (dlopen) modules unlike other implementations, this one reserves memory for new TLS in all pre-existing threads at dlopen-time, and dlopen will fail with no resources consumed and no new libraries loaded if memory is not available. memory is not immediately distributed to running threads; that would be too complex and too costly. instead, assurances are made that threads needing the new TLS can obtain it in an async-signal-safe way from a buffer belonging to the dynamic linker/new module (via atomic fetch-and-add based allocator). I've re-appropriated the lock that was previously used for __synccall (synchronizing set*id() syscalls between threads) as a general pthread_create lock. it's a "backwards" rwlock where the "read" operation is safe atomic modification of the live thread count, which multiple threads can perform at the same time, and the "write" operation is making sure the count does not increase during an operation that depends on it remaining bounded (__synccall or dlopen). in static-linked programs that don't use __synccall, this lock is a no-op and has no cost.
2012-10-05 11:51:50 -04:00
__release_ptc();
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
add support for thread scheduling (POSIX TPS option) linux's sched_* syscalls actually implement the TPS (thread scheduling) functionality, not the PS (process scheduling) functionality which the sched_* functions are supposed to have. omitting support for the PS option (and having the sched_* interfaces fail with ENOSYS rather than omitting them, since some broken software assumes they exist) seems to be the only conforming way to do this on linux.
2012-11-11 15:38:04 -05:00
if (do_sched) {
transition to using functions for internal signal blocking/restoring there are several reasons for this change. one is getting rid of the repetition of the syscall signature all over the place. another is sharing the constant masks without costly GOT accesses in PIC. the main motivation, however, is accurately representing whether we want to block signals that might be handled by the application, or all signals.
2013-04-26 19:48:01 -04:00
__restore_sigs(new->sigmask);
add support for thread scheduling (POSIX TPS option) linux's sched_* syscalls actually implement the TPS (thread scheduling) functionality, not the PS (process scheduling) functionality which the sched_* functions are supposed to have. omitting support for the PS option (and having the sched_* interfaces fail with ENOSYS rather than omitting them, since some broken software assumes they exist) seems to be the only conforming way to do this on linux.
2012-11-11 15:38:04 -05:00
}
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
if (ret < 0) {
a_dec(&libc.threads_minus_1);
use weak symbols for the POSIX functions that will be used by C threads The intent of this is to avoid name space pollution of the C threads implementation. This has two sides to it. First we have to provide symbols that wouldn't pollute the name space for the C threads implementation. Second we have to clean up some internal uses of POSIX functions such that they don't implicitly drag in such symbols.
2014-09-01 00:46:23 +02:00
if (map) __munmap(map, size);
make pthread_create return EAGAIN on resource failure, as required by POSIX
2011-02-15 02:20:21 -05:00
return EAGAIN;
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
}
add support for thread scheduling (POSIX TPS option) linux's sched_* syscalls actually implement the TPS (thread scheduling) functionality, not the PS (process scheduling) functionality which the sched_* functions are supposed to have. omitting support for the PS option (and having the sched_* interfaces fail with ENOSYS rather than omitting them, since some broken software assumes they exist) seems to be the only conforming way to do this on linux.
2012-11-11 15:38:04 -05:00
if (do_sched) {
ret = __syscall(SYS_sched_setscheduler, new->tid,
pthread stack treatment overhaul for application-provided stacks, etc. the main goal of these changes is to address the case where an application provides a stack of size N, but TLS has size M that's a significant portion of the size N (or even larger than N), thus giving the application less stack space than it expected or no stack at all! the new strategy pthread_create now uses is to only put TLS on the application-provided stack if TLS is smaller than 1/8 of the stack size or 2k, whichever is smaller. this ensures that the application always has "close enough" to what it requested, and the threshold is chosen heuristically to make sure "sane" amounts of TLS still end up in the application-provided stack. if TLS does not fit the above criteria, pthread_create uses mmap to obtain space for TLS, but still uses the application-provided stack for actual call frame stack. this is to avoid wasting memory, and for the sake of supporting ugly hacks like garbage collection based on assumptions that the implementation will use the provided stack range. in order for the above heuristics to ever succeed, the amount of TLS space wasted on POSIX TSD (pthread_key_create based) needed to be reduced. otherwise, these changes would preclude any use of pthread_create without mmap, which would have serious memory usage and performance costs for applications trying to create huge numbers of threads using pre-allocated stack space. the new value of PTHREAD_KEYS_MAX is the minimum allowed by POSIX, 128. this should still be plenty more than real-world applications need, especially now that C11/gcc-style TLS is now supported in musl, and most apps and libraries choose to use that instead of POSIX TSD when available. at the same time, PTHREAD_STACK_MIN has been decreased. it was originally set to PAGE_SIZE back when there was no support for TLS or application-provided stacks, and requests smaller than a whole page did not make sense. now, there are two good reasons to support requests smaller than a page: (1) applications could provide pre-allocated stacks smaller than a page, and (2) with smaller stack sizes, stack+TLS+TSD can all fit in one page, making it possible for applications which need huge numbers of threads with minimal stack needs to allocate exactly one page per thread. the new value of PTHREAD_STACK_MIN, 2k, is aligned with the minimum size for sigaltstack.
2013-02-01 22:10:40 -05:00
attr._a_policy, &attr._a_prio);
add support for thread scheduling (POSIX TPS option) linux's sched_* syscalls actually implement the TPS (thread scheduling) functionality, not the PS (process scheduling) functionality which the sched_* functions are supposed to have. omitting support for the PS option (and having the sched_* interfaces fail with ENOSYS rather than omitting them, since some broken software assumes they exist) seems to be the only conforming way to do this on linux.
2012-11-11 15:38:04 -05:00
a_store(new->startlock, ret<0 ? 2 : 0);
__wake(new->startlock, 1, 1);
if (ret < 0) return -ret;
}
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
*res = new;
return 0;
fix stale locks left behind when pthread_create fails this bug seems to have been around a long time.
2013-02-01 22:25:19 -05:00
fail:
__release_ptc();
return EAGAIN;
initial check-in, version 0.5.0
2011-02-12 00:22:29 -05:00
}
use weak symbols for the POSIX functions that will be used by C threads The intent of this is to avoid name space pollution of the C threads implementation. This has two sides to it. First we have to provide symbols that wouldn't pollute the name space for the C threads implementation. Second we have to clean up some internal uses of POSIX functions such that they don't implicitly drag in such symbols.
2014-09-01 00:46:23 +02:00
weak_alias(__pthread_exit, pthread_exit);
weak_alias(__pthread_create, pthread_create);
Reference in New Issue Copy Permalink