This should restore the rule of <=80 characters per line.

llvm-svn: 322841

Summary:
getgrouplist, getgroupmembership -- calculate group access list

Sponsored by <The NetBSD Foundation>

Reviewers: vitalybuka, joerg

Reviewed By: vitalybuka

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D42064

llvm-svn: 322836

Summary:
access, faccessat - check access permissions of a file or pathname

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D42065

llvm-svn: 322831

Summary:
From <pwd.h>: user_from_uid, uid_from_user

From <grp.h>: group_from_gid, gid_from_group

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D42068

llvm-svn: 322829

Summary:
Make common allocator agnostic to failure handling modes and move the
decision up to the particular sanitizer's allocator, where the context
is available (call stack, parameters, return nullptr/crash mode etc.)

It simplifies the common allocator and allows the particular sanitizer's
allocator to generate more specific and detailed error reports (which
will be implemented later).

The behavior is largely the same, except one case, the violation of the
common allocator's check for "size + alignment" overflow is now reportied
as OOM instead of "bad request". It feels like a worthy tradeoff and
"size + alignment" is huge in this case anyway (thus, can be interpreted
as not enough memory to satisfy the request). There's also a Report()
statement added there.

Reviewers: eugenis

Subscribers: kubamracek, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D42198

llvm-svn: 322784

Summary:
acct - enable or disable process accounting

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D42066

llvm-svn: 322646

Summary:
This patch (on top of the previous two (https://reviews.llvm.org/D40898 and
https://reviews.llvm.org/D40899) complete the compiler-rt side of the the Solaris
sanitizer port.

It contains the following sets of changes:

* For the time being, the port is for 32-bit x86 only, so reject the various tests on
  x86_64.

* When compiling as C++, <setjmp.h> resp. <iso/setjmp_iso.h> only declares
  _setjmp and _longjmp inside namespace std.

* MAP_FILE is a Windows feature.  While e.g. Linux <sys/mman.h> provides a
  no-op compat define, Solaris does not.

* test/asan/TestCases/Posix/coverage.cc was initially failing like this:

/vol/gcc/src/llvm/llvm/local/projects/compiler-rt/lib/sanitizer_common/scripts/sancov.py: 4 files merged; 2 PCs total
rm: cannot remove '/var/gcc/llvm/local/projects/compiler-rt/test/asan/I386SunOSConfig/TestCases/Posix/Output/coverage': Invalid argument

  Further digging revealed that the rm was trying to remove the running test's working
  directory which failed as observed.  cd'ing out of the dir before let the test pass.

* Two tests needed a declaration of alloca. I've now copied the existing code from
  test/asan/TestCases/alloca_constant_size.cc, but it may be more profitable and
  maintainable to have a common testsuite header where such code is collected.

* Similarly, Solaris' printf %p format doesn't include the leading 0x.

* In test/asan/TestCases/malloc-no-intercept.c, I had to undef __EXTENSIONS__
  (predefined by clang for no apparent reason) to avoid conflicting declarations
  for memalign.

* test/ubsan/TestCases/Float/cast-overflow.cpp has different platform dependent
  ways to define BYTE_ORDER and friends.  Why not just use __BYTE_ORDER__ and
  friends as predefined by clang and gcc?

Patch by Rainer Orth.

Reviewers: kcc, alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, kubamracek, mgorny, krytarowski, fedor.sergeev, JDevlieghere, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40900

llvm-svn: 322635

This reverts commit r322424: this broke the tsan lint check.

llvm-svn: 322428

It was always intended to be.

Patch By: mcgrathr

Differential Revision: https://reviews.llvm.org/D41513

llvm-svn: 322424

Summary:
itimerval can contain padding that may be legitimately uninitialized.

On NetBSD there are four integers of type "long, int, long, int", the
int argument stands for __sanitizer_suseconds_t. Compiler adds extra
padding in this layout.

Check every field of struct itimerval separately.

Define __sanitizer_suseconds_t as long on FreeBSD, Linux and SmartOS,
and int on NetBSD. Define __sanitizer_timeval and __sanitizer_itimerval.

Sponsored by <The NetBSD Foundation>

Reviewers: eugenis, joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: emaste, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D41502

llvm-svn: 322399

Summary: Patch by David CARLIER

Reviewers: vitalybuka, kcc, dim, emaste, davide, morehouse, george.karpenkov

Reviewed By: morehouse

Subscribers: george.karpenkov, kubamracek, srhines, mgorny, emaste, krytarowski

Differential Revision: https://reviews.llvm.org/D41642

llvm-svn: 322380

NDK < 13 & API_LEVEL < 21 do not define struct mmsghdr.
Newer NDK use unified headers and provide this definition for all api
levels.

Since we can not check for the NDK version, check the api level. This
is more strict than absolutely necessary, but it does not really
matter: it is only a sanity check.

llvm-svn: 321817

We're having some use cases where we have more than 128 (the current maximum) instrumented dynamic libraries loaded into a single process. Let's bump the limit to 1024, and separate the constants.

Differential Revision: https://reviews.llvm.org/D41190

llvm-svn: 321782

Summary: Extend the sendmsg test to cover all recv*.

Reviewers: vitalybuka

Subscribers: llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D41620

llvm-svn: 321774

llvm-svn: 321544

Summary:
Export aligned new/delete to make dynamic runtimes work again.

Remove all valid new/delete cases from ASan test, there's a test in
common for that.

Reviewers: eugenis

Subscribers: srhines, kubamracek, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D41548

llvm-svn: 321394

r318733 introduced a build failure for native MIPS32 systems for xray due
to the lack of __sync_fetch_and_add / __syn_fetch_and_sub support. This patch
extends the existing support providing atomics so that xray can be
successfully built.

The initial patch was reverted in r321292, as I suspected it may have caused the
buildbot failure. Another patch in the updates the bot fetched caused the test
failures which was reverted.

Reviewers: atanasyan, dberris

Differential Revision: https://reviews.llvm.org/D40385

llvm-svn: 321383

Summary:
This patch, on top of https://reviews.llvm.org/D40898, contains the build system
changes necessary to enable the Solaris/x86 sanitizer port.

The only issue of note is the libclang_rt.sancov_{begin, end} libraries: clang relies on the
linker automatically defining __start_SECNAME and __stop_SECNAME labels for
sections whose names are valid C identifiers.  This is a GNU ld extension not present
in the ELF gABI, also implemented by gold and lld, but not by Solaris ld.  To work around
this, I automatically link the sancov_{begin,end} libraries into every executable for now.
There seems to be now way to build individual startup objects like crtbegin.o/crtend.o,
so I've followed the lead of libclang_rt.asan-preinit which also contains just a single
object.

Reviewers: kcc, alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, kubamracek, mgorny, fedor.sergeev, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40899

llvm-svn: 321373

This reverts commit r321260. It appears to have broken the sanitizer
bot sanitizer-ppc64be-linux.

http://lab.llvm.org:8011/builders/sanitizer-ppc64be-linux/builds/5029

Reverting to see if the buildbot turns green.

llvm-svn: 321292

r318733 introduced a build failure for native MIPS32 systems for xray due
to the lack of __sync_fetch_and_add / __syn_fetch_and_sub support. This patch
extends the existing support providing atomics so that xray can be
successfully built.

Reviewers: atanasyan, dberris

Differential Revision: https://reviews.llvm.org/D40385

llvm-svn: 321260

There could be a situation when a specific DSO was built with FORTIFY_SOURCE option. In case asan-ed binary link against that DSO,
libasan can't handle the possible memory error because it does not have interceptors for spinrtf_chk, snprintf_chk, vprintf_chk,
vsnprintf_chk, __fprintf_chk functions. Let's interceptors for them.

Patch by Denis Khalikov.

Differential Revision: https://reviews.llvm.org/D40951

llvm-svn: 320990

Summary:
https://reviews.llvm.org/D41121 broke the FreeBSD build due to that type not
being defined on FreeBSD. As far as I can tell, it is an int, but I do not have
a way to test the change.

Reviewers: alekseyshl, kparzysz

Reviewed By: kparzysz

Subscribers: kparzysz, emaste, kubamracek, krytarowski, #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D41325

llvm-svn: 320931

Summary:
This is the first mostly working version of the Sanitizer port to 32-bit Solaris/x86.
It is currently based on Solaris 11.4 Beta.

This part was initially developed inside libsanitizer in the GCC tree and should apply to
both.  Subsequent parts will address changes to clang, the compiler-rt build system
and testsuite.

I'm not yet sure what the right patch granularity is: if it's profitable to split the patch
up, I'd like to get guidance on how to do so.

Most of the changes are probably straightforward with a few exceptions:

* The Solaris syscall interface isn't stable, undocumented and can change within an
  OS release.  The stable interface is the libc interface, which I'm using here, if possible
  using the internal _-prefixed names.

* While the patch primarily target 32-bit x86, I've left a few sparc changes in.  They
  cannot currently be used with clang due to a backend limitation, but have worked
  fine inside the gcc tree.

* Some functions (e.g. largefile versions of functions like open64) only exist in 32-bit
  Solaris, so I've introduced a separate SANITIZER_SOLARIS32 to check for that.

The patch (with the subsequent ones to be submitted shortly) was tested
on i386-pc-solaris2.11.  Only a few failures remain, some of them analyzed, some
still TBD:

    AddressSanitizer-i386-sunos :: TestCases/Posix/concurrent_overflow.cc
    AddressSanitizer-i386-sunos :: TestCases/init-order-atexit.cc
    AddressSanitizer-i386-sunos :: TestCases/log-path_test.cc
    AddressSanitizer-i386-sunos :: TestCases/malloc-no-intercept.c
    AddressSanitizer-i386-sunos-dynamic :: TestCases/Posix/concurrent_overflow.cc
    AddressSanitizer-i386-sunos-dynamic :: TestCases/Posix/start-deactivated.cc
    AddressSanitizer-i386-sunos-dynamic :: TestCases/default_options.cc
    AddressSanitizer-i386-sunos-dynamic :: TestCases/init-order-atexit.cc
    AddressSanitizer-i386-sunos-dynamic :: TestCases/log-path_test.cc
    AddressSanitizer-i386-sunos-dynamic :: TestCases/malloc-no-intercept.c

   SanitizerCommon-Unit :: ./Sanitizer-i386-Test/MemoryMappingLayout.DumpListOfModules
    SanitizerCommon-Unit :: ./Sanitizer-i386-Test/SanitizerCommon.PthreadDestructorIterations

Maybe this is good enough the get the ball rolling.

Reviewers: kcc, alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, jyknight, kubamracek, krytarowski, fedor.sergeev, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40898

llvm-svn: 320740

Summary:
Changes:

 - Don't attempt to intercept GLIBC specific functions like __strtol_internal.
   This is required to stop intercepting it as we leak dlerror(3) for dlsym(3)
   that cannot manage to find a symbol.
 - Correct interception of fstatat(2).
 - Don't run a test for fgetgrent_r() that is missing on NetBSD.
 - Correct link_map location (offset) in Obj_Entry on x86_64 and i386.
 - Stop intercepting getpshared-like functions in pthread(3). This is feature
   is not enabled by default on NetBSD as it's unfinished.
 - Switch intercepting from UTMP to UTMPX functions.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, eugenis, kcc

Reviewed By: vitalybuka

Subscribers: llvm-commits, srhines, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D41053

llvm-svn: 320663

MADV_DONTNEED on Linux actually mark the pages as free to be overwritten with zeroes, but on Darwin and BSD, it's just an advisory flag (the OS cannot discard the content). We should use MADV_FREE on Darwin and BSD.

Differential Revision: https://reviews.llvm.org/D40666

llvm-svn: 320659

Summary:
See D40657 & D40679 for previous versions of this patch & description.

A couple of things were fixed here to have it not break some bots.
Weak symbols can't be used with `SANITIZER_GO` so the previous version was
breakin TsanGo. I set up some additional local tests and those pass now.

I changed the workaround for the glibc vDSO issue: `__progname` is initialized
after the vDSO and is actually public and of known type, unlike
`__vdso_clock_gettime`. This works better, and with all compilers.

The rest is the same.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, kubamracek, krytarowski, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D41121

llvm-svn: 320594

Summary: D40679 broke a couple of builds, reverting while investigating.

Reviewers: alekseyshl

Reviewed By: alekseyshl

Subscribers: srhines, kubamracek, krytarowski, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D41088

llvm-svn: 320417

Summary:
Redo of D40657, which had the initial discussion. The initial code had to move
into a libcdep file, and things had to be shuffled accordingly.

`NanoTime` is a time sink when checking whether or not to release memory to
the OS. While reducing the amount of calls to said function is in the works,
another solution that was found to be beneficial was to use a timing function
that can leverage the vDSO.

We hit a couple of snags along the way, like the fact that the glibc crashes
when clock_gettime is called from a preinit_array, or the fact that
`__vdso_clock_gettime` is mangled (for security purposes) and can't be used
directly, and also that clock_gettime can be intercepted.

The proposed solution takes care of all this as far as I can tell, and
significantly improve performances and some Scudo load tests with memory
reclaiming enabled.

@mcgrathr: please feel free to follow up on
https://reviews.llvm.org/D40657#940857 here. I posted a reply at
https://reviews.llvm.org/D40657#940974.

Reviewers: alekseyshl, krytarowski, flowerhack, mcgrathr, kubamracek

Reviewed By: alekseyshl, krytarowski

Subscribers: #sanitizers, mcgrathr, srhines, llvm-commits, kubamracek

Differential Revision: https://reviews.llvm.org/D40679

llvm-svn: 320409

Summary:
Include <sys/tls.h> for:

 - struct tls_tcb - thread control block structure
 - __HAVE___LWP_GETTCB_FAST - __lwp_gettcb_fast() is available
 - __HAVE___LWP_GETPRIVATE_FAST -  __lwp_getprivate_fast() is available
 - __HAVE_TLS_VARIANT_I - TLS Variant I for this architecture
 - __HAVE_TLS_VARIANT_II - TLS Variant II for this architecture

Rename ThreadSelfSegbase() to ThreadSelfTlsTcb and switch it
to retrieve in a portable way TCB.

Switch ThreadSelf() to retrieve pthread from struct tcb_tls.

Use dl_iterate_phdr() to find out the size of TLS block of
the main program.

Correct the index of the  TLS block of the main program
(dlpi_tls_modid); it's 1, not 2.

New NetBSD code is now CPU (NetBSD port) agnostic.

Stop sharing the same code with FreeBSD.

Sponsored by <The NetBSD Foundation>

Reviewers: dvyukov, joerg, vitalybuka

Reviewed By: vitalybuka

Subscribers: emaste, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40159

llvm-svn: 320234

Summary:
Runtime library for HWASan, initial commit.
Does not randomize tags yet, does not handle stack or globals.

Reviewers: kcc, pcc, alekseyshl

Subscribers: srhines, kubamracek, dberris, mgorny, llvm-commits, krytarowski

Differential Revision: https://reviews.llvm.org/D40935

llvm-svn: 320231

Summary:
Switch NetBSD from GNU to the POSIX strerror_r(3) interceptor.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, eugenis, vitalybuka

Reviewed By: vitalybuka

Subscribers: srhines, kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D41019

llvm-svn: 320221

llvm-svn: 319984

r319875 caused a sign comparison build failure. Explicitly set the
enum's type to be unsigned.

llvm-svn: 319977

Summary:
NetBSD renames symbols for historical and compat reasons.

Add required symbol renames in sanitizer_common_interceptors.inc:

 - gettimeofday -> __gettimeofday50
 - getrusage -> __getrusage50
 - shmctl -> __shmctl50

Additionally handle sigaction symbol mangling.
Rename the function symbol in the file to SIGACTION_SYMNAME and define
it as __sigaction14 for NetBSD and sigaction for !NetBSD. We cannot use
simple renaming with the proprocessor, as there are valid fields named
sigaction and they must be left intact.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, eugenis, vitalybuka, dvyukov

Reviewed By: vitalybuka

Subscribers: kubamracek, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40766

llvm-svn: 319966

Summary:
This is an attempt at making `PopulateFreeArray` less obscure, more consistent,
and a tiny bit faster in some circumstances:
- use more consistent variable names, that work both for the user & the metadata
  portions of the code; the purpose of the code is mostly the same for both
  regions, so it makes sense that the code should be mostly similar as well;
- replace the while sum loops with a single `RoundUpTo`;
- mask most of the metadata computations behind kMetadataSize, allowing some
  blocks to be completely optimized out if not use metadata;
- `const` the constant variables;
- add a `LIKELY` as the branch it applies to will almost always be taken.

Reviewers: alekseyshl, flowerhack

Reviewed By: alekseyshl

Subscribers: kubamracek, llvm-commits

Differential Revision: https://reviews.llvm.org/D40754

llvm-svn: 319673

Summary:
NetBSD renames symbols for historical and compat reasons.

Add required symbol renames in sanitizer_common_interceptors.inc:

 - clock_gettime -> __clock_gettime50
 - clock_getres -> __clock_getres50
 - clock_settime -> __clock_settime50
 - setitimer -> __setitimer50
 - getitimer -> __getitimer50
 - opendir -> __opendir30
 - readdir -> __readdir30
 - time -> __time50
 - localtime_r -> __localtime_r50
 - gmtime_r -> __gmtime_r50
 - gmtime -> __gmtime50
 - ctime -> __ctime50
 - ctime_r -> __ctime_r50
 - mktime -> __mktime50
 - getpwnam -> __getpwnam50
 - getpwuid -> __getpwuid50
 - getpwnam_r -> __getpwnam_r50
 - getpwuid_r -> __getpwuid_r50
 - getpwent -> __getpwent50
 - glob -> __glob30
 - wait3 -> __wait350
 - wait4 -> __wait450
 - readdir_r -> __readdir_r30
 - setlocale -> __setlocale50
 - scandir -> __scandir30
 - sigtimedwait -> __sigtimedwait50
 - sigemptyset -> __sigemptyset14
 - sigfillset -> __sigfillset14
 - sigpending -> __sigpending14
 - sigprocmask -> __sigprocmask14
 - shmctl -> __shmctl50
 - times -> __times13
 - stat -> __stat50
 - getutent -> __getutent50
 - getutxent -> __getutxent50
 - getutxid -> __getutxid50
 - getutxline -> __getutxline50

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, vitalybuka, cryptoad, dvyukov

Reviewed By: cryptoad

Subscribers: llvm-commits, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40765

llvm-svn: 319670

Summary:
The low-fat STL-like vector container will be reused in MSan.

It is needed to implement an atexit(3) interceptor on NetBSD/amd64 in MSan.

Sponsored by <The NetBSD Foundation>

Reviewers: joerg, dvyukov, eugenis, vitalybuka, kcc

Reviewed By: dvyukov

Subscribers: kubamracek, mgorny, llvm-commits, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D40726

llvm-svn: 319650

[sanitizer] Add 'strip_env' flag to enable/disable removing sanitizer dylib from DYLD_INSERT_LIBRARIES

On macOS, we usually don't require launching the target with DYLD_INSERT_LIBRARIES anymore. However, it is still necessary when running a target that is not instrumented (and e.g. dlopen's an instrument library later). In any case, ASan and TSan currently remove themselves from the DYLD_INSERT_LIBRARIES environment variable to avoid passing it onto children. This works well e.g. when instrumenting a shell. A problem arises when the target is a non-instrumented shim (e.g. "xcrun") that either re-execs or launches a child that is supposed to get DYLD_INSERT_LIBRARIES propagated. To support this mode, this patch introduces 'strip_env' flag that can be used to keep DYLD_INSERT_LIBRARIES untouched.

Differential Revision: https://reviews.llvm.org/D39991

llvm-svn: 319365

Differential Revision: https://reviews.llvm.org/D39987

llvm-svn: 319349

It looks FindDynamicShadowStart has a bug: When iterating over the memory map, we will not consider the very last gap in the address space. Let's fix that.

Differential Revision: https://reviews.llvm.org/D39989

llvm-svn: 319348