The basic design is to create an outer-most parallel team. It is not a regular team because it is only created when the first hidden helper task is encountered, and is only responsible for the execution of hidden helper tasks.  We first use `pthread_create` to create a new thread, let's call it the initial and also the main thread of the hidden helper team. This initial thread then initializes a new root, just like what RTL does in initialization. After that, it directly calls `__kmpc_fork_call`. It is like the initial thread encounters a parallel region. The wrapped function for this team is, for main thread, which is the initial thread that we create via `pthread_create` on Linux, waits on a condition variable. The condition variable can only be signaled when RTL is being destroyed. For other work threads, they just do nothing. The reason that main thread needs to wait there is, in current implementation, once the main thread finishes the wrapped function of this team, it starts to free the team which is not what we want.

Two environment variables, `LIBOMP_NUM_HIDDEN_HELPER_THREADS` and `LIBOMP_USE_HIDDEN_HELPER_TASK`, are also set to configure the number of threads and enable/disable this feature. By default, the number of hidden helper threads is 8.

Here are some open issues to be discussed:
1. The main thread goes to sleeping when the initialization is finished. As Andrey mentioned, we might need it to be awaken from time to time to do some stuffs. What kind of update/check should be put here?

Reviewed By: jdoerfert

Differential Revision: https://reviews.llvm.org/D77609

This is NFC-intended and another step towards supporting
intrinsics as reduction candidates.

The remaining bits of the OperationData class do not make
much sense as-is, so I will try to improve that, but I'm
trying to take minimal steps because it's still not clear
how this was intended to work.

This is another NFC-intended patch to allow matching
intrinsics (example: maxnum) as candidates for reductions.

It's possible that the loop/if logic can be reduced now,
but it's still difficult to understand how this all works.

Reviewed By: lebedev.ri

Differential Revision: https://reviews.llvm.org/D94870

This treats low overhead loop branches the same as jump tables and
indirect branches in analyzeBranch - they cannot be analyzed but the
direct branches on the end of the block may be removed. This helps
remove the unnecessary branches earlier, which can help produce better
codegen (and change block layout in a number of cases).

Differential Revision: https://reviews.llvm.org/D94392

We now have a lot of llc tests for hardware loops in CodeGen, which test
a larger variety of loops and are easier to maintain. This removes the
llc from mixed llc/opt tests.

This patch removes some ancient options as a clean-up before moving
code-gen to use LTOBackend in D94487.

I think it would preferable to remove those ancient options, because

  1. There are no corresponding options in LTOBackend based tools,
  2. There are no unit tests for them,
  3. They are not passed through by Clang,
  4. At least for GNVLoadPRE, users could just use GVN's `enable-load-pre`.

Alternatively we could add support for those options to lto::Config &
co, but I think it would be better to remove them, unless they are
actually used in practice.

Reviewed By: steven_wu, tejohnson

Differential Revision: https://reviews.llvm.org/D94783

According to "9. Vector Memory Alignment Constraints" in V
specification, the alignment of vector memory access is aligned to the
size of the element. In our current implementation, we support ELEN up
to 64. We could assume the alignment of vector registers is 64 under the
assumption.

Differential Revision: https://reviews.llvm.org/D94751

Current code breaks this version of MSVC due to a mismatch between `std::is_trivially_copyable` and `llvm::is_trivially_copyable` for `std::pair` instantiations.  Hence I was attempting to use `std::is_trivially_copyable` to set `llvm::is_trivially_copyable<T>::value`.

I spent some time root causing an `llvm::Optional` build error on MSVC 16.8.3 related to the change described above:

```
62>C:\src\ocg_llvm\llvm-project\llvm\include\llvm/ADT/BreadthFirstIterator.h(96,12): error C2280: 'llvm::Optional<std::pair<std::pair<unsigned int,llvm::Graph<4>::NodeSubset> *,llvm::Optional<llvm::Graph<4>::ChildIterator>>> &llvm::Optional<std::pair<std::pair<unsigned int,llvm::Graph<4>::NodeSubset> *,llvm::Optional<llvm::Graph<4>::ChildIterator>>>::operator =(const llvm::Optional<std::pair<std::pair<unsigned int,llvm::Graph<4>::NodeSubset> *,llvm::Optional<llvm::Graph<4>::ChildIterator>>> &)': attempting to reference a deleted function (compiling source file C:\src\ocg_llvm\llvm-project\llvm\unittests\ADT\BreadthFirstIteratorTest.cpp)
...
```
The "trivial" specialization of `optional_detail::OptionalStorage` assumes that the value type is trivially copy constructible and trivially copy assignable. The specialization is invoked based on a check of `is_trivially_copyable` alone, which does not imply both `is_trivially_copy_assignable` and `is_trivially_copy_constructible` are true.

[[ https://en.cppreference.com/w/cpp/named_req/TriviallyCopyable | According to the spec ]], a deleted assignment operator does not make `is_trivially_copyable` false. So I think all these properties need to be checked explicitly in order to specialize `OptionalStorage` to the "trivial" version:
```
/// Storage for any type.
template <typename T, bool = std::is_trivially_copy_constructible<T>::value
                          && std::is_trivially_copy_assignable<T>::value>
class OptionalStorage {
```
Above fixed my build break in MSVC, but I think we need to explicitly check `is_trivially_copy_constructible` too since it might be possible the copy constructor is deleted.  Also would be ideal to move over to `std::is_trivially_copyable` instead of the `llvm` namespace verson.

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D93510

Differential Revision: https://reviews.llvm.org/D94130

This is a simple utility which allows matching on binaryOperator and
cxxOperatorCallExpr. It can also be extended to support
cxxRewrittenBinaryOperator.

Add generic support for MapAnyOfMatchers to auto-marshalling functions.

Differential Revision: https://reviews.llvm.org/D94129

This is a follow-up fix to commit 03c8d6a0.
Seems like we now end up with NeedInvert being set in the result
from LegalizeSetCCCondCode more often than in the past, so we
need to handle NeedInvert when expanding BR_CC.

Not sure how to deal with the "Tmp4.getNode()" case properly,
but current assumption is that that code path isn't impacted
by the changes in 03c8d6a0 so we can simply move
the old assert into the if-branch and only handle NeedInvert in the
else-branch.

I think that the test case added here, for PowerPC, might have
failed also before commit 03c8d6a0. But we started
to hit the assert more often downstream when having merged that
commit.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D94762

This makes them composable with mapAnyOf().

Differential Revision: https://reviews.llvm.org/D94128

Make it possible to compose a matcher for different base nodes.

This accepts one or more node matcher functors and zero or more
matchers, composing the latter into the former.

This allows composing of matchers where the same inner matcher name is
used for the same concept, but with a different node functor. Currently,
there is a limitation that the nodes must be in the same "clade", so
while

  mapAnyOf(ifStmt, forStmt).with(hasBody(stmt()))

can be used, functionDecl can not be added to the tuple.

It is possible to use this in clang-query, but it will require changes
to the QueryParser, so is deferred to a future review.

Differential Revision: https://reviews.llvm.org/D94127

The ``llvm.experimental.noalias.scope.decl`` intrinsic identifies where a noalias
scope is declared. When the intrinsic is duplicated, a decision must
also be made about the scope: depending on the reason of the duplication,
the scope might need to be duplicated as well.

Reviewed By: nikic, jdoerfert

Differential Revision: https://reviews.llvm.org/D93039

Forgot to check if the predicate is safe to commutate operands.

inline

The stats are printed at InlinePass destruction. When we have 2 of them,
it appears the destruction order of the Passes std::vector of the pass
manager differs in msan builds - example:
http://lab.llvm.org:8011/#/builders/74/builds/2135.

This reproes locally, too.

Temporarily removing the sub-test case, to green the build, and will
follow up with a stat dumping alternative that does not depend on vector
element dtor order.

This test will fail with any toolchains that don't default to C11.

Adding this switch to the clang invocation in the test fixes the issue.

Patch by Justice Adams!

Reviewed By: dyung

Differential Revision: https://reviews.llvm.org/D94829

This patch renames SubsequentDelim to ListSeparator to clarify the
purpose of the class.

Differential Revision: https://reviews.llvm.org/D94649

Expanding from D94808 - we ensure the same InlineAdvisor is used by both
InlinerPass instances. The notion of mandatory inlining is moved into
the core InlineAdvisor: advisors anyway have to handle that case, so
this change also factors out that a bit better.

Differential Revision: https://reviews.llvm.org/D94825

This fixes errors when location strings contains newlines, or other non-ascii characters.

Differential Revision: https://reviews.llvm.org/D94847

This allow using this helper outside of the linalg canonicalization.

Differential Revision: https://reviews.llvm.org/D94826

When a reference to a generic interface occurs in a specification
expression that must be emitted to a module file, we have a problem
when the generic resolves to a function whose name is inaccessible
due to being PRIVATE or due to a conflict with another use of the
same name in the scope.  In these cases, construct a new name for
the specific procedure and emit a renaming USE to the module file.
Also, relax enforcement of PRIVATE when analyzing module files.

Differential Revision: https://reviews.llvm.org/D94815

Differential Revision: https://reviews.llvm.org/D94836