This pass transforms full-length vector instructions or intrinsics calls
to VP ones by recovering the (mask,evl) information from one of the
memory writing VP operations and backpropagating it.

Add new VP Recipes for the Explicit Vector Length (EVL) and add support
for VP memory intrinsics (vp.load, vp.store, vp.gather, vp.scatter).

After function argument lowering, but prior to instruction selection,
dbg declares pointing to function arguments are lowered using special
logic.

Later, during instruction selection (both "fast" and regular ISel), this
logic is "repeated" in order to identify which intrinsics have already
been lowered. This is bad for two reasons:

1. The logic is not _really_ repeated, the code is different, which
could lead to duplicate lowering of the intrinsic.
2. Even if the logic were repeated properly, this is still code
duplication.

This patch addresses these issues by storing all preprocessed
dbg.declare intrinsics in a set inside FuncInfo; the set is queried upon
instruction selection.

Differential Revision: https://reviews.llvm.org/D149682

Improves the codegen for VECREDUCE_{AND,OR,XOR} operations on AArch64.
Currently, these are fully scalarized, except if the vector is a <N x i1>. This
patch improves the codegen down to O(log(N)) where N is the length of the
vector for vectors whose elements are not i1, by repeatedly applying the
bitwise operations to the two halves of the vector. <N x i1> bitwise reductions
are handled using VECREDUCE_{UMAX,UMIN,ADD} instead.

I had to update quite a few codegen tests with these changes, with a general
downward trend in instruction count. Since the vector reductions already have
tests, I haven't added any new tests myself.

Differential Revision: https://reviews.llvm.org/D148185

This is a follow up to D149263 which extends the generic vslide1down handling to use vslidedown (without the one) for undef elements, and in particular for undef sub-sequences. This both removes the domain crossing, and for undef subsequences results in fewer instructions over all.

Differential Revision: https://reviews.llvm.org/D149658#inline-1446673

When the values are in GPRs, the vslide1down lowering is always better. We need to greatly improve the splat-and-mask cost model to handle constants in a meaningful way, so for now, limit this to non-constant vectors.

This does send the "partially constant" case down the vslide1down path. This could cause some regressions, though I don't see any in practice.

The cost modeling for the general case is annoyingly tricky. We have a great amount of inconsistency around immediate operands, and as a result, the exact constant and exact lowering choice matters a lot. I'm hoping that we get a "good enough" result without modeling this exactly, but we may need to do something analogous to getIntMatCost (i.e. a search w/costing).

Differential Revision: https://reviews.llvm.org/D149667

This patch add the instructions of zcmt extension.
[[ https://github.com/riscv/riscv-code-size-reduction/releases/tag/v1.0.0-RC5.7 | spac is here ]]
Which includes two instructions (cm.jt&cm.jalt) and a CSR Reg JVT

co-author: @Scott Egerton

Reviewed By: kito-cheng, craig.topper

Differential Revision: https://reviews.llvm.org/D133863

The previous pattern was matching the RVA `0` to the first character of `0x1010`. Make sure now that the entire export entry is matched.

This tries to push the concat in trunc(concat(rshr, rshr)) into the leaves, so
that we can generate rshrn(concat). This helps improve the codegen for small
types, using the existing rshrn patterns.

Differential Revision: https://reviews.llvm.org/D149636

This reverts commit 22e2db60.

Broke buildbots on Windows. It seems standard headers on Windows contain
flexible array members in unions

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D149152

It was rejected in C, and in a strange way accepted in C++. However, the
support was never properly tested and fully implemented, so just reject
it in C++ mode as well.

This change also fixes crash on attempt to initialize union with flexible
array member. Due to missing check on union, there was a null expression
added to init list that caused crash later.

Fixes https://github.com/llvm/llvm-project/issues/61746

Reviewed By: aaron.ballman

Differential Revision: https://reviews.llvm.org/D147626

See D149636

OutputSection::checkDynRelAddends() incorrectly reports an internal
linker error for large addends on 32-bit targets. This is caused by the
lack of sign extension in DynamicReloc::computeAddend() for 32-bit
addends.

Differential Revision: https://reviews.llvm.org/D149347

Allow shrink-wrapping past memory accesses that only access globals or
function arguments. This patch uses getUnderlyingObject to try to
identify the accessed object by a given memory operand. If it is a
global or an argument, it does not access the stack of the current
function and should not block shrink wrapping.

Note that the caller's stack may get accessed when passing an argument
via the stack, but not the stack of the current function.

This addresses part of the TODO from D63152.

Reviewed By: thegameg

Differential Revision: https://reviews.llvm.org/D149668

Lower vector subscripted designators as values when they appear outside
of the assignment left-hand side and input IO contexts.

This matches Fortran semantics where vector subscripted designators cannot
be written to outside of the two contexts mentioned above: they are
passed/taken by value where they appear.

This patch uses the added hlfir.element_addr to lower vector designators
in lowering. But when reaching the end of the designator lowering, the
hlfir.element_addr is turned into an hlfir.elemental when lowering is
not asking for the hlfir.elemental_addr.

This approach allows lowering vector subscripted in the same way in
while visiting the designator, and only adapt to the context at the
edge.

The part where lowering uses the hlfir.elemental_addr will be
done in further patch as it requires lowering assignments in the
new hlfir.region_assign op, and there is not codegen yet for these
new operations.

Differential Revision: https://reviews.llvm.org/D149480

See the operation description in HLFIROps.td.

Depends on D149442

Differential Revision: https://reviews.llvm.org/D149449

hlfir.region_assign is a Region based version of hlfir.assign: the
right-hand side and left-hand-side are evaluated in their own region,
and an optional region can be added to implement user defined
assignment.

This will be used for:
 - assignments inside where and forall
 - user defined assignments
 - assignments to vector subscripted entities.

Rational:

Forall and Where lowering requires solving an expression/assignment
evaluation scheduling problem based on data dependencies between the
variables being assigned and the one used in the expressions.
Keeping left-hand side and right-hand side in their own region will
make it really easy to analyse the dependency and move around the
expression evaluation as a whole. Operation DAGs are hard to scissor out
when the LHS and RHS evaluation are lowered in the same block. The pass
dealing with further forall/where lowering in HLFIR will need to
succeed. It is not acceptable for them to fail splitting the RHS/LHS
evaluation code. Keeping them in independent block is an approach that
cannot fail.

For user defined assignments, having a region allows implementing all
the call details in lowering, and even to allow inlining of the user
assignment, before it is decided if a temporary for the LHS or RHS is
required or not.

The operation description mention "hlfir.elemental_addr" (operation that
will be used for vector subscripted LHS) and "ordered assignment trees"
(concept/inetrface that will be used to represent forall/where structure
in HLFIR). These will be pushed in follow-up patch, but I do not want t
scissor out the descriptions.

Differential Revision: https://reviews.llvm.org/D149442

Small split change from D146023.

Migrate elf-notes to v4 and fix llvm-readobj to work with PAL metadata.

Reviewed By: kzhuravl

Differential Revision: https://reviews.llvm.org/D146119

There was a discrepancy where the flag was honored when passed through the
command line, but not when passed through the API, which was leading to a
python test failing.

This reverts commit 63f0fdc2.

Since f1431bbf, this environment
variable is always set up by lit itself, so individual test suites
don't need to set it.

Differential Revision: https://reviews.llvm.org/D149356

We previously used a more stricter -nostdlib option which was also removing
compiler-rt/libgcc.

Reviewed By: jhuber6

Differential Revision: https://reviews.llvm.org/D149683

llvm::CodeGen was missing so add them to deps

Reviewed By: csigg

Differential Revision: https://reviews.llvm.org/D149720

Some mid-air collision between a change in the generic format and this
new python test.

Element-wise log(exp) does no operation so that we can fold it into no-op effectively.

Reviewed By: eric-k256

Differential Revision: https://reviews.llvm.org/D149632

We don't have i1 vector shuffle lowering.

This reverts part of D149033 and  rG8f966cedea594d9a91e585e88a80a42c04049e6c. The added test case
is kept to avoid future regression.

Reviewed By: vzakhari, vdonaldson

Differential Revision: https://reviews.llvm.org/D149639

This is part of an on-going migration to adopt Properties inside MLIR.

Differential Revision: https://reviews.llvm.org/D148298

This is part of an on-going migration to adopt Properties inside MLIR.

Differential Revision: https://reviews.llvm.org/D148297

This is part of an on-going migration to adopt Properties inside MLIR.

Differential Revision: https://reviews.llvm.org/D148296

In head comment of CurPPLexer field of class Preprocessor,
'The current top of the stack what we're lexing from' should be
'The current top of the stack that we're lexing from'.

Differential Revision: https://reviews.llvm.org/D149709

Reviewed By: arsenm

Differential Revision: https://reviews.llvm.org/D149568

This should've been part of 8221c316.

This adds all the CodeGen deps all over the place.

I ran

    git show 9cfeba5b > foo2.txt

to get the original patch into a text file and then ran

    #!/usr/bin/env python3
    import os
    in_cmake = False
    for l in open('foo2.txt'):
      if l.startswith('+++ b/'):
        cmake = l[len('+++ b/'):-1]
        in_cmake = 'CMakeLists.txt' in cmake
      if not in_cmake:
        continue
      prefix = 'llvm/utils/gn/secondary/'
      gn_file = os.path.join(prefix, os.path.dirname(cmake), 'BUILD.gn')
      if l.startswith('+ '):
        add = l[1:].strip()
        if add == 'CodeGen':
          try:
            with open(gn_file) as f:
                contents = f.read()
          except:
            print(f'skipping {gn_file}')
            continue
          contents = contents.replace(' deps = [', ' deps = ["//llvm/lib/CodeGen",')
          with open(gn_file, 'w') as f:
              f.write(contents)

to update all the GN files.

(I manually removed the dep on CodeGen that this added to llvm-min-tblgen.)

Finally, I ran

    git ls-files '*.gn' '*.gni' | xargs llvm/utils/gn/gn.py format

to fix up the formatting.

I ran:

    git ls-files '*.gn' '*.gni' | xargs llvm/utils/gn/gn.py format

instruction CSE'ed another.

- This should be a part of D148877. Before that patch, !prof is not added to known-id-set [1], and turns out unknown types of metadata are dropped in the implementation [2].
  - This test is mainly added to make sure there won't be regressions for this kind of pattern. The pattern is observed it in application code; looks like the result of indirect call is used as function arguments initially; after the function is inlined load-after-store CSE opportunity is exposed.

  [1] https://github.com/llvm/llvm-project/blob/f478721231bdb71ba8f0f6fb21673b9b7f652add/llvm/lib/Transforms/Utils/Local.cpp#L2727-L2741
  [2] https://github.com/llvm/llvm-project/blob/ade3c6a6a88ed3a9b06c076406f196da9d3cc1b9/llvm/lib/Transforms/Utils/Local.cpp#L2639

Differential Revision: https://reviews.llvm.org/D149396

Differential Revision: https://reviews.llvm.org/D149390