Recent change enable dropping unit-trip loops of "reduction" iterator
type as well. This is fine as long as there is one other "reduction"
iterator in the operation. Without this the initialized value (value
of `out`) is not read which leads to a correctness issue.

Also fix a bug in the `fill` -> `tensor_reshape` folding. The `out`
operand of the `fill` needs to be reshaped to get the `out` operand of
the generated `fill` operation.

Differential Revision: https://reviews.llvm.org/D100145

Lowerings tosa.max_pool2d to linalg equivalent operations. Includes
adding max pooling operations for linalg, with corresponding tests.

Differential Revision: https://reviews.llvm.org/D99824

This patch doesn't support the optional operands of ImageDrefGather. The support of optional operands will be implemented later.

co-authered-by: Alan Liu <alanliu.yf@gmail.com>

Differential Revision: https://reviews.llvm.org/D100128

Dialects can be translated to LLVM IR when they have the
LLVMTranslationDialectInterface interface registered. In case the
interface isn't explicitly registered, even the LLVM dialect can't be
exported to LLVM IR. This make the error message more explicit on this.

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D96729

This patch unconditionally converts i1 types to i8 types on memrefs. If the
extensions or capabilities are not met, they will be converted to i32. Hence the
logic in IntLoadPattern and IntStorePattern are also updated.

Also added the implementation of SPIRVTypeConverter::getOptions().

Reviewed By: antiagainst

Differential Revision: https://reviews.llvm.org/D99724

Reviewed By: ftynse

Differential Revision: https://reviews.llvm.org/D100114

Non-32-bit scalar types require special hardware support that may not
exist on all GPUs. This is reflected in SPIR-V as that non-32-bit scalar
types require special capabilities or extensions.

Previously when there is a non-32-bit type and no native support, we
unconditionally emulate it with 32-bit ones. This isn't good given that
it can have implications over ABI and data layout consistency.

This commit introduces an option to control whether to use 32-bit
types to emulate.

Differential Revision: https://reviews.llvm.org/D100059

Per the TypeConverter API contract, returning `llvm:None` means
other conversion rules should be tried. But we only have one
rule per input type. So there is no need to try others and we can
just directly fail, which should return `nullptr`. This avoids
unnecessary checks.

Differential Revision: https://reviews.llvm.org/D100058

The patch enables the use of index type in vectors. It is a prerequisite to support vectorization for indexed Linalg operations. This refactoring became possible due to the newly introduced data layout infrastructure. The data layout of a module defines the bitwidth of the index type needed to verify bitcasts and similar vector operations.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D99948

When allocLikeOp is updated in alloc constant folding,
alighnment attribute was ignored. This patch fixes it.

Signed-off-by: Haruki Imai <imaihal@jp.ibm.com>

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D99882

Some sparse matrices operate on integral values (in contrast with the common
f32 and f64 values). This CL expands the compiler and runtime support to deal
with several common type combinations.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D99999

Also factors out out-of-bounds mask generation from vector.transfer_read/write into a new MaterializeTransferMask pattern.

Differential Revision: https://reviews.llvm.org/D100001

When both arguments are the same for these ops, propagate this argument.

Add the `getCapsule()` and `createFromCapsule()` methods to the PyOperation class.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D99927

Table op lowering to linalg.generic for both i8 (behaves like a gather) and a
pair of gathers with a quantized interpolation.

Differential Revision: https://reviews.llvm.org/D99756

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D99954

Linalg fusion on tensors has mismatching assumptions on the operand side than on the region bbArg side.
Relax the behavior on the operand/indexing map side so that we better support output operands that may also be read from.

Differential revision: https://reviews.llvm.org/D99499

The existing implementation was always creating 32-bit constants for
floating-point and integer reductions regardless of the actual type, which
resulted in invalid IR being generated for any types other than f32 and i32
when lowering affine.parallel to SCF. Use the actual type instead.

Reviewed By: chelini

Differential Revision: https://reviews.llvm.org/D99942

Right now Elementwise operations fusion in Linalg fuses everything it
can. This can run up against resource limits of the target hardware
without some checks. This patch adds a callback function that clients
can use to implement a cost function. When two elementwise operations
are deemed structurally fusable, the callback can be used to control
if the fusion applies.

Differential Revision: https://reviews.llvm.org/D99820

The moved `populate` methods are only relevant to Linalg
operations. So they are better of in `linalg` namespace.  Also rename
`populateLinalgTensorOpsFusionPatterns` to
`populateElementwiseOpsFusionPatterns`. This makes the scope of these
patterns explicit and disambiguates it with fusion on tensors using
tile + fuse.

Differential Revision: https://reviews.llvm.org/D99819

NDEBUG builds currently warn because it's only used inside an assert.

Reviewed By: mehdi_amini

Differential Revision: https://reviews.llvm.org/D99721

Added lowerings for Tosa's reduce boolean operations. This includes a fix to
maintain the output rank of reduce operations.

Reviewed By: silvas

Differential Revision: https://reviews.llvm.org/D99228

`ConversionPatternRewriter::applySignatureConversion` did not have a way
to apply a signature conversion that involved materializations.

Differential Revision: https://reviews.llvm.org/D99782

This was missed when verification for creating unregistered operations was added.

Differential Revision: https://reviews.llvm.org/D99684

This commit add utility functions for creating push constant
storage variable and loading values from it.

Along the way, performs some clean up:

* Deleted `setABIAttrs`, which is just a 4-liner function
  with one user.
* Moved `SPIRVConverstionTarget` into `mlir` namespace,
  to be consistent with `SPIRVTypeConverter` and
  `LLVMConversionTarget`.

Reviewed By: mravishankar

Differential Revision: https://reviews.llvm.org/D99725

Rationale:
Small indices and values, when allowed by the required range of the
input tensors, can reduce the memory footprint of sparse tensors
even more. Note, however, that we must be careful zero extending
the values (since sparse tensors never use negatives for indexing),
but LLVM treats the index type as signed in most memory operations
(like the scatter and gather). This CL dots all the i's in this regard.

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D99777

Setting the builder from a block is looking up for a parent operation
to get a context, instead by setting up the builder with an explicit
context we can support invoking this helper in absence of a parent
operation.

Differential Revision: https://reviews.llvm.org/D99229

Fixes a bug in affine fusion pipeline where an incorrect slice is computed.
After the slice computation is done, original domain of the the source is
compared with the new domain that will result if the fusion succeeds. If the
new domain must be a subset of the original domain for the slice to be
valid. If the slice computed is incorrect, fusion based on such a slice is
avoided.

Relevant test cases are added/edited.

Fixes https://bugs.llvm.org/show_bug.cgi?id=49203

Differential Revision: https://reviews.llvm.org/D98239

This revision tightens up the handling of attributes for both named
and generic linalg ops.
To demonstrate the IR validity, a working e2e Linalg example is added.

Differential Revision: https://reviews.llvm.org/D99430

Lowering reciprocal and sigmoid elementwise operations to linalg dialect.

Reviewed By: rsuderman

Differential Revision: https://reviews.llvm.org/D99676

If an operation has been inserted as a key in to the known values
hashtable, then it can not be modified in a way which changes its hash.
This change avoids modifying the operands of any previously recorded
operation, which prevents their hash from changing.

In an SSACFG region, it is impossible to visit an operation before
visiting its operands, so this is not a problem. This situation can only
happen in regions without strict dominance, such as graph regions.

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D99486

This is in preparation for adding a new "mask" operand. The existing "masked" attribute was used to specify dimensions that may be out-of-bounds. Such transfers can be lowered to masked load/stores. The new "in_bounds" attribute is used to specify dimensions that are guaranteed to be within bounds. (Semantics is inverted.)

Differential Revision: https://reviews.llvm.org/D99639

This revision adds support to properly add the body of registered
builtin named linalg ops.
At this time, indexing_map and iterator_type support is still
missing so the op is not executable yet.

Differential Revision: https://reviews.llvm.org/D99578

This reverts commit 883912ab.

* Suggested to be broken out of D99578

Differential Revision: https://reviews.llvm.org/D99638

This allows for the conversion to match `A(B()) -> C()` with a pattern matching
`A` and marking `B` for deletion.

Also add better assertions when an operation is erased while still having uses.

Differential Revision: https://reviews.llvm.org/D99442

Add a "register_runtime" method to the mlir.execution_engine and show calling back from MLIR into Python

This exposes the ability to register Python functions with the JIT and
exposes them to the MLIR jitted code. The provided test case illustrates
the mechanism.

Differential Revision: https://reviews.llvm.org/D99562