Problem:
PCM file includes references to all module maps used in compilation which created PCM. This problem leads to PCM-rebuilds in distributed compilations as some module maps could be missing in isolated compilation. (For example in our distributed build system we create a temp folder for every compilation with only modules and headers that are needed for that particular command).

Solution:
Add only affecting module map files to a PCM-file.

Reviewed By: rsmith

Differential Revision: https://reviews.llvm.org/D106876

Reviewed By: jsji

Differential Revision: https://reviews.llvm.org/D113654

Merge definition visibility the same way we do for other decls. Without
the fix the added test emits `-Wobjc-method-access` as it cannot find a
visible protocol. Make this warning `-Werror` so the test would fail
when protocol visibility regresses.

rdar://83600696

Differential Revision: https://reviews.llvm.org/D111860

Revert "[Clang/Test]: Rename enable_noundef_analysis to disable-noundef-analysis and turn it off by default"

This reverts commit aacfbb95.

Revert "Fix lit test failures in CodeGenCoroutines"

This reverts commit 63fff0f5.

Turning on `enable_noundef_analysis` flag allows better codegen by removing freeze instructions.
I modified clang by renaming `enable_noundef_analysis` flag to `disable-noundef-analysis` and turning it off by default.

Test updates are made as a separate patch: D108453

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D105169

[Clang/Test]: Rename enable_noundef_analysis to disable-noundef-analysis and turn it off by default (2)

This patch updates test files after D105169.
Autogenerated test codes are changed by `utils/update_cc_test_checks.py,` and non-autogenerated test codes are changed as follows:

(1) I wrote a python script that (partially) updates the tests using regex: {F18594904} The script is not perfect, but I believe it gives hints about which patterns are updated to have `noundef` attached.

(2) The remaining tests are updated manually.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D108453

Resolve lit failures in clang after 8ca4b3ef's land

Fix lit test failures in clang-ppc* and clang-x64-windows-msvc

Fix missing failures in clang-ppc64be* and retry fixing clang-x64-windows-msvc

Fix internal_clone(aarch64) inline assembly

Revert "[Clang/Test]: Rename enable_noundef_analysis to disable-noundef-analysis and turn it off by default"

This reverts commit 7584ef76.

Turning on `enable_noundef_analysis` flag allows better codegen by removing freeze instructions.
I modified clang by renaming `enable_noundef_analysis` flag to `disable-noundef-analysis` and turning it off by default.

Test updates are made as a separate patch: D108453

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D105169

For each selector encountered in the source code, we need to load
selectors from the imported modules and check that we are calling a
selector with compatible types.

At the moment, for each module we are storing methods declared in the
headers belonging to this module and methods from the transitive closure
of imported modules. When a module is imported by a few other modules,
methods from the shared module are duplicated in each importer. As the
result, we can end up with lots of identical methods that we try to add
to the global method pool. Doing this duplicate work is useless and
relatively expensive.

Avoid processing duplicate methods by storing in each module only its
own methods and not storing methods from dependencies. Collect methods
from dependencies by walking the graph of module dependencies.

The issue was discovered and reported by Richard Howell. He has done the
hard work for this fix as he has investigated and provided a detailed
explanation of the performance problem.

Differential Revision: https://reviews.llvm.org/D110123

There's a nuanced check about when to use suffixes on these integer
non-type-template-parameters, but when rebuilding names for
-gsimple-template-names there isn't enough data in the DWARF to
determine when to use suffixes or not. So turn on suffixes always to
make it easy to match up names in llvm-dwarfdump --verify.

I /think/ if we correctly modelled auto non-type-template parameters
maybe we could put suffixes only on those. But there's also some logic
in Clang that puts the suffixes on overloaded functions - at least
that's what the parameter says (see D77598 and printTemplateArguments
"TemplOverloaded" parameter) - but I think maybe it's for anything that
/can/ be overloaded, not necessarily only the things that are overloaded
(the argument value is hardcoded at the various callsites, doesn't seem
to depend on overload resolution/searching for overloaded functions). So
maybe with "auto" modeled more accurately, and differentiating between
function templates (always using type suffixes there) and class/variable
templates (only using the suffix for "auto" types) we could correctly
use integer type suffixes only in the minimal set of cases.

But that seems all too much fuss, so let's just put integer type
suffixes everywhere always in the debug info of integer non-type
template parameters in template names.

(more context:
* https://reviews.llvm.org/D77598#inline-1057607
* https://groups.google.com/g/llvm-dev/c/ekLMllbLIZg/m/-dhJ0hO1AAAJ )

Differential Revision: https://reviews.llvm.org/D111477

AIX and z/OS lack Objective-C support, so mark these tests as unsupported for AIX and z/OS.

This patch follows the same reasoning as D109060.

Reviewed By: jsji

Differential Revision: https://reviews.llvm.org/D112390

We keep using the first encountered definition and need to take into
account visibility from subsequent definitions. For example, if the
first definition is hidden and the second is visible, we need to make
the first one visible too.

rdar://82263843

Differential Revision: https://reviews.llvm.org/D110453

With the old approach we were updating `ObjCInterfaceType.Decl` to the
last encountered definition. But during loading modules
`ASTDeclReader::VisitObjCInterfaceDecl` keeps the *first* encountered
definition. So with multiple definitions imported there would be a
disagreement between expected definition in `ObjCInterfaceType.Decl` and
actual definition `ObjCInterfaceDecl::getDefinition` which can lead to
incorrect diagnostic.

Fix by not tracking definition in `ObjCInterfaceType` explicitly but by
getting it from redeclaration chain.

Partially reverted 919fc500 keeping the
modified test case as the correct behavior is achieved in a different
way.

Differential Revision: https://reviews.llvm.org/D110452

Based on post-commit review discussion on
2bd84938 with Richard Smith.

Other uses of forcing HasEmptyPlaceHolder to false seem OK to me -
they're all around pointer/reference types where the pointer/reference
token will appear at the rightmost side of the left side of the type
name, so they make nested types (eg: the "int" in "int *") behave as
though there is a non-empty placeholder (because the "*" is essentially
the placeholder as far as the "int" is concerned).

This was originally committed in 277623f4

Reverted in f9ad1d1c due to breakages
outside of clang - lldb seems to have some strange/strong dependence on
"char [N]" versus "char[N]" when printing strings (not due to that name
appearing in DWARF, but probably due to using clang to stringify type
names) that'll need to be addressed, plus a few other odds and ends in
other subprojects (clang-tools-extra, compiler-rt, etc).

While working on https://reviews.llvm.org/D110280 I've tried to merge
decl contexts as it seems to be correct and matching our handling of
decl contexts from different modules. It's not required for the fix in
https://reviews.llvm.org/D110280 but it revealed a missing diagnostic,
so separating this change into a separate commit.

Renamed some variables to distinguish diagnostic like "declaration of
'x' does not match" for different cases.

Differential Revision: https://reviews.llvm.org/D110287

This reverts the following commits:
37ca7a79
9aa6c72b
705387c5
8ca4b3ef
80dba72a

When using explicit Clang modules, some declarations might unexpectedly become invisible.

This is caused by the mechanism that loads PCM files passed via `-fmodule-file=<path>` and creates an `IdentifierInfo` for the module name. The `IdentifierInfo` creation takes place when the `ASTReader` is in a weird state, with modules that are loaded but not yet set up properly. This patch delays the creation of `IdentifierInfo` until the `ASTReader` is done with reading the PCM.

Note that the `-fmodule-file=<name>=<path>` form of the argument doesn't suffer from this issue, since it doesn't create `IdentifierInfo` for the module name.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D111543

[Clang/Test]: Rename enable_noundef_analysis to disable-noundef-analysis and turn it off by default (2)

This patch updates test files after D105169.
Autogenerated test codes are changed by `utils/update_cc_test_checks.py,` and non-autogenerated test codes are changed as follows:

(1) I wrote a python script that (partially) updates the tests using regex: {F18594904} The script is not perfect, but I believe it gives hints about which patterns are updated to have `noundef` attached.

(2) The remaining tests are updated manually.

Reviewed By: eugenis

Differential Revision: https://reviews.llvm.org/D108453

Mimic the behavior of including headers where a system includer makes an
includee a system header too.

rdar://84049469

Differential Revision: https://reviews.llvm.org/D111476

Looks like lldb has some issues with this - somehow it causes lldb to
treat a "char[N]" type as an array of chars (prints them out
individually) but a "char [N]" is printed as a string. (even though the
DWARF doesn't have this string in it - it's something to do with the
string lldb generates for itself using clang)

This reverts commit 277623f4.

Based on post-commit review discussion on
2bd84938 with Richard Smith.

Other uses of forcing HasEmptyPlaceHolder to false seem OK to me -
they're all around pointer/reference types where the pointer/reference
token will appear at the rightmost side of the left side of the type
name, so they make nested types (eg: the "int" in "int *") behave as
though there is a non-empty placeholder (because the "*" is essentially
the placeholder as far as the "int" is concerned).

During explicit modular build, PCM files are typically specified via the `-fmodule-file=<path>` command-line option. Early during the compilation, Clang uses the `ASTReader` to read their contents and caches the result so that the module isn't loaded implicitly later on. A listener is attached to the `ASTReader` to collect names of the modules read from the PCM files. However, if the PCM has already been loaded previously via PCH:
1. the `ASTReader` doesn't do anything for the second time,
2. the listener is not invoked at all,
3. the module load result is not cached,
4. the compilation fails when attempting to load the module implicitly later on.

This patch solves this problem by attaching the listener to the `ASTReader` for PCH reading as well.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D111560

The following tests are failing due to missing DWARF sections. This patch sets these tests as XFAIL/DISABLED on AIX until a more permanent solution is implemented.

Reviewed By: shchenz

Differential Revision: https://reviews.llvm.org/D111336

Objective C is not supported on AIX or ZOS. This patch excludes the newly added `clang/test/Modules/merge-objc-interface.m` (added by https://reviews.llvm.org/D110280) from AIX and ZOS testing.

Many existing tests are already disabled by https://reviews.llvm.org/D109060.

Reviewed By: jsji

Differential Revision: https://reviews.llvm.org/D111406

When have ObjCInterfaceDecl with the same name in 2 different modules,
hitting the assertion

> Assertion failed: (Index < RL->getFieldCount() && "Ivar is not inside record layout!"),
> function lookupFieldBitOffset, file llvm-project/clang/lib/AST/RecordLayoutBuilder.cpp, line 3434.

on accessing an ivar inside a method. The assertion happens because
ivar belongs to one module while its containing interface belongs to
another module and then we fail to find the ivar inside the containing
interface. We already keep a single ObjCInterfaceDecl definition in
redecleration chain and in this case containing interface was correct.
The issue is with ObjCIvarDecl. IVar decl for IRGen is taken from
ObjCIvarRefExpr that is created in `Sema::BuildIvarRefExpr` using ivar
decl returned from `Sema::LookupIvarInObjCMethod`. And ivar lookup
returns a wrong decl because basically we take the first ObjCIvarDecl
found in `ASTReader::FindExternalVisibleDeclsByName` (called by
`DeclContext::lookup`). And in `ASTReader.Lookups` lookup table for a
wrong module comes first because `ASTReader::finishPendingActions`
processes `PendingUpdateRecords` in reverse order and the first
encountered ObjCIvarDecl will end up the last in `ASTReader.Lookups`.

Fix by merging ObjCIvarDecl from different modules correctly and by
using a canonical one in IRGen.

rdar://82854574

Differential Revision: https://reviews.llvm.org/D110280

AIX and z/OS lack Objective-C support, so mark these tests as unsupported for AIX and z/OS.

Reviewed By: jsji

Differential Revision: https://reviews.llvm.org/D109060

This patch increases the expected line number for one of the checks so that it doesn't have to be updated for any added/removed lines in the RUN section.

This change is in preparation for the following patch: https://reviews.llvm.org/D109060

Reviewed By: jsji

Differential Revision: https://reviews.llvm.org/D109541

When deserializing a RecordDecl we don't enforce that redeclaration
chain contains only a single definition. So if the canonical decl is not
a definition itself, `RecordType::getDecl` can return different objects
before and after an include. It means we can build CGRecordLayout for
one RecordDecl with its set of FieldDecl but try to use it with
FieldDecl belonging to a different RecordDecl. With assertions enabled
it results in

> Assertion failed: (FieldInfo.count(FD) && "Invalid field for record!"),
> function getLLVMFieldNo, file llvm-project/clang/lib/CodeGen/CGRecordLayout.h, line 199.

and with assertions disabled a bunch of fields are treated as their
memory is located at offset 0.

Fix by keeping the first encountered RecordDecl definition and marking
the subsequent ones as non-definitions. Also need to merge FieldDecl
properly, so that `getPrimaryMergedDecl` works correctly and during name
lookup we don't treat fields from same-name RecordDecl as ambiguous.

rdar://80184238

Differential Revision: https://reviews.llvm.org/D106994

@arichardson pointed out in post-commit review for
https://reviews.llvm.org/D95583 (b714f73d) that `-verify` has an
optional argument that works a lot like `FileCheck`'s `-check-prefix`.
Use it to simplify the test for `-fno-implicit-modules-use-lock`!

Add -cc1 flags `-fmodules-uses-lock` and `-fno-modules-uses-lock` to
allow the lock manager to be turned off when building implicit modules.

Add `-Rmodule-lock` so that we can see when it's being used.

Differential Revision: https://reviews.llvm.org/D95583

D31709 added an assertion was added to `FullSourceLoc::hasManager()` that ensured a valid `SourceLocation` is always paired with a `SourceManager`, and missing `SourceManager` is always paired with an invalid `SourceLocation`.

This appears to be incorrect, since clients never cared about constructing `FullSourceLoc` to uphold that invariant, or always checking `isValid()` before calling `hasManager()`.

The assertion started failing when serializing diagnostics pointing into an explicit module. Explicit modules don't have valid `SourceLocation` for the `import` statement, since they are "imported" from the command-line argument `-fmodule-name=x.pcm`.

This patch removes the assertion, since `FullSourceLoc` was never intended to uphold any kind of invariants between the validity of `SourceLocation` and presence of `SourceManager`.

Reviewed By: arphaman

Differential Revision: https://reviews.llvm.org/D106862

For some use-cases, it might be useful to be able to turn off modules for C++ in `-cc1`. (The feature is implied by `-std=C++20`.)

This patch exposes the `-fno-cxx-modules` option in `-cc1`.

Reviewed By: arphaman

Differential Revision: https://reviews.llvm.org/D106864

Fixes a false-positive error if the same std::variant<...> type is
instantiated across multiple modules.

It's useful to be able to load explicitly-built PCH files into an implicit build (e.g. during dependency scanning). That's currently impossible, since the explicitly-built PCH has an empty modules cache path, while the current compilation has (and needs to have) a valid path, triggering an error in the `PCHValidator`.

This patch adds a preprocessor option and command-line flag that can be used to omit this check.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D103802

Prior to this patch when you used `clang -module-file-info` clang would
delete the module on completion because the module was treated as an
output file.

This fixes the issue so you don't need to invoke cc1 directly to get
module file information.

Reviewed By: steven_wu, phosek

Differential Revision: https://reviews.llvm.org/D103547

This patch enables explicitly building inferred modules.

Effectively a cherry-pick of https://github.com/apple/llvm-project/pull/699 authored by @Bigcheese with libclang and dependency scanner changes omitted.

Contains the following changes:

1. [Clang] Fix the header paths in clang::Module for inferred modules.
  * The UmbrellaAsWritten and NameAsWritten fields in clang::Module are a lie for framework modules. For those they actually are the path to the header or umbrella relative to the clang::Module::Directory.
  * The exception to this case is for inferred modules. Here it actually is the name as written, because we print out the module and read it back in when implicitly building modules. This causes a problem when explicitly building an inferred module, as we skip the printing out step.
  * In order to fix this issue this patch adds a new field for the path we want to use in getInputBufferForModule. It also makes NameAsWritten actually be the name written in the module map file (or that would be, in the case of an inferred module).

2. [Clang] Allow explicitly building an inferred module.
  * Building the actual module still fails, but make sure it fails for the right reason.

Split from D100934.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D102491

If a module contains errors (ie. it was built with
-fallow-pcm-with-compiler-errors and had errors) and was from the module
cache, it is marked as out of date - see
a2c1054c.

When a module is imported multiple times in the one compile, this caused
it to be recompiled each time - removing the existing buffer from the
module cache and replacing it. This results in various errors further
down the line.

Instead, only mark the module as out of date if it isn't already
finalized in the module cache.

Reviewed By: akyrtzi

Differential Revision: https://reviews.llvm.org/D100619

This reverts commit c965e14a.