<rdar://problem/13037793> Allow the names of modules to differ from the name of their subdirectory in the include path.

llvm-svn: 177621

<rdar://problem/12368093> Extend module maps with a 'conflict' declaration, and warn when a newly-imported module conflicts with an already-imported module.

llvm-svn: 177577

Configuration macros are macros that are intended to alter how a
module works, such that we need to build different module variants
for different values of these macros. A module can declare its
configuration macros, in which case we will complain if the definition
of a configation macro on the command line (or lack thereof) differs
from the current preprocessor state at the point where the module is
imported. This should eliminate some surprises when enabling modules,
because "#define CONFIG_MACRO ..." followed by "#include
<module/header.h>" would silently ignore the CONFIG_MACRO setting. At
least it will no longer be silent about it.

Configuration macros are eventually intended to help reduce the number
of module variants that need to be built. When the list of
configuration macros for a module is exhaustive, we only need to
consider the settings for those macros when building/finding the
module, which can help isolate modules for various project-specific -D
flags that should never affect how modules are build (but currently do).

llvm-svn: 177466

<rdar://problem/13363214> Eliminate race condition between module rebuild and the global module index.

The global module index was querying the file manager for each of the
module files it knows about at load time, to prune out any out-of-date
information. The file manager would then cache the results of the
stat() falls used to find that module file.

Later, the same translation unit could end up trying to import one of the
module files that had previously been ignored by the module cache, but
after some other Clang instance rebuilt the module file to bring it
up-to-date. The stale stat() results in the file manager would
trigger a second rebuild of the already-up-to-date module, causing
failures down the line.

The global module index now lazily resolves its module file references
to actual AST reader module files only after the module file has been
loaded, eliminating the stat-caching race. Moreover, the AST reader
can communicate to its caller that a module file is missing (rather
than simply being out-of-date), allowing us to simplify the
module-loading logic and allowing the compiler to recover if a
dependent module file ends up getting deleted.

llvm-svn: 177367

This commit introduces a set of related changes to ensure that the
declaration that shows up in the identifier chain after deserializing
declarations with a given identifier is, in fact, the most recent
declaration. The primary change involves waiting until after we
deserialize and wire up redeclaration chains before updating the
identifier chains. There is a minor optimization in here to avoid
recursively deserializing names as part of looking to see whether
top-level declarations for a given name exist.

A related change that became suddenly more urgent is to property
record a merged declaration when an entity first declared in the
current translation unit is later deserialized from a module (that had
not been loaded at the time of the original declaration). Since we key
off the canonical declaration (which is parsed, not from an AST file)
for emitted redeclarations, we simply record this as a merged
declaration during AST writing and let the readers merge them.

Re-fixes <rdar://problem/13189985>, presumably for good this time.

llvm-svn: 175447

llvm-svn: 175027

Order the methods in the global method pool based on when they become visible, not when they become deserialized <rdar://problem/13203033>.

llvm-svn: 175018

the linkage of functions and variables while merging declarations from modules,
and we don't necessarily have enough of the rest of the AST loaded at that
point to allow us to compute linkage, so serialize it instead.

llvm-svn: 174943

lexical storage but not visible storage' case in C++. It's unclear whether we
even need the special-case handling for C++, since it seems to be working
around our not serializing a lookup table for the TU in C. But in any case,
the assertion is incorrect.

llvm-svn: 174931

[Modules] Cope better with top-level declarations loaded after being declared in the current translation unit <rdar://problem/13189985>.

These two related tweaks to keep the information associated with a
given identifier correct when the identifier has been given some
top-level information (say, a top-level declaration) and more
information is then loaded from a module. The first ensures that an
identifier that was "interesting" before being loaded from an AST is
considered to be different from its on-disk counterpart. Otherwise, we
lose such changes when writing the current translation unit as a
module.

Second, teach the code that injects AST-loaded names into the
identifier chain for name lookup to keep the most recent declaration,
so that we don't end up confusing our declaration chains by having a
different declaration in there.

llvm-svn: 174895

visible.

The basic problem here is that a given translation unit can use
forward declarations to form pointers to a given type, say,

  class X;
  X *x;

and then import a module that includes a definition of X:

  import XDef;

We will then fail when attempting to access a member of X, e.g., 

  x->method()

because the AST reader did not know to look for a default of a class
named X within the new module.

This implementation is a bit of a C-centric hack, because the only
definitions that can have this property are enums, structs, unions,
Objective-C classes, and Objective-C protocols, and all of those are
either visible at the top-level or can't be defined later. Hence, we
can use the out-of-date-ness of the name and the identifier-update
mechanism to force the update.

In C++, we will not be so lucky, and will need a more advanced
solution, because the definitions could be in namespaces defined in
two different modules, e.g.,

  // module 1
  namespace N { struct X; }

  // module 2
  namespace N { struct X { /* ... */ }; }

One possible implementation here is for C++ to extend the information
associated with each identifier table to include the declaration IDs
of any definitions associated with that name, regardless of
context. We would have to eagerly load those definitions.

llvm-svn: 174794

included in the same test. Clang gets confused about whether it's already built
a module for this file, when running on a content-addressible filesystem.

llvm-svn: 174694

Fix conflict between r174685 and r174645 (rename -fmodule-cache-path <foo> to -fmodules-cache-path=<foo>).

llvm-svn: 174690

overloads of a name by claiming that there are no lookup results for that name
in modules while loading the names from the module. Lookups in deserialization
really don't want to find names which they themselves are in the process of
introducing. This also has the pleasant side-effect of automatically caching
PCH lookups which found no names.

The runtime here is still quadratic in the number of overloads, but the
constant is lower.

llvm-svn: 174685

llvm-svn: 174683

llvm-svn: 174674

llvm-svn: 174649

Retain all hidden methods in the global method pool, because they may become visible <rdar://problem/13172858>.

llvm-svn: 174648

llvm-svn: 174645

name lookup has been performed in that context (this probably only happens in
C++).

1) Whenever we add names to a context, set a flag on it, and if we perform
lookup and discover that the context has had a lookup table built but has the
flag set, update all entries in the lookup table with additional names from
the external source.

2) When marking a DeclContext as having external visible decls, mark the
context in which lookup is performed, not the one we are adding. These won't
be the same if we're adding another copy of a pre-existing namespace.

llvm-svn: 174577

Be a little more permissive with -fmodules-ignore-macro= by removing everything after the second '=' if it is there.

llvm-svn: 174567

The use of this flag enables a modules optimization where a given set
of macros can be labeled as "ignored" by the modules
system. Definitions of those macros will be completely ignored when
building the module hash and will be stripped when actually building
modules. The overall effect is that this flag can be used to
drastically reduce the number of

Eventually, we'll want modules to tell us what set of macros they
respond to (the "configuration macros"), and anything not in that set
will be excluded. However, that requires a lot of per-module
information that must be accurate, whereas this option can be used
more readily.

Fixes the rest of <rdar://problem/13165109>.

llvm-svn: 174560

This can happen when one abuses precompiled headers by passing more -D
options when using a precompiled hedaer than when it was built. This
is intentionally permitted by precompiled headers (and is exploited by
some build environments), but causes problems for modules.

First part of <rdar://problem/13165109>, detecting when something when
horribly wrong.

llvm-svn: 174554

Different modules may have different views of the various "special"
types in the AST, such as the redefinition type for "id". Merge those
types rather than only considering the redefinition types for the
first AST file loaded.

llvm-svn: 174234

-fno-modules-global-index -cc1 option to allow one to disable the
index for performance testing purposes, but with a 10% win in
-fsyntax-only time, there is no reason a user would do this.

llvm-svn: 173707

llvm-svn: 173419

AST reader.

The global module index tracks all of the identifiers known to a set
of module files. Lookup of those identifiers looks first in the global
module index, which returns the set of module files in which that
identifier can be found. The AST reader only needs to look into those
module files and any module files not known to the global index (e.g.,
because they were (re)built after the global index), reducing the
number of on-disk hash tables to visit. For an example source I'm
looking at, we go from 237844 total identifier lookups into on-disk
hash tables down to 126817.

Unfortunately, this does not translate into a performance advantage.
At best, it's a wash once the global module index has been built, but
that's ignore the cost of building the global module index (which
is itself fairly large). Profiles show that the global module index
code is far less efficient than it should be; optimizing it might give
enough of an advantage to justify its continued inclusion.

llvm-svn: 173405

"-fmodules-global-index" and expand its behavior to include both the
use and generation of the global module index.

llvm-svn: 173404

FIXME: PathV2::unique_file() is assumed to open the file with binary mode on win32.
llvm-svn: 173330

llvm-svn: 173326

The global module index is a "global" index for all of the module
files within a particular subdirectory in the module cache, which
keeps track of all of the "interesting" identifiers and selectors
known in each of the module files. One can perform a fast lookup in
the index to determine which module files will have more information
about entities with a particular name/selector. This information can
help eliminate redundant lookups into module files (a serious
performance problem) and help with creating auto-import/auto-include
Fix-Its.

The global module index is created or updated at the end of a
translation unit that has triggered a (re)build of a module by
scraping all of the .pcm files out of the module cache subdirectory,
so it catches everything. As with module rebuilds, we use the file
system's atomicity to synchronize.

llvm-svn: 173301

that redefined a macro without undef'ing it first.

Proper reconstruction of the macro info history from modules will properly fix this in subsequent commits.

rdar://13016031

llvm-svn: 173281

llvm-svn: 172695

in the LangRef).

llvm-svn: 172692

llvm-svn: 172689

undefined, and don't find methods or protocols within those protocol
definitions. This completes <rdar://problem/10634711>.

llvm-svn: 172686

consider (sub)module visibility.

The bulk of this change replaces myriad hand-rolled loops over the
linked list of Objective-C categories/extensions attached to an
interface declaration with loops using one of the four new category
iterator kinds:

  visible_categories_iterator: Iterates over all visible categories
  and extensions, hiding any that have their "hidden" bit set. This is
  by far the most commonly used iterator.

  known_categories_iterator: Iterates over all categories and
  extensions, ignoring the "hidden" bit. This tends to be used for
  redeclaration-like traversals.

  visible_extensions_iterator: Iterates over all visible extensions,
  hiding any that have their "hidden" bit set.

  known_extensions_iterator: Iterates over all extensions, whether
  they are visible to normal name lookup or not.

The effect of this change is that any uses of the visible_ iterators
will respect module-import visibility. See the new tests for examples.

Note that the old accessors for categories and extensions are gone;
there are *Raw() forms for some of them, for those (few) areas of the
compiler that have to manipulate the linked list of categories
directly. This is generally discouraged.

Part two of <rdar://problem/10634711>.
 

llvm-svn: 172665

when the methods are declared in a submodule that has not yet been
imported. Part of <rdar://problem/10634711>. 

llvm-svn: 172635

users can explicitly enable/disable modules autolinking.

llvm-svn: 172592

module-import dependencies, so we'll get the link order correct for
those silly linkers that need it.

llvm-svn: 172459