llvm-svn: 314373

Refactor MacroArgs to use TrailingObjects when creating a variably sized object on the heap to store the unexpanded tokens immediately after the MacroArgs object.

llvm-svn: 314372

[NFC] Refactor the Preprocessor function that handles Macro definitions and rename Arguments to Parameters in Macro Definitions. 
  - Extracted the reading of the tokens out into a separate function.
  - Replace 'Argument' with 'Parameter' when referring to the identifiers of the macro definition (as opposed to the supplied arguments - MacroArgs - during the macro invocation).

This is in preparation for submitting patches for review to implement __VA_OPT__ which will otherwise just keep lengthening the HandleDefineDirective function and making it less comprehensible.

I will also directly update some extra clang tooling that is broken by the change from Argument to Parameter.

Hopefully the bots will stay appeased.

Thanks!

llvm-svn: 308190

Revert changes from my previous refactoring - will need to fix dependencies in clang's extra tooling (such as clang-tidy etc.).

Sorry about that.

llvm-svn: 308158

[NFC] Refactor the Preprocessor function that handles Macro definitions and rename Arguments to Parameters in Macro Definitions. 
  - Extracted the reading of the tokens out into a separate function.
  - Replace 'Argument' with 'Parameter' when referring to the identifiers of the macro definition (as opposed to the supplied arguments - MacroArgs - during the macro invocation).

This is in preparation for submitting patches for review to implement __VA_OPT__ which will otherwise just keep lengthening the HandleDefineDirective function and making it less comprehensible.


Thanks!

llvm-svn: 308157

correct getNumArguments

StringifiedArguments is allocated (resized) based on the size the 
getNumArguments function. However, this function ACTUALLY currently 
returns the amount of total UnexpArgTokens which is minimum the same as 
the new implementation of getNumMacroArguments, since empty/omitted arguments 
result in 1 UnexpArgToken, and included ones at minimum include 2 
(1 for the arg itself, 1 for eof).

This patch renames the otherwise unused getNumArguments to be more clear 
that it is the number of arguments that the Macro expects, and thus the maximum 
number that can be stringified. This patch also replaces the explicit memset 
(which results in value instantiation of the new tokens, PLUS clearing the 
memory) with brace initialization.

Differential Revision: https://reviews.llvm.org/D32046

llvm-svn: 305425

Modules builds fundamentally have a non-linear macro history. In the interest
of better source fidelity, represent the macro definition information
faithfully: we have a linear macro directive history within each module, and at
any point we have a unique "latest" local macro directive and a collection of
visible imported directives. This also removes the attendent complexity of
attempting to create a correct MacroDirective history (which we got wrong
in the general case).

No functionality change intended.

llvm-svn: 236176

llvm-svn: 221576

We would crash because we used operator[] to access past the end of a
SmallString.  This occured because our token had length zero.

Instead, form the pointer using .data() and arithmetic.  This is safe
because this forms a one-past-the-end pointer and it is only used to
compare with another one-past-the-end pointer.

This fixes PR21379.

llvm-svn: 220614

llvm-svn: 209083

llvm-svn: 208381

llvm-svn: 208380

[Preprocessor] For the MacroExpands preprocessor callback, also pass the MacroArgs object that provides information about
the argument tokens for a function macro.

llvm-svn: 181065

handle raw string literals here. C++11 doesn't yet specify how they will
behave, but discussion on core suggests that we should just strip off
everything but the r-char-sequence.

llvm-svn: 176779

brought into 'clang' namespace by clang/Basic/LLVM.h

llvm-svn: 172323

uncovered.

This required manually correcting all of the incorrect main-module
headers I could find, and running the new llvm/utils/sort_includes.py
script over the files.

I also manually added quite a few missing headers that were uncovered by
shuffling the order or moving headers up to be main-module-headers.

llvm-svn: 169237

llvm-svn: 164555

If we are pre-expanding a macro argument don't actually "activate"
the pragma at that point, activate the pragma whenever we encounter
it again in the token stream.
This ensures that we will activate it in the correct location
or that we will ignore it if it never enters the token stream, e.g:

     \#define EMPTY(x)
     \#define INACTIVE(x) EMPTY(x)
     INACTIVE(_Pragma("clang diagnostic ignored \"-Wconversion\""))

This also fixes the crash in rdar://11168596.

llvm-svn: 153959

(I was going to fix the TODO about DenseMap too, but
that would break self-host right now. See PR11922.)

llvm-svn: 149799

Fix all the transitive include users.

llvm-svn: 149783

llvm-svn: 141012

llvm-svn: 140288

Fixes http://llvm.org/PR10826.

llvm-svn: 139087

llvm-svn: 136210

When a macro instantiation occurs, reserve a SLocEntry chunk with length the
full length of the macro definition source. Set the spelling location of this chunk
to point to the start of the macro definition and any tokens that are lexed directly
from the macro definition will get a location from this chunk with the appropriate offset.

For any tokens that come from argument expansion, '##' paste operator, etc. have their
instantiation location point at the appropriate place in the instantiated macro definition
(the argument identifier and the '##' token respectively).
This improves macro instantiation diagnostics:

Before:

t.c:5:9: error: invalid operands to binary expression ('struct S' and 'int')
int y = M(/);
        ^~~~
t.c:5:11: note: instantiated from:
int y = M(/);
          ^

After:

t.c:5:9: error: invalid operands to binary expression ('struct S' and 'int')
int y = M(/);
        ^~~~
t.c:3:20: note: instantiated from:
\#define M(op) (foo op 3);
                ~~~ ^  ~
t.c:5:11: note: instantiated from:
int y = M(/);
          ^

The memory savings for a candidate boost library that abuses the preprocessor are:

- 32% less SLocEntries (37M -> 25M)
- 30% reduction in PCH file size (900M -> 635M)
- 50% reduction in memory usage for the SLocEntry table (1.6G -> 800M)

llvm-svn: 134587

able to revert these based on a patch I'm working on, but no reason for
people to be spammed with warnings in the interim.

llvm-svn: 130394

recovery for those that need it.

llvm-svn: 98689

this speeds up Eonly on the testcase in PR5888 from 30.5s to 0.85s

llvm-svn: 92203

not a token number.  Fix the reserve logic to get the right
amount of space.

llvm-svn: 92202

as a character literal, not a string literal.  This might fix
rdar://7486575

llvm-svn: 92025

on PR5610 (2.185 -> 2.130s).  The big issue is that this is making
insanely huge macro argument lists with over a million tokens in it.
The reason that mallco and free are so expensive is that we are 
actually going to the kernel to get it, and switching to a bump 
pointer allocator won't change this in an interesting way.

llvm-svn: 91449

We creating and free thousands of MacroArgs objects (and the related
std::vectors hanging off them) for the testcase in PR5610 even though
there are only ~20 live at a time.  This doesn't actually use the 
cache yet.

llvm-svn: 91391

on 64-bit targets.  Pass Preprocessor into create/destroy methods of MacroArgs
even though it isn't used yet.

llvm-svn: 91345

llvm-svn: 81346

two empty arguments.  Also, add an assert so that this bug
manifests as an assertion failure, not a valgrind problem.

This fixes rdar://6880648 - [cpp] crash in ArgNeedsPreexpansion

llvm-svn: 71616

redundant #includes.  Patch by Anders Johnsen!

llvm-svn: 63271

.def file for each library.  This means that adding a diagnostic
to sema doesn't require all the other libraries to be rebuilt.

Patch by Anders Johnsen!

llvm-svn: 63111

instantiation history in an effort to speed up c99-intconst-1.c.
Now that multiple nested instantiations are allowed, we just
make them and don't pay the cost of lookups.  With the other
changes that went in before this, reverting this is actually
a speedup for c99-intconst-1.c, speeding it up from 1.96s to 1.80s,
and preserves much better loc info.

llvm-svn: 63036

Token now has a class of kinds for "literals", which include 
numeric constants, strings, etc.  These tokens can optionally have
a pointer to the start of the token in the lexer buffer.  This 
makes it faster to get spelling and do other gymnastics, because we
don't have to go through source locations.

This change is performance neutral, but will make other changes
more feasible down the road.

llvm-svn: 63028

This reduces fsyntax-only time on c99-intconst-1.c from 2.43s down to 
2.01s (20%), reducing the number of fileid lookups from 2529040 linear 
and 64771121 binary to 5625902 linear and 4151182 binary.

This knocks getFileID down to only 4.6% of compile time on this testcase.
At this point, malloc/free is over 35% of compile time, primarily allocating
MacroArgs objects and their argument preexpansion vectors.

I don't feel like malloc avoiding right now, so I'm just going to call
this good.

llvm-svn: 62994