performance gain. Here's what we see for -Eonly on Cocoa.h (using PTH):

- wall time decreases by 21% (26% speedup overall)
- system time decreases by 35%
- user time decreases by 6%

These reductions are due to not paging source files just to get spellings for
literals. The solution in place doesn't appear to be 100% yet, as we still see
some of the pages for source files getting mapped in. Using -print-stats, we see
that SourceManager maps in 7179K less bytes of source text (reduction of 75%).
Will investigate why the remaining 25% are getting paged in.

With these changes, here's how PTH compares to non-PTH on Cocoa.h:
  -Eonly: PTH takes 64% of the time as non-PTH (54% speedup)
  -fsyntax-only: PTH takes 89% of the time as non-PTH (11% speedup)

llvm-svn: 61913

- Added stub PTHLexer::getSpelling() that will be used for fetching cached
  spellings from the PTH file.  This doesn't do anything yet.
- Added a hook in Preprocessor::getSpelling() to call PTHLexer::getSpelling()
  when using a PTHLexer.
- Updated PTHLexer to read the offsets of spelling tables in the PTH file.

llvm-svn: 61911

avoid the version of Preprocessor::getSpelling that returns an 
std::string.

llvm-svn: 61769

their length.

llvm-svn: 61734

llvm-svn: 61431

Change preprocessor implementation of _cdecl to reference __cdecl.

llvm-svn: 61430

Remove previous __declspec macro that would effectively erase the construct prior to parsing.

llvm-svn: 61422

llvm-svn: 61382

PTH: Remove some methods and simplify some conditions in PTHLexer::Lex().  No big functionality change.

llvm-svn: 61381

- This reduces the PTH size for Cocoa.h by 7%.
- The increases PTH -Eonly speed for Cocoa.h by 0.8%.

llvm-svn: 61377

llvm-svn: 61375

- Encode the token length with 2 bytes instead of 4.
- This reduces the size of the .pth file for Cocoa.h by 12%.
- This speeds up PTH time (-Eonly) on Cocoa.h by 1.6%.

llvm-svn: 61364

- In PTHLexer::Lex read all of the token data from PTH file before
  constructing the token.  The idea is to enhance locality.
- Do not use Read8/Read32 in PTHLexer::Lex.  Inline these operations manually.
- Change PTHManager::ReadIdentifierInfo() to PTHManager::GetIdentifierInfo().
  They are functionally the same except that PTHLexer::Lex() reads the
  persistent id.

These changes result in a 3.3% speedup for PTH on Cocoa.h (-Eonly).

llvm-svn: 61363

- Embed 'eom' tokens in PTH file.
- Use embedded 'eom' tokens to not lazily generate them in the PTHLexer.
  This means that PTHLexer can always advance to the next token after
  reading a token (instead of buffering tokens using a copy).
- Moved logic of 'ReadToken' into Lex.  GetToken & ReadToken no longer exist.
- These changes result in a 3.3% speedup (-Eonly) on Cocoa.h.
- The code is a little gross.  Many cleanups are possible and should be done.

llvm-svn: 61360

llvm-svn: 61223

llvm-svn: 61205

Rewrite PTHLexer::DiscardToEndOfLine() to not use GetToken and instead only read the bytes needed to determine if a token is not at the start of the line.

llvm-svn: 61172

Change PTHLexer::getSourceLocation() to not call GetToken() and instead just read the file offset in the token data buffer directly.

llvm-svn: 61170

llvm-svn: 61152

llvm-svn: 61145

Preprocessor: Allocate MacroInfo objects using a BumpPtrAllocator instead using new/delete.  This speeds up -Eonly on Cocoa.h using the regular lexer by 1.8% and the PTHLexer by 3%.

llvm-svn: 61042

use identifierinfo instead.  Patch by Chris Goller!

llvm-svn: 60992

PTH: Added minor 'sibling jumping' optimization for iterating over the side table used for fast preprocessor block skipping.  This has a minor performance improvement when preprocessing Cocoa.h, but can have some wins in pathologic cases.

llvm-svn: 60966

Added PTH optimization to not process entire blocks of tokens that appear in skipped preprocessor blocks.  This improves PTH speed by 6%.  The code for this optimization itself is not very optimized, and will get cleaned up.

llvm-svn: 60956

llvm-svn: 60942

llvm-svn: 60937

- Added a side-table per each token-cached file with the preprocessor conditional stack.  This tracks what #if's are matched with what #endifs and where their respective tokens are in the PTH file.  This will allow for quick skipping of excluded conditional branches in the Preprocessor.
- Performance testing shows the addition of this information (without actually utilizing it) leads to no performance regressions.

llvm-svn: 60911

PTHLexer: Keep track of the location of the last '#' token and provide the means to jump ahead in the token stream.

llvm-svn: 60905

llvm-svn: 60896

- Added virtual method 'getSourceLocation()' (no arguments) that gets the location of the next "observable" location (e.g., next character, next token).
PPLexerChange.cpp:
- Implemented FIXME by using PreprocessorLexer::getSourceLocation() to get the location in the file we are returning to after lexing a #included file.  This appears to be slightly faster than having the branch (i.e., 'if(CurLexer)').  It's also not a really hot part of the Preprocessor.

llvm-svn: 60860

Declare PerIDCache as IdentifierInfo** instead of void*.  This is just cleaner.  No performance change.

llvm-svn: 60843

llvm-svn: 60559

llvm-svn: 60556

Use an array instead of a DenseMap to cache persistent IDs -> IdentifierInfo*.  This leads to a 4% speedup at -fsyntax-only using PTH.

llvm-svn: 60452

- Remove PTHManager.cpp.  Move all of its functions to PTHLexer.cpp since some of the internal methods are used by PTHLexer (their implementations are intertwined.)  This enables some important inlining opportunities at -O3.

- Don't construct an std::vector<Token> prior to feeding PTH tokens to the Preprocessor.  Stream them off the PTH file directly.

llvm-svn: 60447

- Added method "setPTHManager" that will be called by the driver to install
  a PTHManager for the Preprocessor.
- Fixed some comments.
- Added EnterSourceFileWithPTH to mirror EnterSourceFileWithLexer.

llvm-svn: 60437

Added PTHManager, a utility class that will be used by Preprocessor to lazily create PTHLexer objects for pre-tokenized files.

llvm-svn: 60436

llvm-svn: 60373

__ASSEMBLER__ properly. Patch from Roman Divacky (with minor
formatting changes). Thanks!

llvm-svn: 60362

llvm-svn: 60136