PTH: Remove some methods and simplify some conditions in PTHLexer::Lex().  No big functionality change.

llvm-svn: 61381

- This reduces the PTH size for Cocoa.h by 7%.
- The increases PTH -Eonly speed for Cocoa.h by 0.8%.

llvm-svn: 61377

llvm-svn: 61375

- Encode the token length with 2 bytes instead of 4.
- This reduces the size of the .pth file for Cocoa.h by 12%.
- This speeds up PTH time (-Eonly) on Cocoa.h by 1.6%.

llvm-svn: 61364

- In PTHLexer::Lex read all of the token data from PTH file before
  constructing the token.  The idea is to enhance locality.
- Do not use Read8/Read32 in PTHLexer::Lex.  Inline these operations manually.
- Change PTHManager::ReadIdentifierInfo() to PTHManager::GetIdentifierInfo().
  They are functionally the same except that PTHLexer::Lex() reads the
  persistent id.

These changes result in a 3.3% speedup for PTH on Cocoa.h (-Eonly).

llvm-svn: 61363

- Embed 'eom' tokens in PTH file.
- Use embedded 'eom' tokens to not lazily generate them in the PTHLexer.
  This means that PTHLexer can always advance to the next token after
  reading a token (instead of buffering tokens using a copy).
- Moved logic of 'ReadToken' into Lex.  GetToken & ReadToken no longer exist.
- These changes result in a 3.3% speedup (-Eonly) on Cocoa.h.
- The code is a little gross.  Many cleanups are possible and should be done.

llvm-svn: 61360

llvm-svn: 61205

Rewrite PTHLexer::DiscardToEndOfLine() to not use GetToken and instead only read the bytes needed to determine if a token is not at the start of the line.

llvm-svn: 61172

Change PTHLexer::getSourceLocation() to not call GetToken() and instead just read the file offset in the token data buffer directly.

llvm-svn: 61170

llvm-svn: 61152

llvm-svn: 61145

PTH: Added minor 'sibling jumping' optimization for iterating over the side table used for fast preprocessor block skipping.  This has a minor performance improvement when preprocessing Cocoa.h, but can have some wins in pathologic cases.

llvm-svn: 60966

Added PTH optimization to not process entire blocks of tokens that appear in skipped preprocessor blocks.  This improves PTH speed by 6%.  The code for this optimization itself is not very optimized, and will get cleaned up.

llvm-svn: 60956

- Added a side-table per each token-cached file with the preprocessor conditional stack.  This tracks what #if's are matched with what #endifs and where their respective tokens are in the PTH file.  This will allow for quick skipping of excluded conditional branches in the Preprocessor.
- Performance testing shows the addition of this information (without actually utilizing it) leads to no performance regressions.

llvm-svn: 60911

PTHLexer: Keep track of the location of the last '#' token and provide the means to jump ahead in the token stream.

llvm-svn: 60905

llvm-svn: 60896

Declare PerIDCache as IdentifierInfo** instead of void*.  This is just cleaner.  No performance change.

llvm-svn: 60843

llvm-svn: 60559

llvm-svn: 60556

Use an array instead of a DenseMap to cache persistent IDs -> IdentifierInfo*.  This leads to a 4% speedup at -fsyntax-only using PTH.

llvm-svn: 60452

- Remove PTHManager.cpp.  Move all of its functions to PTHLexer.cpp since some of the internal methods are used by PTHLexer (their implementations are intertwined.)  This enables some important inlining opportunities at -O3.

- Don't construct an std::vector<Token> prior to feeding PTH tokens to the Preprocessor.  Stream them off the PTH file directly.

llvm-svn: 60447

llvm-svn: 60136

In PTHLexer::DiscardToEndOfLine() use Lex() instead of AdvanceToken().  This handles transitions in the preprocessor state.

llvm-svn: 59845

- Move out logic for handling the end-of-file to LexEndOfFile (to match the Lexer) class.  The logic now mirrors the Lexer class more, which allows us to pass most of the Preprocessor test cases.

llvm-svn: 59768

- Move PTHLexer::GetToken() to be inside PTHLexer.cpp.
- When lexing in raw mode, null out identifiers.

llvm-svn: 59744

- Rename 'CurToken' and 'LastToken' to 'CurTokenIdx' and 'LastTokenIdx'
  respectively.
- Add helper methods GetToken(), AdvanceToken(), AtLastToken() to abstract away
  details of the token stream. This also allows us to easily replace their
  implementation later.

llvm-svn: 59733

Rewrote PTHLexer::Lex by digging through the sources of Lexer again.  Now we can do basic macro expansion using the PTHLexer.

llvm-svn: 59724

  LexingRawMode in the ctor of PreprocessorLexer.

- PTHLexer: Use "LastToken" instead of "NumToken"

llvm-svn: 59690

Add (untested) implementation of PTHLexer::isNextPPTokenLParen() and PTHLexer::DiscardToEndOfLine().

llvm-svn: 59687

llvm-svn: 59670

llvm-svn: 59668

- Add variants of IsNonPragmaNonMacroLexer to accept an IncludeMacroStack entry
  (simplifies some uses).
- Use IsNonPragmaNonMacroLexer in Preprocessor::LookupFile.
- Add 'FileID' to PreprocessorLexer, and have Preprocessor query this fileid
  when looking up the FileEntry for a file

Performance testing of -Eonly on Cocoa.h shows no performance regression because
of this patch.

llvm-svn: 59666

llvm-svn: 59424

llvm-svn: 59191

llvm-svn: 59169