llvm-svn: 60136

In PTHLexer::DiscardToEndOfLine() use Lex() instead of AdvanceToken().  This handles transitions in the preprocessor state.

llvm-svn: 59845

- Move out logic for handling the end-of-file to LexEndOfFile (to match the Lexer) class.  The logic now mirrors the Lexer class more, which allows us to pass most of the Preprocessor test cases.

llvm-svn: 59768

- Move PTHLexer::GetToken() to be inside PTHLexer.cpp.
- When lexing in raw mode, null out identifiers.

llvm-svn: 59744

- Rename 'CurToken' and 'LastToken' to 'CurTokenIdx' and 'LastTokenIdx'
  respectively.
- Add helper methods GetToken(), AdvanceToken(), AtLastToken() to abstract away
  details of the token stream. This also allows us to easily replace their
  implementation later.

llvm-svn: 59733

Rewrote PTHLexer::Lex by digging through the sources of Lexer again.  Now we can do basic macro expansion using the PTHLexer.

llvm-svn: 59724

  LexingRawMode in the ctor of PreprocessorLexer.

- PTHLexer: Use "LastToken" instead of "NumToken"

llvm-svn: 59690

Add (untested) implementation of PTHLexer::isNextPPTokenLParen() and PTHLexer::DiscardToEndOfLine().

llvm-svn: 59687

llvm-svn: 59670

llvm-svn: 59668

- Add variants of IsNonPragmaNonMacroLexer to accept an IncludeMacroStack entry
  (simplifies some uses).
- Use IsNonPragmaNonMacroLexer in Preprocessor::LookupFile.
- Add 'FileID' to PreprocessorLexer, and have Preprocessor query this fileid
  when looking up the FileEntry for a file

Performance testing of -Eonly on Cocoa.h shows no performance regression because
of this patch.

llvm-svn: 59666

llvm-svn: 59424

llvm-svn: 59191

llvm-svn: 59169