Skip to content
  • Douglas Gregor's avatar
    Implement two related optimizations that make de-serialization of · 09b6989e
    Douglas Gregor authored
    AST/PCH files more lazy:
      - Don't preload all of the file source-location entries when reading
      the AST file. Instead, load them lazily, when needed.
      - Only look up header-search information (whether a header was already
      #import'd, how many times it's been included, etc.) when it's needed
      by the preprocessor, rather than pre-populating it.
    
    Previously, we would pre-load all of the file source-location entries,
    which also populated the header-search information structure. This was
    a relatively minor performance issue, since we would end up stat()'ing
    all of the headers stored within a AST/PCH file when the AST/PCH file
    was loaded. In the normal PCH use case, the stat()s were cached, so
    the cost--of preloading ~860 source-location entries in the Cocoa.h
    case---was relatively low.
    
    However, the recent optimization that replaced stat+open with
    open+fstat turned this into a major problem, since the preloading of
    source-location entries would now end up opening those files. Worse,
    those files wouldn't be closed until the file manager was destroyed,
    so just opening a Cocoa.h PCH file would hold on to ~860 file
    descriptors, and it was easy to blow through the process's limit on
    the number of open file descriptors.
    
    By eliminating the preloading of these files, we neither open nor stat
    the headers stored in the PCH/AST file until they're actually needed
    for something. Concretely, we went from
    
    *** HeaderSearch Stats:
    835 files tracked.
      364 #import/#pragma once files.
      823 included exactly once.
      6 max times a file is included.
      3 #include/#include_next/#import.
        0 #includes skipped due to the multi-include optimization.
    1 framework lookups.
    0 subframework lookups.
    
    *** Source Manager Stats:
    835 files mapped, 3 mem buffers mapped.
    37460 SLocEntry's allocated, 11215575B of Sloc address space used.
    62 bytes of files mapped, 0 files with line #'s computed.
    
    with a trivial program that uses a chained PCH including a Cocoa PCH
    to
    
    *** HeaderSearch Stats:
    4 files tracked.
      1 #import/#pragma once files.
      3 included exactly once.
      2 max times a file is included.
      3 #include/#include_next/#import.
        0 #includes skipped due to the multi-include optimization.
    1 framework lookups.
    0 subframework lookups.
    
    *** Source Manager Stats:
    3 files mapped, 3 mem buffers mapped.
    37460 SLocEntry's allocated, 11215575B of Sloc address space used.
    62 bytes of files mapped, 0 files with line #'s computed.
    
    for the same program.
    
    llvm-svn: 125286
    09b6989e
Loading