This reverts commit r341628.

llvm-svn: 341631

Summary:
This patch implements a `BlockVerifier` type which enforces the
invariants of the log structure of FDR mode logs on a per-block basis.
This ensures that the data we encounter from an FDR mode log
semantically correct (i.e. that records follow the documented "grammar"
for FDR mode log records).

This is another part of the refactoring of D50441.

Reviewers: mboerger, eizan

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51723

llvm-svn: 341628

Summary:
This change adds a `BlockIndexer` type which maintains pointers to
records that belong to the same process+thread pairs. The indexing
happens with order of appearance of records as they are visited.

This version of the indexer currently only supports FDR version 3 logs,
which contain `BufferExtent` records. We will add support for v2 and v1
logs in follow-up patches.

This is another part of D50441.

Reviewers: eizan, kpw, mboerger

Reviewed By: mboerger

Subscribers: mboerger, mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51673

llvm-svn: 341518

Summary:
This change adds a `RecordPrinter` type which does some basic text
serialization of the FDR record instances. This is one component of the
tool we're building to dump the records from an FDR mode log as-is.

This is a small part of D50441.

Reviewers: eizan, kpw

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51672

llvm-svn: 341447

Summary:
This patch defines two new base types called `RecordProducer` and
`RecordConsumer` which have default implementations for convenience
(particularly for testing).

A `RecordProducer` implementation has one member function called
`produce()` which serves as a factory constructor for `Record`
instances. This code exercises the `RecordInitializer` code path in the
implementation for `FileBasedRecordProducer`.

A `RecordConsumer` has a single member function called `consume(...)`
which, as the name implies, consumes instances of
`std::unique_ptr<Record>`. We have two implementations, one of which is
used in the test to generate a vector of `std::unique_ptr<Record>`
similar to how the `LogBuilder` implementation works.

We introduce a test in `FDRProducerConsumerTest` which ensures that
records we write through the `FDRTraceWriter` can be loaded by the
`FileBasedRecordProducer`. The record(s) loaded this way are written
again through the `FDRTraceWriter` into a separate string, which we then
compare. This ensures that the read-in bytes to create the `Record`
instances in memory can be replicated when written out through the
`FDRTraceWriter`.

This change depends on D51210 and is part of the refactoring of D50441
into smaller, more focused changes.

Reviewers: eizan, kpw

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51289

llvm-svn: 341180

Summary:
This is the first step in the larger refactoring and reduction of
D50441.

This step in the process does the following:

- Introduces more granular types of `Record`s representing the many
  kinds of records written/read by the Flight Data Recorder (FDR) mode
  `Trace` loading function(s).

- Introduces an abstract `RecordVisitor` type meant to handle the
  processing of the various `Record` derived types. This `RecordVisitor`
  has two implementations in this patch: `RecordInitializer` and
  `FDRTraceWriter`.

- We also introduce a convenience interface for building a collection of
  `Record` instances called a `LogBuilder`. This allows us to generate
  sequences of `Record` instances manually (used in unit tests but
  useful otherwise).

- The`FDRTraceWriter` class implements the `RecordVisitor` interface and
  handles the writing of metadata records to a `raw_ostream`. We
  demonstrate that in the unit test, we can generate in-memory FDR mode
  traces using the specific `Record` derived types, which we load
  through the `loadTrace(...)` function yielding valid `Trace` objects.

This patch introduces the required types and concepts for us to start
replacing the logic implemented in the `loadFDRLog` function to use the
more granular types. In subsequent patches, we will introduce more
visitor implementations which isolate the verification, printing,
indexing, production/consumption, and finally the conversion of the FDR
mode logs.

The overarching goal of these changes is to make handling FDR mode logs
better tested, more understandable, more extensible, and more
systematic. This will also allow us to better represent the execution
trace, as we improve the fidelity of the events we represent in an XRay
`Trace` object, which we intend to do after FDR mode log processing is
in better shape.

Reviewers: eizan

Reviewed By: eizan

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51210

llvm-svn: 341029

Summary:
This change implements the profile loading functionality in LLVM to
support XRay's profiling mode in compiler-rt.

We introduce a type named `llvm::xray::Profile` which allows building a
profile representation. We can load an XRay profile from a file to build
Profile instances, or do it manually through the Profile type's API.

The intent is to get the `llvm-xray` tool to generate `Profile`
instances and use that as the common abstraction through which all
conversion and analysis can be done. In the future we can generate
`Profile` instances from `Trace` instances as well, through conversion
functions.

Some of the key operations supported by the `Profile` API are:

- Path interning (`Profile::internPath(...)`) which returns a unique path
  identifier.

- Block appending (`Profile::addBlock(...)`) to add thread-associated
  profile information.

- Path ID to Path lookup (`Profile::expandPath(...)`) to look up a
  PathID and return the original interned path.

- Block iteration.

A 'Path' in this context represents the function call stack in
leaf-to-root order. This is represented as a path in an internally
managed prefix tree in the `Profile` instance. Having a handle (PathID)
to identify the unique Paths we encounter for a particular Profile
allows us to reduce the amount of memory required to associate profile
data to a particular Path.

This is the first of a series of patches to migrate the `llvm-stacks`
tool towards using a single profile representation.

Depends on D48653.

Reviewers: kpw, eizan

Reviewed By: kpw

Subscribers: kpw, thakis, mgorny, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D48370

llvm-svn: 341012

Summary:
This patch moves out the definition of the XRay log file header from
binary logs into its own header and implementation file.

This is one part of the refactoring being done in D50441.

Reviewers: eizan

Subscribers: mgorny, hiraditya, llvm-commits

Differential Revision: https://reviews.llvm.org/D51086

llvm-svn: 340389

Reverted r338825 and all the following tries to fix issues introduced by that commit (r338826, r338827, r338829, r338880).

This commit has broken build bots and has been left unattended for too long.

llvm-svn: 338948

llvm-svn: 338880

Summary:
This change implements the profile loading functionality in LLVM to
support XRay's profiling mode in compiler-rt.

We introduce a type named `llvm::xray::Profile` which allows building a
profile representation. We can load an XRay profile from a file to build
Profile instances, or do it manually through the Profile type's API.

The intent is to get the `llvm-xray` tool to generate `Profile`
instances and use that as the common abstraction through which all
conversion and analysis can be done. In the future we can generate
`Profile` instances from `Trace` instances as well, through conversion
functions.

Some of the key operations supported by the `Profile` API are:

- Path interning (`Profile::internPath(...)`) which returns a unique path
  identifier.

- Block appending (`Profile::addBlock(...)`) to add thread-associated
  profile information.

- Path ID to Path lookup (`Profile::expandPath(...)`) to look up a
  PathID and return the original interned path.

- Block iteration.

A 'Path' in this context represents the function call stack in
leaf-to-root order. This is represented as a path in an internally
managed prefix tree in the `Profile` instance. Having a handle (PathID)
to identify the unique Paths we encounter for a particular Profile
allows us to reduce the amount of memory required to associate profile
data to a particular Path.

This is the first of a series of patches to migrate the `llvm-stacks`
tool towards using a single profile representation.

Depends on D48653.

Reviewers: kpw, eizan

Reviewed By: kpw

Subscribers: mgorny, llvm-commits, hiraditya

Differential Revision: https://reviews.llvm.org/D48370

llvm-svn: 338825

Summary:
This change implements the instrumentation map loading library which can
understand both YAML-defined instrumentation maps, and ELF 64-bit object
files that have the XRay instrumentation map section. We break it out
into a library on its own to allow for other applications to deal with
the XRay instrumentation map defined in XRay-instrumented binaries.

This type provides both raw access to the logical representation of the
instrumentation map entries as well as higher level functions for
converting a function ID into a function address.

At this point we only support ELF64 binaries and YAML-defined XRay
instrumentation maps. Future changes should extend this to support
32-bit ELF binaries, as well as other binary formats (like MachO).

As part of this change we also migrate all uses of the extraction logic
that used to be defined in tools/llvm-xray/ to use this new type and
interface for loading from files. We also remove the flag from the
`llvm-xray` tool that required users to specify the type of the
instrumentation map file being provided to instead make the library
auto-detect the file type.

Reviewers: dblaikie

Subscribers: mgorny, varno, llvm-commits

Differential Revision: https://reviews.llvm.org/D29319

llvm-svn: 293721

Summary:
In this change we move the definition of the log reading routines from
the tools directory in LLVM to {include/llvm,lib}/XRay. We improve the
documentation a little bit for the publicly accessible headers, and
adjust the top-matter. This also leads to some refactoring and cleanup
in the tooling code.

In particular, we do the following:

  - Rename the class from LogReader to Trace, as it better represents
    the logical set of records as opposed to a log.
  - Use file type detection instead of asking the user to say what
    format the input file is. This allows us to keep the interface
    simple and encapsulate the logic of loading the data appropriately.

In future changes we increase the API surface and write dedicated unit
tests for the XRay library.

Depends on D24376.

Reviewers: dblaikie, echristo

Subscribers: mehdi_amini, mgorny, llvm-commits, varno

Differential Revision: https://reviews.llvm.org/D28345

llvm-svn: 291652