[clangd] Avoid wasteful data structures in RefSlab::Builder

Summary: This is worth another 10% or so on InedxBenchmark.DexBuild.

Reviewers: kbobyrev

Subscribers: ilya-biryukov, MaskRay, jkorous, mgrang, arphaman, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D79950