Revert "Following up on PR48517, fix handling of template arguments that refer"

This reverts commit 8c1f2d15.

This is part of 5 commits being reverted due to https://crbug.com/1161059. See bug for repro.