This reverts commit a8e5ce76. It looks like older Xcodes' headers don't allow arm64 builds so it was causing CI failures.