[scudo] s/unsigned long/size_t/ for __scudo_set_rss_limit

Summary:
`__scudo_set_rss_limit`'s `LimitMb` should really be a `size_t`. Update
accordingly the prototype. To avoid the `NOLINT` and conform with the other
Sanitizers, use the sanitizers types for the internal definition. This should
have no functional change.

Additionally, capitalize a variable name to follow the LLVM coding standards.

Reviewers: alekseyshl, flowerhack

Reviewed By: alekseyshl

Subscribers: #sanitizers, llvm-commits

Differential Revision: https://reviews.llvm.org/D41704

llvm-svn: 321803