[AMDGPU] Save/Restore SCC bit across waterfall loop. (#68363)

Waterfall loop is overwriting SCC bit of status register. Make sure SCC
bit is saved and restored across.
We need to save/restore only in cases where SCC is live across waterfall
loop.

Co-authored-by: Sirish Pande <sirish.pande@amd.com>