We can rejoice in God's refuge because it is perfect safety.

But let all who take refuge in you be glad; let them ever sing for joy. Spread your protection over them, that those who love your name may rejoice in you.