Personally, I have never been tempted to use Fate for action refreshes. I’ve always felt comfortable just waiting for them to come back naturally; I reserve Fate for things that cannot be gotten any other way.
(I’ve used Fate for action refreshes a couple times, but that was a unique circumstance. I was reporting live from Winking Isle; some people had sent me Fate gifts, and as I was the only person who could explore the new content, I felt pressure to finish quickly and pass on the card. The point being that this pressure was entirely external to the game itself.)