Really simple idea, not really affecting the game itself. There should be a fifth outfit that could be like "Everyday Wear" or something along those lines. Here you could either put your favorite - or most beneficial - outfits.
Per the FBG Tumblr in response to the same suggestion:
"We’ll see how people get on with it over the next few weeks and hopefully make some improvements before Christmas (time depending)."