If you can work from anywhere, which many can these days, why go out of your way to get somewhere to do that work? Why waste time commuting? Why break that time quickening moment to get out of your pajamas's and go somewhere? If office face-time drives you nuts,
If your boss wants to see you because they are a control freak,
If your peers want you within arms reach because they want to socialize,
If your staff needs you to be there to work out every detail in a meeting or you just can't trust them,
Then change something fast.
Make a drop in center rather than an office for those collaboration sessions, socialize over lunch or dinner together, and if all else fails change jobs or loose the needy peers and un-trustworthy staff.
Do something because your wasting the one finite quantity you have - time.