Your business should be giving you the lifestyle you want, not controlling your life.
Being self employed can be the worlds greatest lie: The worst employer with the least amount of freedom. Or, it can be the most liberating vehicle ever designed.
It all comes down to building with deliberate intent.
You may have to get uncomfortable. You're going to have to perform an honest review of your current situation. What parts of your business are broken? Where are you spending too much time? Where are you finding yourself constantly wishing that people just had 'common sense'?
Odds are, you're the problem. You're probably the bottleneck.
The good news is, there is a fix. You can have the lifestyle you wanted when you first decided to start a business. You can have time to pursue your hobbies, play with your kids, travel, and breath.