While a CPA may offer bookkeeping services, they're technically two separate roles.
A CPA is a Certified Public Accountant and they typically help business owners with tax planning & tax filing.
You don't need to work with a CPA to file your taxes because of the DIY tools out there, but as your business grows and you have more complexity, it usually makes sense to start getting tax advice.
Enrolled Agents (EAs) are another tax professional similar to CPAs, but they focus more on tax-specific scenarios than they do accounting. The EA certification is the highest credential awarded by the IRS.
On the other hand, a standard bookkeeper doesn't do tax prep or filing and they only help with the finances inside of your business. Some of their tasks may include managing your accounting software, categorizing tax-deductible expenses, or sending you cash flow and profit & loss reports.
Many bookkeepers are also CPAs or EAs, but there isn't a specific license to become a bookkeeper. This means you need to do some research and due diligence before choosing one.
For me, I would look for an EA that has experience with my style of business & provides the services I'm looking for, get a feel for how they communicate, ask other business owners for recommendations, and go with whoever feels like the best fit after meeting with them.