There are small businesses that form their business entity in a state other than one in which they conduct business because they've heard that lots of big corporations are incorporated in states like Delaware, Wyoming, Nevada, Colorado, etc and assume those must be the best places to organize their business entity too. After all, the big companies must know what they are doing and what's good for the big guys must be good for their small business too. That's flawed logic. There is a huge difference between a national or multinational corporation with billions of dollars in revenue and millions of shareholders whose shares trade on a stock exchange and a small business owned by one person or a small group of people that are privately held and don't have shares listed on a stock exchange.
The common perception among a lot of people is that the big businesses organize in places like Delaware because it provides some kind of great tax advantage to do so. But companies that operate nationwide are taxed by every state in which they do business as well as the state in which they are incorporated. For many corporations, that means they pay either income or franchise taxes to every state that levies those taxes. The reasons that big corporations with millions of shares trading on an exchange like incorporating in states like Delaware usually have little to do with taxes. Rather, those states, have laws that favor management in management/shareholder disputes and in the case of Delaware, has a court that hears exclusively corporate governance cases, which provides fast resolution to shareholder challenges, merger/acquisition disagreements, etc. Speed is important in a lot of those cases because lingering uncertainty over the resolution can hurt the stock price on the exchange. Those are not concerns that a small, closely held business has.
For a small business just starting out it's rare that forming the business entity in some state other than one in which it does business will be of any benefit and may result in the business paying more rather than less. As a result, for most small businesses, forming their business entity in the state where they manage and control the business is the best choice since they have to register in that state anyway (if they don't form the business entity there then they need to register it as an out of state entity, and the costs and obligations are very similar) and reduces the number of states that the company must file annual reports, have a registered agent (and they charge a fee, of course), and limits the states in which people with disputes with the company may file their lawsuit.
Many small businesses today start out as limited liability companies (LLCs). LLCs provide similar protection for the owners of the business that corporate shareholders get but provides more favorable tax treatment because it avoids the problem of double taxation of corporation income and more useful treatment of losses that may be incurred in the first year or two of the business. Limited liability partnerships (LLPs) also provides similar tax and limited liability benefits, but a LLC can have just one owner; any partnership needs at least two owners. It's easy to change the tax classification of a LLC to a C-corporation or S-corporation without changing the LLC registration with the state should either of those tax regimes become better as the business grows.
There are a lot of options and things to consider when starting up a new business. Some of them are things small business owners don't know they need to deal with when they set up until they run into a problem where they discover they are going to pay a whole lot more because they didn't do it right when they started. Get advice from a business attorney and a tax professional who handles business tax matters for advice before the business gets up and running. The fees you pay for that will be well worth it over the life of the business in the money saved by avoiding the pitfalls that occur when it's not done right.