ISP’s vary by location. I’m maintaining a list of providers with affordable internet only plans you can use to cut the cord. If you can’t find one on that page, try your existing provider. Now I know the “big” internet providers seem only to offer “triple play” packages bundling phone, TV, and internet. However, if you dig around on their site you should be able to find an internet service offering.
The main piece of equipment you will need to turn off cable is a streaming device. Luckily, most of us already have one. Many of the TVs sold these days are smart TVs that support various streaming services. Modern Blu-Ray players also tend to have smart features, and if you are a gamer, you can use your Xbox One or PlayStation 4 as a streaming device.
The question of "when does it make sense?" can easily be rephrased as "how much money can I save?" and one of the biggest factors will be the price difference between combined internet and TV service or just internet. These are also the options that vary the most widely based on where you live, and the availability of promotions or contracts that can keep prices down for a short time.