Overselling of the last mile infrastructure. If you want to see what a not-oversold last mile connection (with a real minimum speed SLA) would cost, take a look at the “for business” pricing and weep.
Most people’s usage is bursty, so the present oversold state is probably the least bad option compared to having a fixed <1Mbps line speed or paying thousands per month for a truly dedicated 50-100Mbps.
They can and do provide guaranteed speeds; it costs a lot more. They can't provide guarantees (or 99% guarantees) of existing advertised speeds at anywhere near the existing price structure because the economics are based on overselling.
It really shouldn't cost /that/ much more. If there were actual competition in a sane market the price would be practically whatever share of the electricity for powering the network infrastructure along the path plus some part replacement overhead and a small profit margin.
That's why the natural monopoly of the last mile should be owned by the municipality (the people in the area) and competition should begin at designated peering points where backbone access can be brokered in an actually competitive way.
Why do you think it shouldn't cost a lot more? A consumer-grade fiber service will put 16-32 passive fiber strands on a single GPON node and some multiple of that on each 10 gbps network port in the intermediate routers that eventually dump into the backbone. If you're selling dedicated bandwidth, you need active fiber out to the customer site, and your brutally expensive core routers can only have 5 customers per port instead of 50.
Moreover, there is actually quite a bit of competition in the dedicated bandwidth business line space. We were recently shopping around our contract at work and had at least 3-4 options IIRC.
Having the municipality own the infrastructure out to the peering point is a terrible idea. There's a lot of very expensive hardware between the fiber strand going to the customer site and the peering point.
Well, then they have to advertise the bandwidth that they can actually guarantee for 99%?! What is the justification for advertising speeds that you can expect to not actually achieve?
I mean, yes, it's obvious that overselling is in the interest of customers, and as such customers should accept that under more or less exceptional circumstances the service does not reach the advertised performance, so you maybe shouldn't use it for stuff that reacts catastrophically when bandwidth requirements aren't being met, fine. But how is it anything but deception to advertise bandwidth that the user can as a matter of fact not expect to be available under normal circumstances?
For comparison, take electricity: My connection to the grid might be built to be able to supply 40 kW, say. Now, if everyone were to actually pull 40 kW from their connection at the same time, that absolutely would not work--but that does not mean that electricity supply is regularly insufficient for my needs. Except for very rare exceptions, everyone can always pull as much power as they currently need. There is no brownout when everyone is cooking in the evening (or whatever). The capacity of the network and generators is also oversold, but utilities still take care to ensure statistically that situations where the actual instantaneous demand is not being met are few and far between. I don't see how the same could not be done with IP connectivity--and how advertising anything for which this cannot be expected to work out is anything but simple deception. If you cannot supply 1 MW of electricity at some point in your network ... how does that legitimize advertising 1 MW connections and then simply lowering the voltage when the customer tries to actually use it?
In short: Overselling is fine, as long as your customer doesn't notice, except very rarely under exceptional circumstances. If some product doesn't work under those circumstances, you'll have to raise the price or lower the advertised bandwidth.
How else am I, as a customer, supposed to distinguish an offer that effectively is going to deliver only 50 Mb/s from one that actually does 100 Mb/s during peak times, if both are allowed to advertise as "up to 200 Mb/s"? Where is the incentive for ISPs going to come from when there is no limit on how much they can oversubscribe?
"I don't see how the same could not be done with IP connectivity"
But it is exactly how it's done. The only ISPs that neglect even that are large evil monopolies, but since they are monopolies they can get away with a lot of things and ask 20x the price.
The other thing people don't seem to understand, is that guaranteed bandwidth is not about speed.
Well, yes, good ISPs do just that, sure, I guess my point is that anything else should just outright be rejected as obviously unacceptable, and it should thus be the legal responsibility of all ISPs and thus breach of contract if customers do actually notice that the network is oversold.
No, no one provides guaranteed speeds. Guarantees you are talking about that ISPs provide are only for you being able to saturate the link 100% of the time, which you shouldn't do on home or business connections.
Most people’s usage is bursty, so the present oversold state is probably the least bad option compared to having a fixed <1Mbps line speed or paying thousands per month for a truly dedicated 50-100Mbps.