This is more “home networking” than “homelab,” but I imagine the people here might be familiar with what in talking about.
I’m trying to understand the logic behind ISPs offering asymmetrical connections. From a usage standpoint, the vast majority of traffic goes to the end-user instead of from the end-user. From a technical standpoint, though, it seems like it would be more difficult and more expensive to offer an asymmetrical connection.
While consumers may be connected via fiber, cable, DSL, etc, I assume that the ISP has a number of fiber links to “the internet.” Those links are almost surely some symmetrical standard (maybe 40 or 100Gb). So if they assume that they can support 1000 users at a certain download speed, what is the advantage of limiting the upload? If their incoming trunks can support 1000 users at 100Mb download, shouldn’t it also support 1000 users at 100Mb upload since the trunks themselves are symmetrical?
Limiting the upload speed to a different rate than download seems like it would just add a layer of complexity. I don’t see a financial benefit either; if their links are already saturated for download, reducing upload speed doesn’t help them add additional users. Upload bandwidth doesn’t magically turn into download bandwidth.
Obviously there’s some reason for this, but I can’t think of one.
Historically, last-mile technologies like dial-up, DSL, satellite, and DOCSIS/cable had limitations on their uplink power. That is, the amount of energy they can use to send upload through the medium.
Dial-up and DSL had to comply with rules on telephone equipment, which I believe limited end-user equipment to less power than what the phone company can put onto the wires, premised on the phone company being better positioned to identify and manage interference between different phone lines. Generally, using reduced power reduces signal-to-noise ratio, which means less theoretical and practical bandwidth available for the upstream direction.
Cable has a similar restriction, because cable plants could not permit end-user “back feeding” of the cable system. To make cable modems work, some amount of power must be allowed to travel upstream, but too much would potentially cause interference to other customers. Hence, regulatory restrictions on upstream power. This also matched actual customer usage patterns at the time.
Satellite is more straightforward: satellite dishes on earth are kinda tiny compared to the bus-sized satellite’s antennae. So sending RF up to space is just harder than receiving it.
Whereas fibre has a huge amount of bandwidth, to the point that when new PON standards are written, they don’t even bother reusing the old standard’s allocated wavelength, but define new wavelengths. That way, both old and new services can operate on the fibre during the switchover period. So fibre by-default allocates symmetrical bandwidth, although some PON systems might still be closer to cable’s asymmetry.
But there’s also the backend side of things: if a major ISP only served residential customers, who predominantly have asymmetric traffic patterns, then they will likely have to pay money to peer with other ISPs, because of the disparity. Major ISPs solve this by offering services to data centers, which generally are asymmetric but tilted towards upload. By balancing residential with server customers, the ISP can obtain cheaper or even free peering with other ISPs, because symmetrical traffic would benefit both and improve the network.
This is a really good explanation; thank you!
There is one thing I’m having a hard time understanding, though; I’m going to use my ISP as an example. They primarily serve residential customers and small businesses. They provide VDSL connections, and there isn’t a data center anywhere nearby, so any traffic going over the link to their upstream provider is almost certainly very asymmetrical. Their consumer VDSL service is 40Mb/2Mb, and they own the phone lines (so any restriction on transmit power from the end-user is their own restriction).
To make the math easy, assume they have 1000 customers, and they’re guaranteeing the full 40Mb even at peak times (this is obviously far from true, but it makes the numbers easy). This means that they have at least a 40Gbit link to their upstream provider. They’re using the full 40Gb on one side of the link, and only 2Gbit on the other. I’ve used plenty of fiber SFP+ modules, and I’ve never seen one that supports any kind of asymmetrical connection.
With this scenario, I would think that offering their customers a faster uplink would be free money. Yet for whatever reason, they don’t. I’d even be willing to buy whatever enterprise-grade equipment is on the other end of my 40/2 link to get a symmetrical 40/40; still not an option. Bonded DSL, also not an option.
With so much unused upload bandwidth on the ISP’s part, I would think they’d have some option to upgrade the connection. The only thing I can think is that having to maintain accounts for multiple customers with different service levels costs more than selling some of their unused upload bandwidth.
The routing equipment at the distribution boxes is likely a limit. Both in regards to power consumption and heat production, plus especially with older equipment the total throughput it is capable of.
My last post didn’t substantially address smaller ISPs, and from your description, it does sound like your ISP might be a smaller operator. But essentially, on the backend, a smaller ISP won’t have the customer base to balance their traffic in both directions. But they still need to provision for peak traffic demand, and as you observed, that could mean leaving capacity on the table, err fibre. This is correct from a technical perspective.
But now we touch up on the business side of things again. The hypothetical small ISP – which I’ll call the Retail ISP, since they are the face that works with end-user residential customers – will usually contract with one of more regional ISPs in the area for IP transit. That is, upstream connectivity to the broader Internet.
It would indeed be wasteful and expensive to obtain an upstream connection that guarantees 40 Gbps symmetric at all times. So they don’t. Instead, the Retail ISP would pursue a bustable billing contract, where they commit to specific, continual, averaged traffic rates in each direction, but have some flexibility to use more or less than that commited value.
So even if the Retail ISP is guaranteeing each end-user at least 40 Gbps download, the Retail ISP must write up a deal with the Upstream ISP based on averages. And with, say, 1000 customers, the law of averages will hold true. So let’s say the average rates are actually 20 Gbps down/1 Gbps up.
To be statistically rigorous though, I should mention that traffic estimation is a science, with applicability to everything from data network and road traffic planning, queuing for the bar at a music venue, and managing electric grid stability. Looking at historical data to determine a weighed average would be somewhat straightforward, but compensating for variables so that it can become future-predictive is the stuff of statisticians with post-nominative degrees.
What I can say though, from what I remember in calculus at uni, is that if each end-user’s traffic rates are independent from other end-users (a proposition that is usually true but not necessarily at all times of day), then the Central Limit Theorem states that the distribution of the aggregate set of end-users will approximate a normal distribution (aka Gaussian, or bell curve), getting closer for more users. This was a staggering result when I first learned it, because it really doesn’t matter what each user is doing, it all becomes a bell curve in the end.
The Retail ISP’s contract with the Upstream ISP probably has two parts: a circuit, and transit. The circuit is the physical line, and for the given traffic, probably a 50 Gbps fibre connection might be provisioned for lots of burstable bandwidth. But if the Retail ISP is somewhat remote, perhaps a microwave RF link could be set up, or leased from a third-party. But we’ll stick with fibre, as that’s going to be symmetrical.
As a brief aside, even though a 40 Gbps circuit would also be sufficient, sometimes the Upstream ISP’s nearby equipment doesn’t support certain speeds. If the circuit is Ethernet based, then a 40 Gbps QSFP+ circuit is internally four 10 Gbps links bundles together on the same fibre line. But supposing the Upstream ISP normally sells 200 Gbps circuits, then 50 Gbps to the Retail ISP makes more sense, as a 200 Gbps QSFP56 circuit is internally made from four 50 Gbps, which oftentimes can be broken out. The Upstream and Retail ISPs need to agree on the technical specs for the circuit, but it certainly must provide overhead beyond the averages agreed upon.
And those averages are captured in the transit contract, where brief exceedances/underages are not penalized but prolonged conditions would be subject to fees or even result in new contract negotiations. The “waste” of circuit capacity (especially upload) is something both the Retail ISP (who saves money, since guaranteed 50 Gbps would cost much more) and the Upstream ISP willingly accept.
Why? Because the Upstream ISP is also trying to balance the traffic to their upstream, to avoid fees for imbalance. So even though the Retail ISP can’t guarantee symmetric traffic to the Upstream ISP, what the Retail ISP can offer is predictability.
If the Upstream ISP can group the Retail ISP’s traffic with a nearby data center, then that could roughly balance out, and allow them to pursue better terms with the subsequent higher tier of upstream provider.
Now we can finally circle back on why the Retail ISP would decline to offer end-users some faster upload speeds. Simply put, the Retail ISP may be aware that even if they offer higher upload, most residential customers won’t really take advantage of it, even if it was a free upgrade. This is the reality of residential Internet traffic. Indeed, the unique ISPs in the USA offering residential 10 Gbps connections have to be thoroughly aware that even the most dedicated of, err, Linux ISO afficionados cannot saturate that connection for more than a few hours per month.
But if most won’t take advantage of it, then that shouldn’t impact the Retail ISP’s burstable contract with the Upstream ISP, and so it’s a free choice, right? Well, yes, but it’s not the only consideration. The thing about offering more upload is that while most customers won’t use it, a small handful will. And maybe those customers are the type that will complain loudly if the faster upload isn’t honored. And that might hurt Retail ISP’s reputation. So rather than take that gamble through guaranteeing faster upload for residential connections, they’d prefer to just make it “best effort”, whatever that means.
EDIT: The description above sounds a bit defeatist for people who just want faster upload, since it seems that ISPs just want to do the bare minimum and not cater to users who are self-hosting, whom ISPs believe to be a minority. So I wanted to briefly – and I’m aware that I’m long winded – describe what it would take to change that assumption.
Essentially, existing “average joe” users would have to start uploading a lot more than they are now. With so-called cloud services, it might seem that upload should go up, if everyone’s photos are stored on remote servers. But cloud services also power major sites like Netflix, which are larger download sources. So net-net, I would guess that the residential customer’s download-to-upload ratio is growing wider, and isn’t shrinking.
It would take a monumental change in networking or computer or consumer demand to reverse this tide. Example: a world where data sovereignty – bonafide ownership of your own data – is so paramount that everyone and their mother has a social-media server at home that mutually relays and amplifies viral content. That is to say, self-hosting and upload amplification.