Update: The conclusions in this post might be invalid. I’ve left the post unchanged but I added a footnote explaining why. I never thought I’d apologize to Comcast, but … sorry I said–in great detail–that your service is bad. It turns out maybe it isn’t? Ok here’s the original post:
Since the pandemic, I’ve been working 99% remotely, often from my in-laws’ house. Recently, my father-in-law and I have both noticed a suspicious pattern. Every day, randomly throughout the afternoon, our internet speed drastically decreases, and occasionally our connection drops completely.
Comcast “knows nothing”, but I know it’s happening, my father-in-law knows it’s happening, my coworkers who try and speak to me on zoom know it’s happening, but I have no data to back it up. Unfortunately for Comcast1, I’m a programmer and I am willing to sink a stupid number of hours into proving that I’m being throttled.
Consider this my first piece of evidence:
So a few weeks ago I whipped up a script, and now I have data!
Methods
Before I discuss the results, here’s a brief overview of what I did.
- Using the speedtest cli and a simple script, I ran a speed test every 5 minutes each working day for a week, from approximately 8:30am – 8:30pm
- I ran this script for a week at my in-laws’, to gather Comcast data
- I ran this script for a week at my apartment, to gather Fios data
I wrote another post with technical details, including the actual code, but that’s the gist. On to the results!
Results
Summary
- Comcast has a significantly higher average download speed – 329 Mbps
- Fios has a significantly higher average upload speed – 119 Mbps
- Comcast drops packets at around 12 times the rate of Fios
- Comcast failed to run the test entirely 3.24% of the time, with some version of
internet connection error
Is Comcast throttling me?
To answer my own question from the title of this post: No. It appears that Comcast is not throttling my connection. However, it also appears that their service is very inconsistent and unreliable. The graphs below make it a bit more clear.
Download speeds
Comcast
After a week of data, Comcast’s average download speed was almost twice as fast as that of Fios, 329 Mbps vs 166 Mbps. But that’s not the whole story. Here’s an example of a typical day’s Comcast data:
It’s all over the place. The average speed for this day was 358 Mbps, even higher than the weeklong average. However, it was very inconsistent.
From 10am until 2pm, download speed ping-ponged between ~20 Mbps and ~400Mbps, with a few actual 0 values which mean the test failed entirely. Then in the afternoon the speed increased dramatically, averaging around 600 Mbps from 3:00pm until 6:30pm.
Comcast’s results were similar in each day of data, and they match my experience. When Comcast’s service is working, it’s fast, but there are often long stretches where it’s very slow bordering on non-functional. Interestingly, this single day of Comcast data contains almost as many test failures (4) as Fios experienced during an entire week (6).
Fios
Compare Comcast with a typical day of Fios:
Fios download speed is also quite variable, but within a tighter range2. This graph is from a slightly below average day, at 148 Mbps on average, but you can see most of the day is spent right around 100 Mbps, and it never dropped below 50 Mbps. Importantly, the speed test didn’t fail once.
This also matches my personal experience. Fios is fast enough that I never have issues with video calls, streaming, or anything else, and I very rarely experience any kind of dropout.
Upload speeds
Comcast
When it comes to upload speed, Comcast was the clear and obvious loser. The graph below is the upload trend for the same day as above:
Comcast averaged 20.6 Mbps upload. That’s not fast, but it’s probably plenty for your average home internet user. Xfinity’s 200 Mbps plan only claims to provide 10 Mbps upload anyway, so they’re actually exceeding their advertised speed.
Fios
Fios averaged 98 Mbps upload for this day, a bit lower than the weeklong average of 119 Mbps. It was also highly variable throughout the day, going as low as 4 Mbps around 5:30pm and then back up to 235 Mbps less than an hour later.
Still, almost every Fios test run was faster than Comcast’s average speed. That said, Fios’ upload speed is probably overkill for an ordinary home user, and if they offered a cheaper plan with less upload I would likely buy it.
Additional thoughts
Comcast
When I started this project, I thought Comcast was throttling me. It turns out that Comcast’s download speed was actually extremely fast, above and beyond their advertised speed. It topped out at a blazing 876 Mbps during the week.
However, the speed test failed entirely 3.24% of the time, which is a lot. From personal experience, I’d bet that if I ran the test once every minute (instead of every five minutes), I’d see clusters of failures in the data; usually the connection issues last 2-3 minutes. These failures alone are enough to make me seek out another provider despite the higher overall download speed; 2-3 minutes without internet is a big deal when you’re in the middle of a meeting.
Fios
Fios was slower than I thought it would be, averaging 166 Mbps for the week. Importantly, though, it was extremely consistent. I rarely ever experience issues with Fios, or see Zoom’s Your internet connection is unstable warning.
That being said, Fios claims 200/200 for my internet plan, which they are decidedly not delivering. What are the odds a random blog article could get them to give me a refund?3
Confounding Factors
These comparisons weren’t exactly apples to apples. For one, the Comcast test was run on a new-ish wifi mesh system through a wall, whereas the Fios test was run on a fairly old Netgear router with direct line of sight. If I wanted to go further I’d use the same router at both locations and wire directly into it.
Conclusion
- Comcast is fast but unreliable
- Comcast is achieving its advertised speeds (when it works)
- Verizon Fios is very reliable, but slower (it’s fast enough)
- Fios is not achieving its advertised speeds
- If given the choice, I’d pick Fios despite its slower download; reliability is important
Validity of results
In the confounding factors section above I mention that the Comcast test was run on a mesh wifi system. After writing this post, I did a side-by-side test of Comcast’s performance with and without the mesh network. It looks like a lot of the issues–packet loss, test failures–might have been due to the mesh system, and not Comcast’s service. I’d like to do a more detailed follow up post, but I’m not sure if/when I’ll be able to collect the data.
Notes