I think that it's cause and effect issue. Developers who've proven themselves to be reliable and experienced can 'force' their preferences onto employers. And they happen to cost more as well. I don't believe that going remote magically makes you able command higher salary.
Agreed, which is why I didn't say,"working remote causes developers to earn 22% more", only that developers who work remote earn 22% more, which is an interesting fact in itself.
Early in the article I adjust this for various controls to better get at causality. This includes observable factors like age, years of experience, hours worked, size of employer, programming languages used, etc.
As I note in the article:
>Much of the apparent premium earned by remote developers is in fact driven by seniority and tenure. These are older, more experienced developers who either prefer to work remote or whose organizations grant them that privilege.
However, controlling for manually selected factors doesn't imply causality, so I use principled covariate selection to select the best set of controls and get closer to something that could be called causal. You can read more about this method, called Double Lasso Selection, in Urminsky, Hansen, and Chernozhukov [0].
This results in an adjusted pay premium of 9.4% for remote developers relative to those that never work remote. Hard to know for sure if this is causal either, but it's likely much closer to whatever the true causal impact is. Unsurprisingly, it's a lower number.
One easy dumb trick I use to filter out "interesting" results that are likely just getting causality backwards is to flip the title around and see if it immediately sounds like an obvious truism. In the case of this article, it goes from:
> Remote Software Developers Earn 22% More Than Non-Remote Developers
To something like:
> Higher-paid software developers more likely to be working remotely
And at that point, yes, it's fairly obviously going to be true. Working remotely is a perk for those who prefer it so we should expect it to be positively correlated with other forms of compensation.
I think all that your data really shows is a hidden variable: competence. Better software developers get paid more, work remotely more, and probably also get more paid time off, larger bonuses, and all sorts of other perks.
You could also speculate that employers who offer WFH are more likely to pay more. It’s seen as a perk by many people, and suggesting that employers who offer more perks might pay more doesn’t seem unlikely to me. The headline would seem silly if it said “engineers who get free hot meals at work earn more than those who don’t”.
I feel like for those Sr. Eng's who can negotiate a remote position, if they don't go remote their careers and pay would go even further. It's one reason why I don't go remote, it's hard to reach staff or sr management positions typically when you do that.
There's a lot of us that don't want to move into positions where we don't work on software development at the code level. Unfortunately, that also tends to put a cap on how much we can make because a lot of people see management as an upward movement rather than a lateral one.
I work at a company with dual paths (management vs individual contributor) and a requirement to work at the code level would cap your progress upward on either path. As a senior engineer you are expected to spend significantly more time doing architecture work, mentoring other developers, and other non-code activities.
> Early in the article I adjust this for various controls to better get at causality. This includes observable factors like age, years of experience, hours worked, size of employer, programming languages used, etc.
There are likely many other factors which aren't "observable" by your methodology, which are causing the 9.4% pay-premium. Factors like competency, domain knowledge, and how essential they are to the organization. I find it much more believable that these unobservable factors are causing both the pay-premium, and working-remote flexibility.
This is a common problem that comes up in such studies. Adjusting for various observable factors is great, but it still leaves behind the other unobservable factors. At which point you have to use your judgement to figure out whether those unobservable factors are more compelling than the hypothesis being tested.
That said, this is still a very cool analysis that demonstrates an interesting correlation. Thanks for sharing!
My go-to quip for making decisions based on observable variables when the actual causal relationship is between unobservable variables and outcomes is, "Looking for your watch under the street lamp."
---
For those who aren't familiar with the aphorism, it's from a joke that goes:
One night, I came upon a man staring at the ground under a street lamp. "Looking for something?" I asked, and the man nodded, "My watch." "You lost it here?" I asked, but the man shook his head. "I lost it further down the street."
The "etc" I used in my quote is doing a lot of work here - I control for a fairly extensive list of observed factors from the dataset. I just listed out a few because otherwise the list gets absurdly long.
I control for domain knowledge via what programming languages, technologies, and tools the developers know. I use a proxy for importance to the organization by controlling for the control the developer has over buying decisions within their organization. I control for education levels and college major. I control for employment status (full-time, part-time, independent). Hours worked per week.
The list goes on but those are a few.
But your point remains that unobservables could be confounding the results, so one can never be sure.
I tried to find the factors you controlled for. You wrote
> (including age, experience, hours worked, size of employer, programming languages, and more)
but the "and more" link didn't give me more insight into what controls you used. The interpretation here is super duper sensitive to what those were, so you have just a straight up list of what your controls were? (before & after feature selection)
I am most curious about geographic controls, which are going to be tricky, especially since you can't just one-hot encode them and throw them into double-LASSO. The association might be as simple as "remote workers more likely to be paid in USD" or something analogous, and I'm trying to determine whether/how you controlled for that.
You can't really remove the selection effect of higher paid developers choosing to be remote by adding external controls. You could try using an instrument for remote work or use a Simultaneous Equation Model to remove the selection effect bias.
Counter to your analysis, I'd actually expect the pay premium for remote work would be negative simply because the opportunity to work remote is worth paying for.
Pure speculation, but maybe you have to look around or negotiate to find remote work? Those who tend to do that probably also get better salaries than those that stick with one job for very long or don't compare much when looking for jobs.
One possibility is that remote developers are just more productive (or if you prefer, more productive developers are more able to negotiate a remote working allowance from their employer). In standard economic theory, the equilibrium wage is at the workers marginal revenue product (or the marginal revenue that their work produces for the organization). So you would expect remote workers to either earn more because they are more productive or, if remote work makes you more productive, to earn more as a result of working remotely and being more productive
> Agreed, which is why I didn't say,"working remote causes developers to earn 22% more", only that developers who work remote earn 22% more, which is an interesting fact in itself.
... but you have a heading right at the start of the article: "Remote work pays", what was that supposed to mean then?
Seniority is also implicitly more about _independence_ than _capability_. This is something Junior engineers are often confused about.
Its easier to hire, or let a senior work at home, because generally, they have a much stronger track record of "not becoming blocked" and "not needing someone to prioritize tasks" etc.
In short, its less about "forcing" preferences, and more about managements have seen this person execute with less oversite. Which is almost definitionally seniority.
A lot of you will respond to this "I don't need oversite to get things done" and a lot of you will be wrong. Because a lot of engineers need oversite to get _the right things_ done.
Generally agreed, just wanted to point out that "oversite" is "a layer of concrete used to seal the earth under the ground floor of a house", while "oversight" is "the action of overseeing something". So yes, a lot of civil engineers need oversite to get the right things done :)
Independence and reliability are both driving factors for me as a manager. I have a junior dev with 2 years experience and he had to move due to family situation. He asked for permission earlier this year, and I didn't hesitate in saying yes. There are some "almost senior" devs on the team I would have told to walk, if they had made the same request.
Yes, but if the headline said "Experienced developers tend to have more freedom, like the ability to work remotely", we wouldn't click on this thread and talk about it.
Even as a noob it plays out that way, I've only been coding for a couple years.
First I was in the office all the time, then I started working from home maybe one day a week....
After everyone got comfortable with me and I proved I could produce my pay increased dramatically, and at the same time work from home turned into 3-5 days a week.
Trust, pay, and working from home and / or remote all are generally related I think.
Don’t we also expect to see this result just by the fact the remote employees working for say Bay Area companies get closer to Bay Area comp then low cost of living level comp? On average then you’ll get a higher average salary for remote employees because non remote employees includes a large number of people outside of Seattle, Bay Area, New York etc.
Another factor may be subjective vs absolute data points in the source.
"Do you work from home more than 50% of your working hours?" has a simple yes/no answer.
"What is your salary, including average annual bonus, after tax?" has a single numeric answer.
"How many years have you been a developer?" sounds simple; but has interpretation issues. Do I count from my first line of code when playing as a kid, my first computer science class, my first paying job as a developer, or some other time? If this is a measure of seniority (e.g. we define senior as anyone who's been coding for 10+ years) there will be a mixed bag.
This may mean that despite trying to factor out variables they're hiding in the noise of bad data; so may still have a hand in the result. i.e. Those developers who've counted 9 years developing from their first salaried job and now earn more and work from home are in the same group as those who've just graduated but counted their time coding from their 8-year-old-self's first foray who are in their first year of the job, and thus on lower salary and less perks.
It lets me escape the local (underpaid) market. That's why I won't work for Gitlab, Mozilla or similar company which pays based on the location, not contribution.
I don't even think it's that complicated. Organizations that offer remote usually won't do it for interns and junior engineers and it probably isn't great for those engineers anyways.
If you just took out all these engineers I wouldn't be surprised if the average jumps by 22%.
You may be onto something. Before Covid managers at my place considered remote work a reward for good behavior and delivering results. Needless to say, those people also tended to get better reviews and salaries.
Or what about location? I know a lot of remote workers who go live in countries that are much cheaper where they couldn't find a job that would pay like this.
