FFS The creation and use of software is an amoral and unethical process

(Marc Cooper) #1

I’m still irritated that I couldn’t make FFS. This talk expresses what I’ve been feeling and failing to express – mostly because I lack the philosophy knowledge. You should read it.


(Andy Wootton) #2

That’s a coincidence, I was thinking about your recent words about all algorithms embedding a set of ethics. You don’t need anything as complex as AI.

Imagine a sales system for a DIY store. There are 14 fence posts in stock. At the same time, 2 people try to order. One wants 7, the other wants 10. What do you do?

I was thinking about possible scenarios for about 20 minutes. Do you prioritise new or existing customers? Does size matter? Do you disappoint 1 or both? If both, do you disappoint by number or percentage? Do you run an auction? Do the extra profits go to the loser, as compensation or to you, as an incentive to cause future shortages? Do you reward patience with a discount or free delivery?

(Jon) #3

(The cartoon mentioned in the talk contains a number of redirects, so I’ll link to it directly here).

That’s a very interesting and encouraging exchange. However, I’d argue that my experience of the UK industry (and my remote view of the US industry) is that most engineers see themselves as cogs that don’t have any responsibility to consider the moral impact of their work.

Here, on this very board, a few years ago, I wrote a frustrated piece about that responsibility. At that time I had recently terminated a telephone interview when, to my surprise, I discovered my keen interviewers wanted me to build a system to carry British materiel, which I believe is a euphemism for “expensive explosive devices to drop accidentally on foreign civilians, for which the Armed Forces will quietly apologise ten years later”.

What depressed me at the time was that my piece was not very interesting to people, and the only substantive replies were standard-issue “but someone else will do it” stuff that essentially excuses the horrific status quo. (I’ll refrain from linking to it here, since I’m not looking to pick a fight).

One of the floor contributions in that talk was:

I’m seeing a lot of narcissistic behaviour coming into the industry

I’m seeing it too, and I think it is not helping the cause of technologists making moral decisions. I give careers advice elsewhere on the web, and I’m seeing several unpleasant trends:

  • Young grads and juniors looking to maximise their salary over anything else, regardless of our understanding of the unhealthy side of the industry and the phenomenon of burnout;
  • Facebook devs and engineers at other Big-N companies telling grads to seek out £80k starting salaries in London, without any understanding of the rarity of such roles, or the talent one would have to demonstrate in order to get such an offer;
  • Recommendations to seek out the most awful firms by name, such as Raytheon and General Dynamics, specifically because they pay handsomely;
  • Morally compromised corporations of various stripes go into the careers fairs of British universities to put the glossy-brochure version of their work to young engineers, who in turn don’t have the negotiating power to turn job offers down.

The general ethos of these chase-the-money types feels like a parody of greed-is-good from the eighties, and there is barely room for a discussion about quality of life, never mind moral considerations that affect other people, thousands of miles away.

My writing above causes me to ponder: should we expect more of this kind of thinking from older and more experienced engineers? As a person ages, they gain the life experience necessary to make moral decisions, and (in general) they achieve the financial independence that allows them the freedom to turn immoral work down. How should we draw our lines of expectation here?

(Andy Wootton) #4

Morality is highly subjective. I think it’s an evolution thing. Individuals exhibit a range of behaviour from selfishness (moderate right) to altruism (moderate left), where self means: self, immediate family, wider family and friends, ‘people like me’ as several hierarchies of tribal identity. Currently, nation is very important, though we don’t all agree what that means and don’t necessarily agree if religion is more important than European. Some people think ‘other’ cultures are a huge threat and must be driven out or destroyed. Brits used to think they had to be converted. People are weird.

(Marc Cooper) #5

Or give them both swords and let them do battle :thinking:

Seriously, though, there are also interesting options around turning it into a problem for the customers to resolve themselves.

One of the issues that I took from that piece is that “platforms” are making all the decisions, hence the ethical issues.

(Marc Cooper) #6

I agree with all of that.

Devs can only rarely be responsible for the ethics of their employer. However, providing they are aware of it, which is not necessarily at all obvious, then they are making a moral choice; I guess you do in any case, though without sufficient information.

(Marc Cooper) #7

John S Nolan, speaker of the above, this morning.

(Andy Wootton) #8

I once accepted a job programming real-time industrial control systems. I resigned from my safe job in the Council because the money and prospects seemed better, only to discover that I would actually be working on laboratory software for animal testing in the pharmaceutical industry. The interesting thing was that I discovered animal testing was the last thing the labs wanted to do. It was forced on them by the US Food & Drugs Administration though many of the tests were considered useless by toxicology experts and others could have been replaced by software simulations. In Germany, the pharmaceutical industry was the major funder of anti-vivisection protests but the government wouldn’t listen. Ethics can be hard. I never went to a lab but colleagues told me the people in the labs were very concerned about the welfare of the animals in their care, like most farmers are.

System Managing in the nuclear industry, I discovered they REALLY cared about safety and the electric companies are far more aware of their environmental responsibilities than the people who demand cheap electricity but they have to balance shareholder profits, low prices, security of supply and environmental damage, which are all contradictory, while successive governments screw them over with incompetent energy policies then blame them for the mess.