Writes someone on the internet:
Software was a field in which human genius, even individual genius, could flourish against
corporate titans. (See Linux or Minecraft).
But a future in which software is created by burning through tokens is one where capital again rules[1]
which echoes this quote from my post from yesterday:
Man’s collective mastery of nature, moreover—even if we could ignore the mounting evidence that this too is largely an illusion—can hardly be expected to confer a sense of confidence and well-being when it coexists with centralizing forces that have deprived individuals of any mastery over the concrete, immediate conditions of their existence. The collective control allegedly conferred by science is an abstraction that has little resonance in every-/day life. Scientific technology has made life more secure in many ways, but its destructive side, most dramatically revealed by the development of nuclear weapons, adds to the feeling of insecurity that derives from the individual’s diminishing control over his immediate surroundings. […] The structure of modern experience gives […] far more encouragement to a sense of helplessness, victimization, cynicism, and despair.[2]
It seems to be an unfortunate fact about technological progress that, while it may — favourably seen — provide many comforts, these benefits accrue at the collective level, while individuals are left with ever less agency.
I remember an episode from my younger years when, after a computer outage, outrage stopped me, having witnessed the impasse this brought the (woman) clerk into when trying to conclude business in a local tobacco & magazines shop. I knew that if it was the shop of someone who trusted her, she would have seen the possibility and been able to register the transaction on a piece of paper and, though things would have moved slower for a little while, no feelings of helplessness would have been necessary.
Examples abound. You can’t repair your car on your own anymore. Another one: I know someone who bought an off-road buggy. The thing shuts itself off after 100 km and a guy has to come out and inspect it, no matter what. The same, I’ve heard, goes for many big agricultural machines.
Christopher Lasch, whose words I quoted above in the longer quote, notes that even the state socialists of the early twentieth century — the people who claimed to speak for the worker — simply assumed that workers would remain “a cog in the machine” but “would gladly submit to factory discipline if their material position could be made tolerably secure. The pleasures of consumption would make up for the monotony of the job.” De-risked, comfortable, and with nothing left to decide.[3]
The particular relevance to the current moment is, of course … drumroll … AI. And having said all that, despite immersing myself for a couple of days in AI-skeptic material, I haven’t budged an inch from my AI-maximalist standpoint as a developer, which is basically that there is no point to anything without AI. To me it seems pointless not to use it. Mind you — pointless, not futile; futile in the sense that this wave rolls over us and we can’t do anything about it. It’s more from the ego perspective: I can’t see the point of doing anything the hard way when there is an easy way (with a couple of provisos — but I’m talking here about getting things done).
There is this quote from Wendell Berry which I came across recently:
We must achieve the character and acquire the skills to live much poorer than we do. We must waste less.[4]
I agree with it in sentiment. Good art, it seems to me, is often subtractive. A line of wisdom I once heard: “The more I know, the less I use.” Elsewhere, writing about why he would not buy a computer, Berry lets us know:
My standards are not speed, ease, and quantity. I have already left behind too much evidence that, writing with a pencil, I have written too fast, too easily, and too much. I would like to be a better writer, and for that I need help from other humans, not a machine.[5]
While a specific word for too much of “mid” quality will immediately be on the tip of everyone’s tongue, we need to remember that Berry’s lines are from the late ’80s and early ’90s, when we worried a lot about the limited carrying capacity of Mother Earth. Not that this is no longer a topic — but as long as resources seem more or less unlimited, I am not sure what would make us collectively resist the temptation. Not that arguments “from externalities” cannot be made. Far from it. But I think the economics are just too compelling. To put it crudely, even distastefully: an argument can be made that reduces the human elevator operator to someone who digs a hole in the ground only to shovel the earth back in. In both cases, one needs to remember, that person must be fed — and that takes resources as well. There appears to be simply no point in doing things the hard way. Thus:
The tokens must flow!
But remember: it is always the empire, never the one mining the spice, that controls the fates.
I am second-guessing my own motivation here. I wrote about this in another essay, where I adduced the example of the “breakers” from Stephen King’s The Dark Tower, comparing them to the archetype of the engineer: the one who is addicted to cracking problems while being used by Capital — but willingly so.[6] I recognise myself in this. I want to crack problems, but not already-solved ones — only exciting ones. I’m currently working under the assumption that those exciting problems will end up being solved by me, but my argument here will be that this might not be the case: I suspect AI automation follows a trend I see in technological change generally, namely that decision-making percolates upwards. Already I am becoming something of an architect or product manager, even for my own stuff. But it might not stop there.
Consider the following: I see that during the last year I kept returning to the stance of “we as an industry will figure out how to harness this thing” — meaning, to make something unreliable (AI, in this case) reliable, to echo Steve Yegge’s definition of engineering. Secretly hoping to make some sort of contribution, to find something out, to be at the forefront of employing AI the right way, of building the right harness around it. Only to see it become obsolete three months later, because the big companies have trained it into their model or harness. In case you’re wondering: I actually (maybe surprisingly) feel nothing about it. This may have to do with that right now one can see that AI excels at verifiable tasks but sucks clearly at writing. This shall be my cope, as the kids call it — or, with a bit more dignity: my consolation. It still seems clearly bad at writing.
What would be lost? One is inclined to say: human civilisation is literally built around text. Texts get written, people exchange thoughts. People read books, synthesise, write. Incredible new connections are made — in individuals’ brains. It is deeply ingrained into our humanistic and enlightened self-understanding that participation in this process, even if you yourself don’t play mayor league, but merely by having visited a school, are part of. Having this done by machines is utterly degrading.
Francis Fukuyama recently tried to frame things[7] positively, saying, sure, people might have shorter attention spans (and not think deeply on an individual level), but exchange happens fast, and memetically. I did not find this entirely helpful. Literal memes aside, you can see this play out in the podcast and YouTube sphere, where influencers echo variations of the “current thing” endlessly off of each other. Precious few of them actually read anything. In contrast to those in power, who, rest assured, will give up neither reading nor thinking, nor let their kids become iPad kids. A reversion to the historical mean, so to speak, when relatively few were in possession and control of the written word.
On a grander scale, I hate to say, all this can still be considered a win for the collective called “humankind”, the hivemind of the technological animal. Onwards, then, to the stars! Actually, I am on board with that. Call me conflicted.
As for the possibilities opened up by the increasingly incredible capabilities of LLMs on verifiable tasks, I got to see them up close this week. I had written some code-review automation, and then asked myself: wait a sec — I can feed the results directly back to my agent. Eureka! In that moment, beyond merely knowing about the idea of the “dark factory” for software, it became real for me. I could see it.
This is intelligence. Intelligence without wisdom.
Footnotes
User Dov Jacobson made this comment on the Substack article What it feels like to work with Mythos (link).
Christopher Lasch, The True and Only Heaven: Progress and Its Critics (1991), pp. 385–86.
Ibid., p. 319.
Wendell Berry, Word and Flesh, in What Are People For? - Essays (1990); p. 201.
Wendell Berry, Feminism, the Body, and the Machine, in Ibid., p. 190.
Mindless Drones - AI and the future of work, @eighttrigrams.net.
In some YouTube video.