Their Technocautionism
I enjoy listening to sci-fi authors who suggest that technology has neutral valence. Despite being queried about AI risk in a recent Existential Hope podcast episode, Ken Liu offered a more holistic assessment of technology:
As for risk: technology itself is neither problem nor solution.
Liu references Shelley’s Frankenstein, comparing the creature’s consumption of human text to large language models; he sees parallels in how it develops an internal structure of the world and interrogates what it is to be human. Having not read Frankenstein yetIt’s on the bedside table though., the parallel to large language models (LLMs) was striking for me. Whether an embodied creation—such as Frankenstein’s or a humanoid robot—or LLMs (and other software-based “intelligence”) manifest into monsters (or friends) would be an indictment (or affirmation) of who we are, as humans.
The monster’s reaction is not merely testament to how valence emerges; more importantly, it’s a reminder of the centrality of human nature in shepherding society to better outcomes. Issues arise when “technology” uses us. My current feeling is that the lion’s share of technologies are initially pitched as solvers of problems (or meaning-making machines) but, if its creator manages to survive, the objective of the entity creating the technology eventually pivots to profit-making. This happens because the incentive itself is to generate profits, not to solve the actual issue. It’s why I have cynical takes—like Dating Apps are Indian Parents-as-a-Service—but I feel like a lot of this is limited to software solutions.
Yes, users of the technology do play their part too—or they must believe that they do. This comment by Josh Kline comes to mind:
..it’s not just Big Tech and corporations are going to determine what our futures are like but that individuals can do this and, in a sense, ought to do it.
Josh Kline
Despite writing one of the dystopian sci-fi hall-of-famers because of which many cast Aldous Huxley a techno-pessimistCitation needed but I have seen this claimed on Twitter because of his Brave New World., I see him in a more optimistic light on account of two reasons.
The first is my reading of his words on the nature of technology:
Sabbath was made for man and not man for the Sabbath. And in the same way technology was made for man and not man for technology but unfortunately the development of recent social and scientific history has created a world in which man seems to be made for technology rather than the other way around.
which asserts that netural valence mentioned earlier.
The second is what he says a little later in the interview about his final book, The Island, which is his perspective on what a “practical utopia”, might look like:
I’ve just finished a kind of utopian fantasy, which is the opposite of Brave New World, which is about a society in which a serious effort is made to help its members to realise their desirable potentialities. This is an attempt to write what may be called a practical utopia. Nothing is easier, of course, than to enunciate ideals and to say “well wouldn’t it be nice if everybody were good and kind and loving etc etc”. Of course, it would be very nice but the point is how do you implement these ideals? How do you fulfil your good social and psychological intentions? And when you come down to this problem, you see it’s a very complex problem of organising family life, organising education, organising sexual life, organising social and economic life.
I will continue down this track in a future post but, for now, I will end this short essay by labelling those who believe in the neutral valence of technology while also presenting ideas for its cautious development (as opposed to preventing its development altogether) as technocautionists. The broader movement is technocautionism.