{"id":829,"date":"2025-04-26T05:20:04","date_gmt":"2025-04-26T05:20:04","guid":{"rendered":"https:\/\/cjai.co.uk\/?post_type=usersubpost&p=829"},"modified":"2026-02-02T18:52:32","modified_gmt":"2026-02-02T18:52:32","slug":"agi-fear-and-loathing-in-silicon-valley","status":"publish","type":"usersubpost","link":"https:\/\/cjai.co.uk\/?usersubpost=agi-fear-and-loathing-in-silicon-valley","title":{"rendered":"AGI: Fear and Loathing in Silicon Valley"},"content":{"rendered":"

Four years ago, one of the most prominent AI safety researchers wrote about their warnings: \u201cThis is not fear-mongering; we don\u2019t have an adequate amount of fear in the AI researcher community \u2014 the amount necessary to ensure sufficient precautions.\u201d (1, p. 15) Today, the landscape has shifted dramatically. The current media ecosystem is saturated with discussions of AI and its advancement, from claims of near Artificial General Intelligence to both its disruptive potential and catastrophic risks. The media, in giving a broad platform to discourse around AGI\u2014particularly to evangelists and companies actively pursuing it\u2014does a disservice to the public and often functions merely \u201cas the press office of tech giants and their private interests.\u201d (2) Both utopian and dystopian AGI narratives ultimately fuel the same hype cycle, empowering Silicon Valley venture capital. And while existential risks from AGI merit consideration, public resources and critical attention might be better placed in reducing the real, present-day harms caused by narrow AI.<\/p>\n

AGI risk and hype<\/strong>
\nThe discourse around artificial general intelligence is deeply polarized. On one side, AI safety researchers point to scenarios in which an AGI could induce a technological singularity, leading to existential risks or new forms of suffering. (3) On the other, AGI systems that do not necessarily surpass human intelligence but still cause substantial disruption: enabling social manipulation, new types of warfare, or global power shifts. (3, 4)<\/p>\n

Since the release of ChatGPT in 2022, discourse on AI and AGI has exploded, with public debate rapidly steering toward AI safety and existential risks\u2014though expert opinions remain divided. (5) At the same time, fuelled by rapid advances in large language models, an equally vocal camp promotes the idea that AGI is near and will disrupt every industry. (6) Neither side is free from vested interests, and both contribute to a public conversation shaped less by clarity than by hype.<\/p>\n

AGI narratives<\/strong>
\nStudies have shown the current prevalence of negative \u201cdystopian\u201d narratives on AI, with the exception of \u201csolutionism,\u201d which is \u201ca bright-siding alternative where AI would simply solve all humanity\u2019s problems.\u201d (7) These narratives are deeply influenced by societal expectations of the future, which also shape social action. (8) Moreover, \u201cAI, with its false narratives, coupled with the human psyche that seems to inherently crave hype, is a susceptible nexus.\u201d (9)
\nThe result is a public conversation dominated by extremes\u2014with AGI either elevated to a utopian \u201csilver bullet\u201d capable of solving all human problems or condemned as an existential threat, fuelling moral panic. These polarised portrayals risk distracting the public from understanding the actual capabilities of current AI systems, and from recognizing the immediate social, environmental, and political harms AI already enables. (7, 9)<\/p>\n

Techno-futuristic eschatology<\/strong>
\nTo better understand these dynamics, we can look at the ideological foundations of the people and companies that might profit from them. Silicon Valley\u2019s obsession with AGI is deeply rooted in its cultural and ideological environment. According to Gebru and Torres (2024), this is broadly centred around the so-called TESCREAL bundle, a cluster of connected beliefs including \u201ctranshumanism, Extropianism, singularitarianism, (modern) cosmism, Rationalism, Effective Altruism, and longtermism.\u201d (10)<\/p>\n

These frameworks drive Silicon Valley\u2019s vision of AGI as a transformative solution to existential challenges, reinforcing the region\u2019s characteristic faith in technology as humanity\u2019s saviour. Effective Altruism, central to this ethos, encourages individuals to optimize their resources to maximize global impact, often through strategies like “earn to give,” where wealth generation is channelled into high-utility causes. This approach fuses capitalist ambition with moral self-justification, echoing what Paulina Borsook described over twenty years ago as a \u201ctechnolibertarian culture of crypto-individualism.\u201d (11)<\/p>\n

Silicon Valley in particular favours longtermism, an influential offshoot of EA that focuses on humanity\u2019s distant future and the mitigation of existential risks, particularly those posed by AGI. (12) Figures like Sam Altman and Elon Musk are closely associated with these ideas, embedding them within the missions of organizations such as OpenAI and Anthropic. OpenAI\u2019s stated goal of ensuring AGI benefits all of humanity reflects such longtermist priorities, as does Anthropic\u2019s emphasis on building safe and aligned AI systems.<\/p>\n

So what?<\/strong>
\nAs we have explored, the narratives around AGI are symptomatic of a broader cultural and ideological struggle, where Silicon Valley and its venture capitalists play a central role. The pervasive media hype around AGI\u2014whether optimistic or dystopian\u2014contributes to a distorted public understanding, driven by competing business interests, ideological commitments, and a desire for dominance in the technological and cultural zeitgeist. By elevating AGI to a status of existential importance, these narratives divert attention from the immediate harms already enabled by current AI systems, including exploitation, bias, and power concentration.<\/p>\n

As argued by Gebru and Torres, one critical issue is the lack of scrutiny over why AGI is considered a desirable goal by many in the field. (10) Indeed, this techno-utopian belief in a system capable of solving humanity\u2019s grandest challenges conveniently ignores technology\u2019s historic tendency to exacerbate inequality and marginalization.<\/p>\n

What\u2019s more, the pursuit of AGI has already resulted in significant harms, from environmental degradation to the amplification of harmful ideologies. These real-world consequences are often downplayed or ignored in favour of speculative debates about superintelligence and its risks, even though the former are far more pressing. The ideological frameworks of TESCREAL, particularly longtermism, prioritize hypothetical futures over present-day concerns. Thus, in their effort to control the future, they risk reinforcing authoritarian tendencies under the guise of altruism. Moreover, the TESCREAL ideologies have clear ties to historical instances of authoritarianism and illiberal politics, such that \u201cThe biggest risk AI poses right now is that alarmists will use the fears surrounding it as a cudgel to enact sweeping policy reforms.\u201d (13)<\/p>\n

In the ideological landscape of Silicon Valley, AGI is not merely a technical endeavour but a cultural project rooted in transhumanism, libertarianism, and effective altruism. These ideologies present a vision of the future where technology and those who control it hold unrivalled power over humanity\u2019s fate. And while this is framed as a benevolent mission, it is essential to contest this concentration of influence and capital, and the risks of subordinating public welfare to private ambitions that come with it.
\n—<\/strong>
\nThe current discourse around AGI remains polarized between utopian promises and apocalyptic fears. This dynamic, fuelled by Silicon Valley\u2019s vested interests, leaves little room for the nuanced skepticism necessary for informed decision-making. Even strong advocates for AGI risks like Yampolskyi (1) and Baum (14) suggest that healthy skepticism is vital to maintaining intellectual integrity and preventing such narratives from being weaponized for political or financial gain.
\nWe should not dismiss the potential risks of AGI outright but ensure that discussions remain grounded in reality. Ultimately, the AGI debate underscores the need to refocus on present challenges. The future of AI should not hinge on speculative fantasies of general intelligence but on the deliberate and equitable deployment of technology to benefit humanity as it exists today. Only by disentangling ourselves from the seductive myths of AGI can we begin to chart a more just and sustainable technological future.<\/p>\n

1. YAMPOLSKIY, Roman V. AI Risk Skepticism. Online. 17 July 2021. arXiv. arXiv:2105.02704. [Accessed\u00a013\u00a0January\u00a02025].
\n2. SIGNORELLI, Andrea Daniele. L\u2019intelligenza artificiale generale ha rotto. Wired Italia. Online. 4 November 2024. [Accessed\u00a016\u00a0January\u00a02025]. Available from: https:\/\/www.wired.it\/article\/intelligenza-artificiale-generale-ricerca-miti\/
\n3. EVERITT, Tom, LEA, Gary and HUTTER, Marcus. AGI Safety Literature Review. Online. 21 May 2018. arXiv. arXiv:1805.01109.
\n4. YAMPOLSKIY, Roman V. Taxonomy of Pathways to Dangerous AI. Online. 11 November 2015. arXiv. arXiv:1511.03246. [Accessed\u00a016\u00a0January\u00a02025].
\n5. STRICKLAND, Eliza and ZORPETTE, Glenn. The AI Apocalypse: A Scorecard. How worried are top AI experts about the threat posed by large language models like GPT-4? IEEE Spectrum. Online. 21 June 2023. Available from: https:\/\/spectrum.ieee.org\/artificial-general-intelligence
\n6. SIEGEL, Eric. Elon Musk Predicts Artificial General Intelligence In 2 Years. Here\u2019s Why That\u2019s Hype. Forbes. Online. 10 April 2024. [Accessed\u00a016\u00a0January\u00a02025]. Available from: https:\/\/www.forbes.com\/sites\/ericsiegel\/2024\/04\/10\/artificial-general-intelligence-is-pure-hype\/
\n7. CHUBB, Jennifer, REED, Darren and COWLING, Peter. Expert views about missing AI narratives: is there an AI story crisis? AI & SOCIETY. June 2024. Vol.\u00a039, no.\u00a03, p.\u00a01107\u20131126. DOI\u00a010.1007\/s00146-022-01548-2.
\n8. BRENNEN, J Scott, HOWARD, Philip N and NIELSEN, Rasmus K. What to expect when you\u2019re expecting robots: Futures, expectations, and pseudo-artificial general intelligence in UK news. Journalism. January 2022. Vol.\u00a023, no.\u00a01, p.\u00a022\u201338. DOI\u00a010.1177\/1464884920947535.
\n9. KLARMANN, Noah. Artificial Intelligence Narratives: An Objective Perspective on Current Developments. Online. 18 March 2021. arXiv. arXiv:2103.11961. [Accessed\u00a013\u00a0January\u00a02025].
\n10. GEBRU, Timnit and TORRES, \u00c9mile P. The TESCREAL bundle: Eugenics and the promise of utopia through artificial general intelligence. First Monday. Online. 14 April 2024. [Accessed\u00a016\u00a0January\u00a02025]. DOI\u00a010.5210\/fm.v29i4.13636.
\n11. BORSOOK, Paulina. Cyberselfish: Ravers, Guilders, Cyberpunks, and Other Silicon Valley Life-Forms. Yale Journal of Law and Technology. Spring 2001. Vol.\u00a04, no.\u00a01.
\n12. ONGWESO, Edward Jr. OK, WTF Is \u2018Longtermism\u2019, the Tech Elite Ideology That Led to the FTX Collapse? Vice. Online. 23 November 2022. [Accessed\u00a016\u00a0January\u00a02025]. Available from: https:\/\/www.vice.com\/en\/article\/ok-wtf-is-longtermism-the-tech-elite-ideology-that-led-to-the-ftx-collapse\/
\n13. TROY, Dave. The Wide Angle: Understanding TESCREAL \u2014 the Weird Ideologies Behind Silicon Valley\u2019s Rightward Turn. The Washington Spectator. Online. 1 May 2023. [Accessed\u00a016\u00a0January\u00a02025]. Available from: https:\/\/washingtonspectator.org\/understanding-tescreal-silicon-valleys-rightward-turn\/
\n14. BAUM, Seth D. Superintelligence Skepticism as a Political Tool. Information. 22 August 2018. Vol.\u00a09, no.\u00a09, p.\u00a0209. DOI\u00a010.3390\/info9090209.
\n15. Yasmine Boudiaf & LOTI \/ \u00a0Better Images of AI<\/a>\u00a0\/\u00a0CCBY-4.0<\/a><\/p>\n<\/div><\/div><\/div><\/div><\/div>\n","protected":false},"featured_media":830,"parent":0,"template":"","usp-category":[],"class_list":["post-829","usersubpost","type-usersubpost","status-publish","has-post-thumbnail","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/cjai.co.uk\/index.php?rest_route=\/wp\/v2\/usersubpost\/829","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cjai.co.uk\/index.php?rest_route=\/wp\/v2\/usersubpost"}],"about":[{"href":"https:\/\/cjai.co.uk\/index.php?rest_route=\/wp\/v2\/types\/usersubpost"}],"version-history":[{"count":2,"href":"https:\/\/cjai.co.uk\/index.php?rest_route=\/wp\/v2\/usersubpost\/829\/revisions"}],"predecessor-version":[{"id":899,"href":"https:\/\/cjai.co.uk\/index.php?rest_route=\/wp\/v2\/usersubpost\/829\/revisions\/899"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cjai.co.uk\/index.php?rest_route=\/wp\/v2\/media\/830"}],"wp:attachment":[{"href":"https:\/\/cjai.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=829"}],"wp:term":[{"taxonomy":"usp-category","embeddable":true,"href":"https:\/\/cjai.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fusp-category&post=829"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}