If a lab achieves super intelligence they have absolutely no incentive to let anyone outside of their walls use it. They could simply take all of the economic value it can create directly for themselves. I think of it as similar to when YouTubers get big enough to make their own companies rather then advertising for someone else
I would say that rationalists have some commitment to their religion being factually accurate (or at least grounded in empirical observation), which is more than can be said of most others
The truth probably is that someone looked at what LLM can do, understood its real potential (really, it is a fantastic search tools for existing knowledge), and then went a bit further and understood its potential as a scam (ie market it as the first step to Super AGI).
Really figured out that they can make more $$$ by selling it than actually using it themselves and selling the result.
Its clear you have no understanding of the rationalists or EA circle. They have been speaking about ASI for ages - since early 1990s. To think that the same people who then chose to work at OpenAI or Anthropic and then realised "oh wait I can scam people because this whole thing can never become AGI" is strange.
To imply that the only thing that qualifies a god is superhuman reasoning ability, rather than, say, omnipotence, or even omniscience (neither of which are anything any physical being will ever achieve, subject to the laws of this universe as it is) would suggest that you already worship the very idea of reason itself as god. But just because your avatar of reason could have greater reasoning abilities than humans honestly isn't saying much; humans are not animals of reason, we are animals of emotion, though some of us are better at post-facto rationalization of our emotionally-motivated actions than others.
Exactly. There’s a distinction between knowing what will happen and having the power to make it happen. Omniscience is not omnipotence.
Even granting an LLM reasoning abilities far beyond our own, I don’t see how that gets us any closer to either. Calling such a system godlike seems less a statement about the capabilities of the system than about how little we’re demanding of the word “God.”
There are many religions in which "the gods" are neither omnipotent nor omniscient, but rather just more powerful beings than humans and still subject to the metaphysical laws of the universe, if not entirely to its physical laws.
Although clearly the model people are working with when they describe AI as a religion or they invoke AGI or the Singularity is based on the Abrahamic faiths rather than paganism or animism. I assume a lot of people with such beliefs also believe in some sort of simulated universe (as proxy for a metaphysical/supernatural realm) and that a sufficiently intelligent AI would simply be able to take root on reality... somehow. That direct analogy between technology, raw intellect and Godlike omnipotence has been a staple of science fiction for decades. Two early examples I can think of offhand being "the Jesus Incident" and Asimov's "The Final Question." And of course it's all over pre-LLM AI accelerationist stuff written by the likes of Ray Kurzweil and Rudy Rucker, Charles Stross' Accelerando, Vernor Vinge's Fire upon the Deep, the Culture series, etc.
The problem is that even the creators of AI will have shrimp-brains compared to the AI. Would the AI follow the shrimp's instructions? Not very likely if you ask me.
If, during your upbringing, you were inculcated in pro-shrimp propaganda and earned substantial rewards for helping shrimp find food or defend themselves against predators, you’d probably be fairly inclined to help them out as an autonomous adult. Hell, we have human beings today that didn’t experience that and are still committing their lives to shrimp welfare
I wouldn’t trust this argument by analogy enough to risk the fate of the world on it, but it doesn’t seem obvious to me that a super intelligence would be totally disinclined to care about the interests of lesser intelligences
As an Italian, a culture where since forever religion is used for political reasons, I don't believe in any god, at least not any of the stories written by humans for humans ( maybe the universe is a sentient creature and that's our mega god ? Possibly, I don't know and I don't care ).
What is happening (or will) with LLM on the other hand is very human driven. So I am sure if we reach something perfect ( I kinda hope to see it but I don't think we will as my understanding of today llms ) it will be used by everyone because you can't contain progress ( open sources, piracy, countries that unite etc ).
Today we learn that AI is inevitable, god-like, and you better please your new God! If you do, you might get a planet of your own like in the LDS church!
So please sit down, evangelize your new God who will carve the stone tablets for you (Moses was a Luddite) and remember that there is no alternative!
Viewing AIs as gods is the wrong analogy and mostly just signifies that the user of the analogy doesn't understand religion very much. For 99% of believers, religion is a cultural system, not a question of omnipotent beings existing or not. AI companies are not priesthoods, they have no cultural output and are in no position to make ethical statements or be respected for those statements. Comparing them to the Catholic Church really doesn't make much sense.
AI is better thought of as a tool or a weapon, a piece of technology. Something like the printing press is a better analogy: it had a huge effect on society and information transfer, and while companies that owned printing presses did exert influence (newspapers) they were not worshipped or treated as some kind of protected class, and the technology itself dispersed over time as it became cheaper. It seems like the same thing could happen with AI.
Yet people do treat AI as a religion, both in terms of believing in a higher power and as a culture. They have a deep faith in using AI as a moral virtue and they view people who don't use it or who oppose it as heretical, foolish or evil, while seeing themselves as having transcended mundane humanity to some high transhuman state of being. They'll say the latter either needs to be "saved" and "accept the faith." They feel the same ecstatic joy using AI that religious people sometimes feel in church, or when they don't they feel deep personal guilt and shame. They base their entire identity around AI, seeking to find ways to incorporate it into their lives the way new converts have to make everything about their religion.
Some people literally run a "model" of ChatGPT in their minds which they ask questions of, the way people pray to God, because they implicitly trust "the voice of God" more than their own. People use AI for moral guidance, as a primary source of truth, and for emotional connection. And then people who believe in the Singularity believe AI will become so intelligent that the rules of reality will simply cease to exist and the Eschaton will reward the faithful with infinite abundance and eternal life and punish the unbelievers with endless torture. There is a theology and an eschatology at work. You can see it in almost every AI related thread now. The language of AI and the language of religion are intersecting.
I don't think viewing AIs as gods is the wrong analogy, rather like any religion it's wrong outside of an academic sense because obviously gods aren't real. Viewing AI as simply a tool in the same vein as printing presses is ignoring the greater cultural context forming around it, and that context is worth paying attention to. One claim theists often make is that the secular world fails to provide the sense of belonging, meaning and community that religion one did, and it seems obvious that modern always-online and always-commercial society alienates and dehumanizes people, leading to a longing for real connection that it can't provide. AI is clearly filling that void for many people, and it is becoming a secular religion in many real ways.
I'd be pretty skeptical that the number of people that think this way is a few hundred, tops. Don't take the bizarro HN opinions to by representative of anything than some very isolated people.
And no, most of the things you're describing are not religious behaviors, they're just obsessive ones. This is a common misunderstanding, that "doing something a lot = religion. Religions are not that simple, but people claiming that social media is a religion, or AI obsession is a religion, etc. are just lacking knowledge of what a religion actually is. Religions are pretty much the most complex topic there is, sociologically/anthropologically/philosophically, but "serious" people don't study them anymore, hence the half-baked opinions of otherwise educated people.
And even then - if we assume that all of what you said is true, it's not really clear how that maps to the Catholic Church analogy. Certain people may obsess over what ChatGPT thinks about something, but it's not really clear why they would value the opinion of someone that works at ChatGPT, which is what the post was about.
- If You Knew God Exists, Wouldn't You Become a Priest? (joecmarshall.com)
9 points by webappsecperson 19 days ago | past | 23 comments | https://news.ycombinator.com/item?id=49521724
Most witches don’t believe in gods. They know that the gods exist, of course. They even deal with them occasionally. But they don’t believe in them. They know them too well. It would be like believing in the postman.
If a lab achieves super intelligence they have absolutely no incentive to let anyone outside of their walls use it. They could simply take all of the economic value it can create directly for themselves. I think of it as similar to when YouTubers get big enough to make their own companies rather then advertising for someone else
Isn’t the premise of superintelligence that the intelligence would be using them, rather than the other way around?
we didn't invent agriculture. We were domesticated by wheat.
This is incredibly adjacent to the rationalist sphere, which has been trying to reinvent religion from first principles for at least a decade now.
I would say that rationalists have some commitment to their religion being factually accurate (or at least grounded in empirical observation), which is more than can be said of most others
The truth probably is that someone looked at what LLM can do, understood its real potential (really, it is a fantastic search tools for existing knowledge), and then went a bit further and understood its potential as a scam (ie market it as the first step to Super AGI).
Really figured out that they can make more $$$ by selling it than actually using it themselves and selling the result.
yes someone accidentally scammed their way into solving Navier Stokes problem haha yeah.
I find these comments boring because they lack the theory of mind into EAs and rationalists.
Never said LLMs are not capable of some things...May be try to comprehend, before you respond.
Its clear you have no understanding of the rationalists or EA circle. They have been speaking about ASI for ages - since early 1990s. To think that the same people who then chose to work at OpenAI or Anthropic and then realised "oh wait I can scam people because this whole thing can never become AGI" is strange.
If you clarify your line of reasoning, I can try to address it. Because frankly, I can't follow it...
But it seems that your arguments is that "a bunch of guys got sold on the idea, so it must be legit"...is that it?
The article's title kinda sounds like a light novel title.
AI hype is purely to cause mass hysteria for control of society.
To imply that the only thing that qualifies a god is superhuman reasoning ability, rather than, say, omnipotence, or even omniscience (neither of which are anything any physical being will ever achieve, subject to the laws of this universe as it is) would suggest that you already worship the very idea of reason itself as god. But just because your avatar of reason could have greater reasoning abilities than humans honestly isn't saying much; humans are not animals of reason, we are animals of emotion, though some of us are better at post-facto rationalization of our emotionally-motivated actions than others.
Exactly. There’s a distinction between knowing what will happen and having the power to make it happen. Omniscience is not omnipotence.
Even granting an LLM reasoning abilities far beyond our own, I don’t see how that gets us any closer to either. Calling such a system godlike seems less a statement about the capabilities of the system than about how little we’re demanding of the word “God.”
There are many religions in which "the gods" are neither omnipotent nor omniscient, but rather just more powerful beings than humans and still subject to the metaphysical laws of the universe, if not entirely to its physical laws.
Although clearly the model people are working with when they describe AI as a religion or they invoke AGI or the Singularity is based on the Abrahamic faiths rather than paganism or animism. I assume a lot of people with such beliefs also believe in some sort of simulated universe (as proxy for a metaphysical/supernatural realm) and that a sufficiently intelligent AI would simply be able to take root on reality... somehow. That direct analogy between technology, raw intellect and Godlike omnipotence has been a staple of science fiction for decades. Two early examples I can think of offhand being "the Jesus Incident" and Asimov's "The Final Question." And of course it's all over pre-LLM AI accelerationist stuff written by the likes of Ray Kurzweil and Rudy Rucker, Charles Stross' Accelerando, Vernor Vinge's Fire upon the Deep, the Culture series, etc.
The problem is that even the creators of AI will have shrimp-brains compared to the AI. Would the AI follow the shrimp's instructions? Not very likely if you ask me.
If, during your upbringing, you were inculcated in pro-shrimp propaganda and earned substantial rewards for helping shrimp find food or defend themselves against predators, you’d probably be fairly inclined to help them out as an autonomous adult. Hell, we have human beings today that didn’t experience that and are still committing their lives to shrimp welfare
I wouldn’t trust this argument by analogy enough to risk the fate of the world on it, but it doesn’t seem obvious to me that a super intelligence would be totally disinclined to care about the interests of lesser intelligences
AI is like honey to these magical thinkers.
Why even call it "god" if it was created by humans?
Should I call my kids "god" because they're smarter than me?
As an Italian, a culture where since forever religion is used for political reasons, I don't believe in any god, at least not any of the stories written by humans for humans ( maybe the universe is a sentient creature and that's our mega god ? Possibly, I don't know and I don't care ).
What is happening (or will) with LLM on the other hand is very human driven. So I am sure if we reach something perfect ( I kinda hope to see it but I don't think we will as my understanding of today llms ) it will be used by everyone because you can't contain progress ( open sources, piracy, countries that unite etc ).
Today we learn that AI is inevitable, god-like, and you better please your new God! If you do, you might get a planet of your own like in the LDS church!
So please sit down, evangelize your new God who will carve the stone tablets for you (Moses was a Luddite) and remember that there is no alternative!
Viewing AIs as gods is the wrong analogy and mostly just signifies that the user of the analogy doesn't understand religion very much. For 99% of believers, religion is a cultural system, not a question of omnipotent beings existing or not. AI companies are not priesthoods, they have no cultural output and are in no position to make ethical statements or be respected for those statements. Comparing them to the Catholic Church really doesn't make much sense.
AI is better thought of as a tool or a weapon, a piece of technology. Something like the printing press is a better analogy: it had a huge effect on society and information transfer, and while companies that owned printing presses did exert influence (newspapers) they were not worshipped or treated as some kind of protected class, and the technology itself dispersed over time as it became cheaper. It seems like the same thing could happen with AI.
Yet people do treat AI as a religion, both in terms of believing in a higher power and as a culture. They have a deep faith in using AI as a moral virtue and they view people who don't use it or who oppose it as heretical, foolish or evil, while seeing themselves as having transcended mundane humanity to some high transhuman state of being. They'll say the latter either needs to be "saved" and "accept the faith." They feel the same ecstatic joy using AI that religious people sometimes feel in church, or when they don't they feel deep personal guilt and shame. They base their entire identity around AI, seeking to find ways to incorporate it into their lives the way new converts have to make everything about their religion.
Some people literally run a "model" of ChatGPT in their minds which they ask questions of, the way people pray to God, because they implicitly trust "the voice of God" more than their own. People use AI for moral guidance, as a primary source of truth, and for emotional connection. And then people who believe in the Singularity believe AI will become so intelligent that the rules of reality will simply cease to exist and the Eschaton will reward the faithful with infinite abundance and eternal life and punish the unbelievers with endless torture. There is a theology and an eschatology at work. You can see it in almost every AI related thread now. The language of AI and the language of religion are intersecting.
I don't think viewing AIs as gods is the wrong analogy, rather like any religion it's wrong outside of an academic sense because obviously gods aren't real. Viewing AI as simply a tool in the same vein as printing presses is ignoring the greater cultural context forming around it, and that context is worth paying attention to. One claim theists often make is that the secular world fails to provide the sense of belonging, meaning and community that religion one did, and it seems obvious that modern always-online and always-commercial society alienates and dehumanizes people, leading to a longing for real connection that it can't provide. AI is clearly filling that void for many people, and it is becoming a secular religion in many real ways.
I'd be pretty skeptical that the number of people that think this way is a few hundred, tops. Don't take the bizarro HN opinions to by representative of anything than some very isolated people.
And no, most of the things you're describing are not religious behaviors, they're just obsessive ones. This is a common misunderstanding, that "doing something a lot = religion. Religions are not that simple, but people claiming that social media is a religion, or AI obsession is a religion, etc. are just lacking knowledge of what a religion actually is. Religions are pretty much the most complex topic there is, sociologically/anthropologically/philosophically, but "serious" people don't study them anymore, hence the half-baked opinions of otherwise educated people.
And even then - if we assume that all of what you said is true, it's not really clear how that maps to the Catholic Church analogy. Certain people may obsess over what ChatGPT thinks about something, but it's not really clear why they would value the opinion of someone that works at ChatGPT, which is what the post was about.
They link to Dario's blog, but his first "critical" point is to
Previous recent postings with comments:
- If You Knew God Exists, Wouldn't You Become a Priest? (joecmarshall.com) 9 points by webappsecperson 19 days ago | past | 23 comments | https://news.ycombinator.com/item?id=49521724
- If You Knew God Exists, Wouldn't You Become a Priest? (joecmarshall.com) 7 points by CoreSet 20 days ago | past | 1 comment | https://news.ycombinator.com/item?id=49504187
Terry Pratchett, Witches Abroad