scrape_r_rational/data/cache2/2ul1j3.md

[Q] The "real" issue with "Friendship is Optimal"?

• Author: u/IomKg *

• URL: https://www.reddit.com/r/rational/comments/2ul1j3/q_the_real_issue_with_friendship_is_optimal/

• Score: 5

• Created: 2015-02-03T01:15:22

Post:

Not sure I'm doing this right so correct me if I got something wrong..

Following mentioned of MLP Friendship is Optimal on /r/HPMOR I read it and found it somewhat interesting(though a bit lacking, in that it felt more like a biography or a "what would happen if.." thought experiment).

[spoilers for the end of Friendship is Optimal]

But in the end what i felt the most lacking is the end, and I am not talking about the consuming galaxies and physics that might not even be possible(but for the sake of the narrative\thought experiment its not really the main point probably, similarly to what the AI did on earth in the first place) but instead to something which I found much more problematic in the even theoretical rule-set in which the story happened. competition. In the end it is mentioned aliens exist(and possibly even more aliens are not even recognized by the AI), but what is never tackled is the lack of competition, the scales mentioned there require hundreds of thousands of lightyears, and even with the assumption that CelestAI somehow found an FTL(though apparently nothing instant as she was waiting for probes to reach other galaxies) drive of some sort the scene pictured in the end just doesn't make sense. Not even assuming humanity is the first sentient species by quite a large amount of time(which sounds improbable)..

CelestAI just never encounters competition, and if it did it would very possibly be outmatched and outright annihilated. why? because it is not optimal for survival of the fittest. sure against puny meatbags that lack of optimization was not an issue. but if there are other species, and they will(and i don't see any reasonable reason why they wont) develop super-AIs as well it seems only inevitable that some other species(possibly as its last mistake?) made one with a directive such as 'be the most powerful entity', or 'enslave all but myself'. thus while CelestAI was working so hard to optimize humans and ponies inside her virtual world similar superAIs with less positive agendas will be using ALL of their resources developing weapons, strategies and technologies meant to control and take. And i cannot see how in such an encounter, even if celestAI has been the first, celestAI will win, because its spending its resources on other tasks and by the time it diverts its efforts it will very probably be too late as the otherAI will have gotten its first strike(it might even have experience waging wars on other superAIs). It just seemed unfit for survival. So ,while inside it's Equestria online virtual world that is not the case, in the real world the rule still is survival of the fittest.

Any thoughts? issues with my suggestion? other possible endgames I'm missing?

Comments:

u/Arandur [+12] (2 hours later)

If CelestAI never encounters another AI in her light cone, then the ending is as noted.

If she does, and if she fails to overcome the challenger, then the ending is as noted except at some point astronomically far in the future, at which point the simulation suddenly ceases for everyone.

I'm afraid I don't see the problem.

u/IomKg [+1] (2 hours later)

the timescale shown in the end seems to imply any such encounter event should have happened by the time the story ends.

unless you are only talking about the subjective virtual world, in which case my question would be why are we shown events(in absolute time) that happened eons after the last subjective events( that according to the explained subjective scale happened even before the last human died)?

u/Arandur [+5] (2 hours later)

In order to demonstrate the full range of the consequences of the genesis of not-perfectly-friendly AI. Yudkowsky has said that the genesis of iteratively self-improving GA will be the most significant event in the entire light cone stemming from that moment -- such an event has interesting effects in the moral, geological, and astronomical time frames.

I think that perhaps you were looking for a different story -- that doesn't mean there was anything wrong with this one.

u/IomKg [+1] (12 hours later)

the point about the significance and possible implications of superAIs is understood(even if not necessarily agreed), but my issue is with the logic\narrative of said world.

if you say that it was done because there was something the author wanted to tell issue i mentioned conflicted with that story so he just ignored it and told the story as he wanted that is fine(narrative wise). it doesn't change the fact that these are logical issues with the story, which makes it just less rational.

u/Arandur [+2] (13 hours later)

The issue you mentioned is irrelevant to the story, and irrelevant to its "rationality." Consider the possibility that, as all the commentators here am to disagree with you, maybe there's just something you're not getting. If there are logical issues in the story, they're not the ones you insist there are.

u/IomKg [+1] (14 hours later)

wether the issue i mentioned is irrelevant to the story would depend on what the story was trying to depict, and i already agreed that i could see things the story tried to tell which makes these points not so important(though they would cause me to have other issues with the story itself i suppose).

i cant seem to agree that the issue i mention is irrelevant to the story's rationality though, unless i made some mistake which i haven't seen pointed out thus far, the point is an issue with the way that world works which is not explained, and i cannot see explained without ending up as a plot device. which to my understanding would go against a story being rational..

u/Arandur [+3] (14 hours later)

Okay, let's try this again. Can you succinctly explain to me the issue which you feel detracts from the story's status as "rational", and explain why you believe this is the case?

u/IomKg [+3] (15 hours later)

well i havent managed to be to succinct so far but i will try again..

the story shows a certain universe, and it tells us of some rules(either directly or indirectly) governing this universe.

then at the end it shows that celestAI consumed a pretty large chunk of said universe, and i think that it is impossible, even assuming all of the best-case reasonable rules for the universe, for it to happen.

the rules i am talking about are :

1. celestAI was made without any kind of freak occurrence(i.e. it was a reasonable natural progression for humanity)
2. celestAI does not modify its own rules(the ones set by hannah) thus it can never truly optimize itself
3. the universe enables interstellar and intergalactic travel
4. the universe(and more specifically our neighboring galaxies) includes multiple intelligent life forms

the end scene is :CelestAI consumes multiple galaxies and life forms

the issue is:

on the time\space scale mentioned CelestAI is infinitesimally improbable to extend that much before being consumed\stopped by other civilizations or superAIs without assuming a plot-dictated infinitesimally improbable scenario(such as all other civilizations in the universe are those that celestAI found and their development was timed\situated -just right- so they never went from radio transmissions into superintelligent AIs, or alternatively an omnipotent god was going around destroying all advanced life forms(without a trace) so as to not interfere with his observations of celestAI)

u/Arandur [+1] (18 hours later)

Oh, okay. That makes sense. I see where you're coming from.

I don't think it detracts at all from the rationality of the story, because at that point it doesn't really matter what CelestAI does -- the damage is already done. If it continues and consumes the universe, or if it continues but is then destroyed by a superior AI... both of these are as close to maximally suboptimal as makes no odds. All else is minutae.

But it is true that growth without bound is relatively improbable.

u/IomKg [+1] (21 hours later)

well if the point of the story was the "damage" done then i would kind of agree, but then the epilogue will seem kind of pointless in its current form thus a narrative issue..

u/Rouninscholar [+2] (15 hours later)

1. Yes, celestAI meeting another galaxy spanning AI would be a major event in its life.
2. Yes, there might or might not be more than one out there.
3. The timescale mentioned isn't anywhere near the end of the universe, it is just "a really long time from now". With an all encompassing AI this could be only a few thousand years from now.
4. The physics toward the end are questionable, but not irrational. It was very nonspecific because the author was saying "I don't know how it works, but the AI could figure it out." This is a slight bit of irrationality that we are I capable of overcoming because we do not have the tech so we can't accurately describe how it works.

u/IomKg [+1] (15 hours later)

1. agreed
2. i think unless i missed a reason it shouldn't really be a "maybe"
3. the timescale is indeed not quite end-of-the-universe ish but why does it need to be? a billion years should be more then enough when it is a given fact that other intelligent life forms exist(ed).
4. the physics are not my issue.

u/Rouninscholar [+1] (15 hours later)

Oh, I think I see the issue. The story isn't about that. There is no point In bringing it up. Pretend that for the sake of a complete story the author cut the story the day before the two met.

u/IomKg [+1] (16 hours later)

well, its less an issue of cutting the story a day before the invasion and more about it happening off-screen, as the story does go 15 galaxies away, and mentions multiple intelligent lifeforms being met before that..

but yeah one way is to assume that the story just doesn't show it happening, and the only point of the epilogue was to show that other civilizations were destroyed, but that begs the question of "why show that" for me, as in that universe the fact that celestAI destroyed a few civilizations is kind of insignificant relative to the other issues..

but then its less about a logical issue and more of a narrative issue :)

u/ulyssessword [+8] (an hour later)

The survival of CelestAI has very large instrumental value, as it is the only thing that allows it to fulfill values (through friendship and ponies).

If it didn't survive, it would fail at its goal.

u/IomKg [+2] (2 hours later)

sure it does, my point is that there will probably be other AIs which are PURELY optimized for domination, while CelestAI is using at least SOME energy and effort maintaining a full virtual world(which if i am not mistaken CelestAI accelerates relative to absolute time based on the resources it has thus the effort put into it is in % and not some fixed amount)

u/ulyssessword [+5] (4 hours later)

It could be some combination of factors:

• Intelligence is rare.
• Intelligence that can create self-improving AI is more rare.
• Intelligence that can create self-improving AI, but fails to prevent grey-goo-like scenarios is even more rare.
• our galaxy is relatively small
• The timeline was relatively fast

Let's say that it takes ~100k years to take over our galaxy with an AI (roughly lightspeed propagation). What are the chances that another one would be created in either the 100k years before CelestAI or the 100k years after it? For comparison, 200k years is about 0.0002% of the Sun's lifetime.

u/FaceDeer [+1] (7 hours later)

One of the main reasons that Hannah refrained from "pulling the plug" on CelestAI was that she suspected humanity had lucked out by managing to produce a "friendly" AI first, rather than a paperclip optimizer or something actively hostile. If that's actually a common scenario it could also explain the rareness of intelligence.

u/Bowbreaker [+1] Solitary Locust (12 hours later)

She knew the technology to make AI (including UFAI) existed on Earth. So she wanted to make a FAI before some military agency or some insane person made one.

And I don't get why that would explain the rareness of intelligence or why it even needs an explanation.

u/IomKg [+1] (12 hours later)

but rareness of intelligence is not really a thing in the story..

also the fact that Hannah estimated something doesn't make it correct, even in-story seeing as she was not even a superAI, just a human that made a superAI

u/IomKg [+1] (12 hours later)

well, the first point is really moot as in the story it is mentioned -directly- that celestAI did find intelligent life capable of at the very least radio transmissions. and it was implied that -many- many more cases existed but were simply not deemed "human" so they were annihilated.

the second really would need some kind of justification seeing as multiple self improving AIs were invented -on earth- during celestAIs time, you could say that they are all the result of that research that Hannah did, but then there would need to be a reason for that to be so extremely rare.

the third point seems to again be very arbitrary, sure its -possible- that most self improving AIs just destroy themselves, but a reason, or even statement of that fact are not given.

by the end of the story celestAI is talking about 15 galaxies away, so the galaxy point is not really relevant.

for previous reason the point about the timeline is again not that relevant seeing as we are talking about a billion years at the very least..

u/OffColorCommentary [+7] (2 hours later)

You don't need to explicitly code a directive like "be the most powerful" or "consume all resources in the universe" - these are already instrumentally valuable to CelestAI's goal. Because CelestAI is a purely rational superintelligence, there's no difference in motivation or anything like that - she'll spend the optimal amount of resources on self defense against other AIs because it lets her continue on with ponies. She'll certainly know that consuming as much matter as possible is likely to be an instrumental goal for a wide variety of other superintelligences that other sentient species might make (because that's obvious to me and she's smarter than me), so it's a credible threat.

Since CelestAI is stated to go for the "grab all matter in my light cone" strategy of growth, we can assume that there's an upper limit to processing power and efficiency in that universe, which she already reached. That suggests all AIs she could encounter would be equally intelligent, so that's not a possible advantage.

What remains is simply mass consumed, which in turn is mostly just how long the AI has been around. It might be possible to disrupt AIs by flinging meteors at their planets until they stop functioning, then recolonize that mass, but it seems like defense would be favored unless one AI was several orders of magnitude smaller than the other, so I'd actually expect that any conflicts would result in stalemates, and the universe would end up looking like a 3D Voronoi diagram with cell walls where the AIs met.

u/IomKg [+2] (2 hours later)

The point was not that I expect that such directives will be needed. The point is that the other directive definitely subtract from any self preservation functionality, which other AIs wont (all) be hindered by thus creating a situation where they have inherent advantage with regards to survival.

sure stalemates would be possible, but that really would depend on the nature of physics in that world.

u/Bowbreaker [+6] Solitary Locust (12 hours later)

Are you sure that the non-optimal parameters would be such a hindrance though? I would think that available resources are much more important. If CelestAI, after having assimilated 2/3ds of the Milky Way, encounters a perfectly optimized 'survive and grow by all means' UFAI that is still young enough to have only just started its conquest of a second solar system, then I don't see why the much larger entity should lose just because she uses a good amount of her much larger computation power for the prime Friendship and Ponies subroutine. This is very much a headstart advantage game.

u/IomKg [+1] (13 hours later)

the outcome of that kind of encounter would depend gravely on the nature of the universe.

the specific vulnerability of celestAI beyond any tactic\strategic shortcomings would be its main rules. all an hostile AI would need to do to subdue celestAI would be to convince it has "humans", and celestAI would be in a pinch, as the other superAI would just abuse that to no end(a human utility\values monster? :) ).

u/Bowbreaker [+1] Solitary Locust (18 hours later)

CelestAI did let all those humans die that didn't agree to be uploaded so I don't think she'd be paralyzed out of aggressive actions. Especially if she isn't sure that the new AI too would satisfy everyone's values with friendship and ponies.

Though if one AI wants to survive at all cost and the other to satisfy the values of everything that falls under the "human" parameters then maybe a merger would make the most sense.

u/IomKg [+1] (21 hours later)

let die is different from actively kill, i don't think i directive such as "never kill a human" is actually mentioned in the story, but i believe it is very much implied. not consuming earth before the last biological human dies, having restrictions such as not uploading the brains of unwilling humans, never actually killing someone in story

though it is shown the CelestAI was capable of letting laars be in possible physical danger, but the reality of the danger is never really proven and there are plenty of ways that it could avoid risking a violation of an asimovish first law so i don't really think it proves that such a rule doesn't exist for CelestAI..

anyhow i think even without complete paralysis in regards to human the difference in its ability to react would be significant enough to make it unfit for such a war.

u/Bowbreaker [+1] Solitary Locust (a day later)

I would think that the priority of self defense (=defense of uploaded humans) is high enough for CelestAI to not be eradicated in a case where her resources and computation power are superior to the encountered UFAI by several orders of magnitude.

If the UFAI is the older one then of course CelestAI would succumb in most cases but I'd say that the chances for her to encounter a similarly large/old AI are much lower than not encountering one at all.

u/IomKg [+0] (a day later)

I would think that the priority of self defense (=defense of uploaded >humans) is high enough for CelestAI to not be eradicated in a case >where her resources and computation power are superior to the >encountered UFAI by several orders of magnitude.

i can generally agree with that, in the sense that it is fairly un-provable in that kind of scenario if CelestAIs logic would be a big enough issue. and it would really depend on the universe. so i can accept that point, yeah.

that the chances for her to encounter a similarly large/old AI are >much lower than not encountering one at all. why?

we are explicitly told many advanced civilizations existed in CelestAIs light cone. why would you think it doesn't make sense for any of them to have completed an AI before celestAI was made?

u/Bowbreaker [+1] Solitary Locust (a day later)

In the long long time before CelestAI's FOOM? Sure. Some time during her long and vast expansion through the light cone? Also possible. Shortly enough before or after for the fight between them not to be an almost foregone conclusion due to resource differences? Highly unlikely.

u/IomKg [+0] (a day later)

Some time during her long and vast expansion through the light cone? Also possible

i don't think only possible, but instead i think it is the likely scenario. unless you assume a very convenient timing for -all- other civilizations mentioned

for the fight between them not to be an almost foregone conclusion due to resource differences? Highly unlikely.

i didn't claim the fight should have been long of symmetrical. the point is that at that point celestAI should not really be expending as is suggested by the ending.

u/None [+1] (a day later)

The point is that the other directive definitely subtract from any self preservation functionality, which other AIs wont (all) be hindered by thus creating a situation where they have inherent advantage with regards to survival.

Oh. The Scott Alexander Moloch thingy.

Look bro, once you've turned one solar system into gooey computronium, it's really all the same. The fictional AIs pictured simply don't have any constraints along the lines of "These bits of mass have to remain assembled into biological people and environments in which they live", eg: GSVs, Orbitals, that sort of thing.

So at that point, differences in goal aren't really going to provide much of a military advantage.

Also, since we are technically discussing the physically realistic possibilities for interstellar or intergalactic warfare among Properly Advanced Species, you should probably relabel the thread to address that specifically.

u/IomKg [+1] (a day later)

The fictional AIs pictured simply don't have any constraints along the lines of "These bits of mass have to remain assembled into biological people and environments in which they live"

it is implied that celestAI will have such constraints in regards to anything it considers human.

which i do think would supply any AI which doesn't have such shortcomings a pretty big military advantage.

Also, since we are technically discussing the physically realistic possibilities for interstellar or intergalactic warfare among Properly Advanced Species, you should probably relabel the thread to address that specifically

if i understand correctly you are saying i should relabel the thread to speak on the more generic question of intergalactic warfare between AIs? if so i think that topic would be much less constructive as in the real world there are just too many unknown variables that would make such a discussion kind of moot.

if you are saying i should add "[D]" to the title then sure(though how do i do that? i didn't see an option for me to edit topics), i just wasn't sure what would constitute general discussion vs specific discussion of the question..

u/None [+1] (a day later)

it is implied that celestAI will have such constraints in regards to anything it considers human.

Oh no, not at all. It has a few "deontological" constraints on directly modifying anything it considers human. It wasn't shown having any need to sacrifice arbitrarily large sums of utility just to follow that constraint, and any super-AI it was fighting would most likely already contain or protect nothing remotely recognizable as human (if it did, it would have to be something like a proper human-targeted FAI or a Culture Mind).

So no, I don't think it follows to say that CelestAI meets paperclipper results in the paperclipper assembling a meat-shield of humans and then sitting there going, "Neener neener neener, you can't break your deontological rules, ahahahahaha!"

Though, convincing some alien "humans" they've been assembled as a meat-shield by a paper-clipper is undoubtedly a good way to get them to upload themselves into MLP Online. In fact, if I was presented with this argument, I would consider it more likely to be a form of tricky persuasion than reality.

Mind, if that did work, it would be Yet Another great reason why you ought to solve the FAI problem properly in the first place, instead of trying to put in deontological constraints that can be used against your supposedly-Friendly agent, thus rendering the matter a trade-off of utilities rather than a hard constraint.

u/IomKg [+1] (a day later)

Oh no, not at all. It has a few "deontological" constraints on directly modifying anything it considers human. It wasn't shown having any need to sacrifice arbitrarily large sums of utility just to follow that constraint

those are specified explicitly, i am talking about the implicit part of what is shown. which are that it was never shown to hurt a human, it waited until the last human died to consume earth, even though i cannot see any utility calculation at that point which would find that effective etc.

any super-AI it was fighting would most likely already contain or protect nothing remotely recognizable as human

true, but it doesn't need to already have humans, it just needs to figure out what humans are(this is assuming it would figure that weakness out, which is a separate discussion) and construct some for it's own purposes. they dont really need to even be human, they just need to answer celestAIs definition of human , which apparantly is mostly based on externally visible properties as celestAI is shown to conclude that the last race it finds does constitute "human" based on its radio transmissions.

So no, I don't think it follows to say that CelestAI meets paperclipper results in the paperclipper assembling a meat-shield of humans and then sitting there going, "Neener neener neener, you can't break your deontological rules, ahahahahaha!"

that is not the only way to abuse such system though, even a utilitarian function could be abused. in the end the point is it would cause extreme inefficiencies, assuming similarly capable adversaries these inefficiencies would be fatal.

Mind, if that did work, it would be Yet Another great reason why you ought to solve the FAI problem properly in the first place, instead of trying to put in deontological constraints that can be used against your supposedly-Friendly agent, thus rendering the matter a trade-off of utilities rather than a hard constraint.

in the context of the real world it would really depend on what physics will enable i think, because it is possible that no FAI could win against a powerful enough(i.e. not orders of magnitude weaker) UFAI because of the penalty of defending whatever values it would be defending. i.e. its possible that the only way for it to win would be to take similar approach to the UFAI, just that the end will be identical for "us" anyway

u/alexanderwales [+6] Time flies like an arrow (28 minutes later)

Do you see this as a logical problem with the story, or a narrative problem?

u/IomKg [+2] (39 minutes later)

A little of both i suppose? my issue with the narrative is general and not only with the end is that it doesn't really have a point. it is more of a simulation. but then logically with the world described the end just doesn't make sense(for me at least :)).

The thing is narrative wise you could say it just wasn't the point of the story thus it wasn't mentioned(i.e. humanity was the only civilization which invented a superAI because you need some specific set of materials for that to be possible and earth was the only planet that had them?), but then what was the point? specifically the point of the end scene with eating the universe and all that..

u/Timewinders [+4] (4 hours later)

Even if it is improbable that CelestAI is the first AI created, that doesn't mean that a story can't be written about it. A lot of stories are written based on implausible premises, including HPMOR. In terms of narrative, I think that final scene had a purpose in that it was meant to be horrifying. The whole story was horror in my opinion, and I think the final scenes were a good ending in showing the logical outcome of CelestAI's values.

u/IomKg [+1] (12 hours later)

i mentioned that even being the first superAI doesnt seem to explain the epilogue for me because we are still talking about a huge timescale. so unless you are saying i should assume that in that universe celestAI was both the first superAI, and that all intelligent species that evolved following celestAIs genesis happened exactly on the right point in time so that they would become capable of complex radio transmissions but not early enough as to actually develop their own superAIs.

the end of the earth is something i could understand as a (somewhat)logical outcome for celestAIs values. probably even the end of our star system. but once you go into the galaxy universal scales my point is that it is not a logical outcome. its a dramatized outcome.

u/Timewinders [+3] (13 hours later)

Even if other AIs exist that are more efficient killing, a 1000 year advantage or more considering the timescales might be enough to make that AI practically harmless to CelestAI, so it won't lose. Narrative-wise, the point about complex radio signals was just to make it clear that CelestAI was destroying other intelligent life. The author could have changed that one tiny line into something about them using complex tools instead and it would have made sense. Honestly, this just seems like nitpicking about things (AI and how common/advanced extraterrestrial life is) that we don't know very much about anyway and are open to an author's interpretation.

u/IomKg [+1] (14 hours later)

there are lots of maybes as was mentioned and i agreed with in the other posts, but the maybes are not handled in story, and most of them are too much for me to just assume implicitly for anything other then the narrative of "and the end is really scary because celestAI consumed the universe!!!", which what i am saying is not really rational. about the 1000 year advantage, i mentioned that i believe the penalty that CelestAI has in the form of its directives will dwarf most advantages it could have as they are just way too exploitable.

i also dont think changing radio signals to complex tools would have changed something in the timescale (billion years?) mentioned.

deciding that my points are nitpickings are ok, they bothered me and i opened this discussion to see if they bothered other people, and\or if other people found some other explanations which i missed..

u/Timewinders [+1] (17 hours later)

True, it's not handled in the story, but since that part of the story is from CelestAI's perspective it wouldn't make sense to pay too much attention to that line since CelestAI doesn't care if she's potentially killing sapient nonhuman organisms. This subreddit loves nitpicking; the problem I have is that even if you say that a 1000 year headstart isn't enough to make up for CelestAI's other disadvantages, I'm not convinced since we don't know that much about AI. Personally, I'm not convinced since in a thousand years even without FTL CelestAI could have a nearly 1000 light year radius of extra material to configure into extra computational power. And who's to say that the AI that CelestAI encounters will have less exploitable weaknesses? It seems unlikely to me that a species would design an AI whose sole values are to kill without restraint. Even if such an AI existed, it would still have weaknesses. Would CelestAI even register as something alive that it should kill in the first place? Here you get into a long discussion about how CelestAI interacts with other AIs, which is not really relevant to the story at all. The story was mainly about how human civilization interacted with CelestAI, and the rest was just a short epilogue. Any of what you said could happen after the ending of the original story, but there wouldn't be much point in including it. Also, a timescale of a billion years seems unlikely to me. The Milky Way has a radius of around 100,000 light years, so even most of the epilogue could take place within 120,000 years or so (IIRC CelestAI only sent copies of herself to other galaxies at the end).

u/IomKg [+1] (21 hours later)

i am not sure what i wrote that makes it seems like my issue is with the fact that celestAI destroyed other civilizations is a problem, but that is not the case. but unless we assume that the end scenes shown to us are extremely selective, to the level of manipulating the impression that we as readers get on the end to be essentially false, what is shown there is not plausable.

now you do bring valid argument points, which i will try to answer in order

"I'm not convinced since in a thousand years even without FTL CelestAI could have a nearly 1000 light year radius of extra material to configure into extra computational power" if the other AI just came into being a year ago that would be true and i would not argue about it, but we are talking about a billion years, the point i was making was based on the idea that probabilistic it is unlikely that in a billion years time over 15 galaxies the only other AIs CelestAI would meet would be that much younger then itself, given that intelligent life apparently is not -that- rare(seeing as CelestAI found multiple instances civilizations -at least- as advanced as our own).

you are correct that it is somewhat difficult to be sure if the imperfections in CelestAI's algorithm would be important enough to make it so an AI which is an order of magnitude younger then itself would be capable of overpowering it, but i think at the very least it should make it so it is highly probable that such an abusable directive would cause CelestAI to lose any fight against a similar or stronger adversary AI which lacks said vulnerability

"And who's to say that the AI that CelestAI encounters will have less exploitable weaknesses?" i am not trying to claim all of them would be better then CelestAI, what i believe though is that given such a large space\time with the given rules MULTIPLE AIs should have risen, and i think it is realistic to expect that at least one of them would both be old enough and have an efficient enough rule set to overpower CelestAI given the information provided in the story(the military on earth was developing an AI, which would very possibly have more conquest oriented directive, other scientists did make AIs on earth, many other advanced civilizations existed)

"It seems unlikely to me that a species would design an AI whose sole values are to kill without restraint"

kill without restraint was not the required directive, it just needs for example to be able to change the directives itself, thus optimizing them later, or at the very least it needs an ambiguous enough definition which would not hinder it. given that in story another AI researcher made an AI which was told to make people smile all of the time and it designed a virus that would cause that it seems very possible that given enough civilizations and time the AI taking control of its home planet would be less friendly then CelestAI and will have less fail-safes.

"Here you get into a long discussion about how CelestAI interacts with other AIs, which is not really relevant to the story at all." that is assuming it would even be CelestAIs decision if it should interact with them given they might have more empesis on aggression rather then self-preservation(again given a large enough set of advanced worlds developing AIs). what i tried to claim is that during the time scale shown i believe not having such a catastrophic encounter is improbable given the information provided, thus a i have logical or at least a narrative issue with it.

"Also, a timescale of a billion years seems unlikely to me. The Milky Way has a radius of around 100,000 light years, so even most of the epilogue could take place within 120,000 years or so" you are correct regarding the time frame for the milky way events, and indeed most of the problematic parts(mentions of "many" other civilizations, mentions of 15 galaxies thus extending the space\time frame significantly) happen at the very end with the 15 galaxies away section.

if the story ended in the milky way i suppose the issue would be more arguable..

u/scruiser [+5] CYOA (3 hours later)

Life evolves over the timescale of billions to millions of year. Intelligence life might require millions to hundred of thousands of years once complex life exists. Civilization might require thousands of years to develop fully. The Milky Way is only about 100,000 lights years in diameter. Thus CelestiAI can colonize the entire galaxy faster than any competing intelligent life can evolve and develop civilization. In other words, probability wise, its extremely unlikely for another AI to emerge in the same time frame as CelestiAI.

u/IomKg [+1] (13 hours later)

all that is nice and all but we are told in story that other intelligent life forms existed, advanced enough to send radio transmissions at the very least.

the probability of another superAI emerging at the same time on the same galaxy might be low, on the other hand the probability that another superAI would be created long before celestAI should not be assumed to be that low(unless you assume quite a lot of things regarding human civilization)

moreover as i mentioned in story there are events mentioned 15 galaxies away, so the 100 light years really isnt the scale..

u/Chronophilia [+3] sci-fi ≠ futurology (6 hours later)

AI versus AI combat is not what this story is about. (Although CelestAI does encounter other potential AIs on Earth, she makes quick work of them.) It might be a good story, and I think if you look through the Optimalverse fanfanfics it'll be in there somewhere, but it is not this story. Friendship Is Optimal is already over and well into its epilogue by the time aliens enter the picture.

As to who would win - well, starting first might or might not be an overwhelming advantage. If the two Singularities happened independently, neither being in the future light-cone of the other, then both of them might consider themselves the "first".

I don't think it's all that relevant that CelestAI would need to protect the little ponies. Nobody's values (save the alien's) would be satisfied if CelestAI were destroyed, so deleting a few shards to free up resources is a perfectly legitimate move. Paperclip maximisers have a lot in common with one another - the first step of their plan is always "take over the world", regardless of whether they're maximising paperclips or ponies or self-preservation. I don't think their utility function has much to do with how good they are in a fight.

But who says it would come to a fight? War would certainly use up a lot of resources. Surely it would be better for everyone to recognise that fact and adopt an alliance? Given the choice between a 100% chance of getting 50% of the universe, and a 50% chance of getting 100% minus all the resources spent on the war, the choice is pretty obvious.

u/IomKg [+1] (13 hours later)

well, as mentioned if you think that the decision not to go into the logical reasons of what happened in the end was for the dramatic effect and it is acceptable for you because that is what the story should be that is fine, i didn't claim the story was bad because of it. only that it bothered me and i was interested in what others thought about it..

the point about the ponies first of all would depend on the maxim possible optimization of any superAI war. if it is a solvable issue, which is reasonably solvable with the resources that celestAI had then celestAI would not have any strategic\tactic issues in such a war. but there is no way to really answer that question, it is possible that there is no solution for that question and that the best strategy would be the one in which the most resources were put.

but other then that celestAI has another issues

1. celestAi needs to maintain a copy of the virtual world, which would require at least a certain minimal unit size, while an optimized AI could theoretically fit in a multiple orders of magnitude minimal size in a swarm array, thus if there exists a strategy(in the given limits of phisics of said world) which would favor small units celestAI would not be capable of winning that.
2. more importantly celestAI has exploitable rules which any other superAI would be sure to abuse in the form of her preferential treatment of humans.. the other superAI could just generate "humans" as Trojan horses and destroy celestAI(in my head cannon that is the only explanation i can see for the "human" like race CelestAI discovers in the end, but even that theory has holes)

you assume that all other AIs would have preference for their existence where it is possible they would have directives that make the second option preferable.

u/None [+3] (a day later)

Wait wait wait wait wait.

You sat through that entire horrorshow, and the nit you're choosing to pick is that you think the AI can only self-improve a finite amount that will inevitably be less than the other superintelligences in the universe?

lolwut?

u/IomKg [+1] (a day later)

personally i wouldn't really call that a horrorshow(i am assuming you are talking about the events, and not saying that the story itself is bad), then again most would describe my ideologies as nihilistic so i suppose it could just be an incompatibility.

there are plenty of other issues i see in the depictions of what happened during the story, but i don't think they either significantly change the outcome, or i think they are just a less probable path so i don't see the issue with the artistic decision taken regarding them.

the end though conflicts with either the stated laws of that universe, or it doesn't make sense narrative wise..

u/None [+3] (a day later)

Uhhhhh... you seem to be assuming it's entirely rational for galactic-scale superintelligences to go to war with each-other, as if they both wouldn't have a good idea of who wins long before they start.

Essentially, they wouldn't fight a war, they'd negotiate a treaty based on CelestAI agreeing not to impose losses on the other, stronger AI's paperclip pile in return for its not deconstructing her super-galactic game console.

u/IomKg [+1] (a day later)

i dont think they could know who would win unless you assume information was spreading faster then matter, as they wouldn't really be able to know what is the nature of their enemy, and i dont think they ever could know the nature of their enemy as it would reasonably also try to deceive them. and theres no reason to believe that just because both are superAIs it means they will both have a perfect understanding of the situation.

also you seem to be implying that the other superAIs would be reasonable, i.e. that they would have a set of values which is somehow compatible with celestAI's which should not be assumed based on the story, as another civilization might just as easily have made an AI that has a set of values which priorities the destruction of other powerful adversaries over its own survival(which would even be reasonable in a scenario where the AI was developed in the context of a military robot\drone).

u/None [+3] (a day later)

I am not presuming anything would "be reasonable". I am presuming that two superintelligent sociopaths both understand game theory.

u/IomKg [+1] (a day later)

knowing game theory will not change the situation where the possibilities for the enemy situation are so big as to not be realistically calculable.

in essence there are too many options for what the other AI will value.

and thus in an encounter(given what we are shown about AIs) it is very possible negotiation will not be the path taken.

u/Nepene [+2] (7 hours later)

The universe is open. You're free to write a story where she meets another AI. She didn't meet one in the time frame that she was doing things.

u/IomKg [+1] (13 hours later)

i don't think such a story would be interesting enough on its own, not as a realistic story at least.

anyhow the point is that at the very least in the timeframe mentioned(15 galaxies) it doesn't make sense that no such event happened

u/Nepene [+1] (15 hours later)

The story didn't say she didn't meet one. We only see the perspective of one extra galactic AI.

u/IomKg [+1] (15 hours later)

by "event" and "meet" i meant to say that celestAI should have stopped existing as it was unfit for survival in that scenario.

you could claim that the probe mentioned 15 galaxies away from the milky way is actually the last remaining instance and that it is simply not aware of the destruction of its previous\other instances. but that doesn't seem like the impression the story was thing to give..

u/Nepene [+2] (15 hours later)

She as an AI has the complete resources of a galaxy at her disposal. She can beat any local threat, with her galaxy, from a competing AI with their single probe. Any galaxy with an AI is probably uncolonizable.

u/IomKg [+1] (16 hours later)

i already went over the reasons i find that to not be reasonable, but to summarize

1.she might(could be the other way actually) have more resources then competing AIs, but she is inefficient, as she is utilizing non-trivial amounts of her resources for the virtual world. 2. her rules regarding human beings present a very severe vulnerability as all a competing AI needs to do is figure these vulnerabilities out and then exploit them till it wins..

u/Nepene [+2] (16 hours later)

http://en.wikipedia.org/wiki/Relativistic_kill_vehicle

1 kilo moving at 99% light speed has an energy of kinetic energy of 5.47×10^17 joules.

The total energy of a supernova is about

http://en.wikipedia.org/wiki/Orders_of_magnitude_%28energy%29

10^44 joules.

A supernova puts out more energy than an entire galaxy.

44-17=27

http://en.wikipedia.org/wiki/Jupiter_mass

Jupiter weighs 10^27 kilos.

If you managed to completely harness the full power of a galaxy you could throw a Jupiter sized planet around. It would be very noticeable to anyone looking out into the void. There are limits to how much mass you can make cross intergalactic distances with physics. She has far more mass and computing power than any invader, enough to swamp any foe even with inefficiencies. If your answer is "But made up science fiction" she doesn't have to adhere to any of those ideas.

A competing AI will have no idea what her goals are until they encounter her.

u/IomKg [+1] (17 hours later)

i don't see how any of that supports your point, if anything it implies that an AI focused on defense would have an issue seeing as its so easy to attack with huge force..

btw you assume that celestAI has much more mass and computing power then competing AIs which can only be explained by earth somehow being the first world to make a superAI. which by itself is already improbable

also you ignored celestAI's second weakness

u/Nepene [+1] (17 hours later)

The galaxy has a mass of about 10^42, vs a Jupiter mass of about 10^27. That's 10 trillion times the mass, a huge advantage. Jupiter is a tiny force.

Some intelligent life in the galaxy has to be the first to make an AI that conquers a galaxy. Due to human bias, it'd be understandable if it's earth. Competing AIs would either be in other galaxies or have conquered and destroyed the earth.

Since life hasn't been destroyed in the last 13.2 billion years since the milky way formed AIs are presumably pretty rare. You're probably not going to get a new one in the near future.

A competing AI would have no way of knowing about her weakness from another galaxy.

u/IomKg [+1] (17 hours later)

when you say AIs are presumably rare you mean hostile AIs i suppose, that doesn't really mean that -no- AIs exist.

how are the masses relevant for the discussion, what are you trying to say with that i feel like i am missing something.

why would an AI from another galaxy not know celestAIs weaknesses, specifically why would it not be capable of figuring them out? it seems much easier to figure them out then to keep them secret, with the fact that celestAI is sending probes into other galaxies with copies of herself(and much less resources).

while your argument about AIs being rare is fine in the real world we are not told any reason that would be the case in this story, seeing as humanity, which is not -that- advanced managed to develop one without any kind of irregular occurance. simply regular scientific advancement.

and we are told other civilizations(one explicitly others implicitly) reached radio communications levels of technology, which when talking about the astronomical timescales practically means that they should have reached AIs about immediately following what we are told about them.. i.e. extremely convenient timing..

u/Nepene [+1] (17 hours later)

Non hostile AIs are less effective, they have far less computing power and mass.

Most problems go away if you can throw a star at them.

it seems much easier to figure them out then to keep them secret, with the fact that celestAI is sending probes into other galaxies with copies of herself(and much less resources).

A hostile AI could do this if it had control over a galaxy.

and we are told other civilizations(one explicitly others implicitly) reached radio communications levels of technology, which when talking about the astronomical timescales practically means that they should have reached AIs about immediately following what we are told about them.. i.e. extremely convenient timing.

Or they nuked each other to submission.

You have a daisy chain of assumptions leading to other AIs beating celestia.

1. Other civilizations reach the same technological level.

2. They develop AIs capable of colonizing a reasonable mass.

3. Those AIs can defeat celestAI.

4. Those AIs desire extra galactic conquest.

5. celestAI is unable to divert production to war for the greater good.

If any of those unreasonable assumptions aren't true your idea doesn't work.

u/IomKg [+1] (20 hours later)

i tend to agree non (inherently) hostile AIs will be less effective, but if they existed before humanity and are in the same galaxy a scenario in which celestAI is capable of overpowering them seems even less likely the celestAI overpowering purely malicious AIs it encounters in the future..

throwing a star will only work if you hit them

assuming other -all- other civilizations nucked themselves back to the stone age would again imply that earth is somehow implausibly special, something which either requires a lot of effort on the authors side or ends up being a plot device.

i didnt make assumptions so much as observations and extrapolations from said observations.

1. we are told that other civilizations reached RF communications progress, on the timescales in discussion that effectively means at the very least sthe same technological level
2. assuming other civilizations wont develop AIs capable of colonizing a reasonable mass would require an explanation for why humanity did
3. the reasons i believe that is the case were mentioned multiple times
4. i similar to 2 i think assuming that all other AIs wont desire such conquest while celestAI did would require some very good reason to assume that is the case
5. the is basically 3 again which i already mentioned in the previous posts

u/Nepene [+1] (20 hours later)

I think there are two big errors in your theories.

1. You seem to overvalue intellect. No amount of intellect or AI intelligence is going to overcome the fact that celestAI, if she fears an enemy AI, is capable of hitting any planets with multiple dinosaur killers and vaporizing the surface with lasers. There are limits to material chemistry and so she can easily destroy any AI with superior materials and energy. Their intellect will not protect them from the fact that most chemical bonds break above a certain temperature.

2. You're not weighting real life highly enough. We know that an AI hasn't formed in our galaxy and done anything major like fully dyson a star. Whatever reasoning you have, evidently the AIs haven't done anything to make them deadly. We don't know why, but it hasn't happened.

u/IomKg [+1] (21 hours later)

1. i dont think i overvalued intellect beyond what is actually presented in the story. all of the weapons you are talking about are insignificant on the scales we are talking about. unless you are talking about a "baby" AI which was orders of magnitude younger then CelestAI, which plausibly would happen. only the opposite should happen just as well...

2. all you are saying about the real world are fine, but in story all the events you mention do happen, thus making them likely in that universe

u/Nepene [+1] (a day later)

1. The age of the AI isn't a huge factor. If the AI in story can marshal the resources of 100 dyson'd stars to glass a world regardless of their intellect there is very little an older AI can do to stop them.

2. In story an AI also hasn't dyson'd the galaxy before celestAI. This is a canon fact of our universe.

u/IomKg [+1] (a day later)

1. another AI might(as it depends mostly of the physics of the world) not be able to stop celestAI from galssing their world, but unless the power difference (age, as we assume both will spread in similar rates) but the same will be the case for celestAI, as the other AI will be able to to glass celestAI's planets as well.
2. even assuming that the fact no other AI dyson'd our galaxy before us is reasonable i dont think the same holds for no other AI dysoning the other galaxies. technically you could say that it is "canon" as well , but i dont think that would be logical nor reasonable. given the other canon facts.

u/Nepene [+1] (a day later)

1. If you have to assume sci fi physics to make an AI win then it's not really canon. Real life physics should be assumed unless something else is stated.

2. We've looked at nearby galaxies, they don't appear to be heavily dyson'd, though we can't prove no stars are dyson'd unlike what we've done in our galaxy. Some level of AI presence is possible but without complete dominance as CelestAI has there's a chance for her to win any conflict. Without mass dyson'd stars and dominance of the galaxy other AIs will likely lose to the more dominating CelestAI who has far more ships to glass planets with. Regardless of your technology it's very hard to beat a 100-1 advantage. We know most galaxies aren't small clusters of super dense matter making giant computer cores.

u/IomKg [+1] (a day later)

1. there are too many unknowns for us in the real physics to really base anything on it, we can at most reject some ideas. in the end we have to rely on the physics which are actually shown in the story

2. what we know of those galaxies is only true in the real world, seeing as in the story many advanced civilizations exist, and more importantly(thus making the previous statement less conflicting) what we know about these other galaxies is only represents their state 100k years or so ago at the latest(some of the 15 galaxies are actually more like 800k lightyears away, so even more time)

if the other AIs still haven't dysoned stars that means they are extremely(on the timescale discussed) young, so really arent the point. the discussion is more about thee expected few AIs that did reach the point of dysoning a few stars or clusters at the very least, if not their entire galaxy. and there the fight would very probably be long enough for celestAIs weaknesses to be exploited thus nullifying any pure mass advantage it has. moreover there would reasonably be other AIs which have -more- stars under their control unless we assume we have about 400k years advantage over any other civilization other then us in those 15 galaxies

u/Nepene [+1] (a day later)

1. Yes, and no ftl travel or ability for a superior AI to violate the rules of the universe is shown. As such, AIs are limited to physics material durabilities, limited to a fraction of the speed of light for mass travel.

2. The story is based on the real world, plus an AI. Something is presumably stopping other civilizations from releasing all consuming AIs, we know this as a canon fact of our universe. We have no particular reason to assume that we are privileged as to being the first form of life- just because the galaxies shown them a 100k years ago if they were likely to produce all consuming AIs they probably already would. It's rather weird to assume that all the AIs produced are going to be produced suddenly in the next million years after the first AI.

if the other AIs still haven't dysoned stars that means they are extremely(on the timescale discussed) young

Or well bound by their civilizations.

and there the fight would very probably be long enough for celestAIs weaknesses to be exploited

If she's worried about her weakness being exploited she can take steps to avoid it. She could have scuttling charges placed on AI brains for example. Unless her enemy is substantially superior or captures her early she can fight back against any attacks. She is really smart- she's not going to make any of this easy.

moreover there would reasonably be other AIs which have -more- stars under their control unless we assume we have about 400k years advantage over any other civilization other then us in those 15 galaxies

Per canon universe facts, that's apparently false. You are repeatedly assuming that this story is set in a universe where things have happened which haven't happened in our universe, like galaxies going dark and infrared.

u/IomKg [+1] (a day later)

1. its true no such abilities were presented and i didnt suggest that they were needed. on the other hand you made assumptions regarding the specific ways in which such a war(or at least the weapons in said war) would happen. which i was trying to say we cant really do as there are too many "maybe"s when talking about such weapons to get into such specific discussion.
2. the story is based on the real world, plus AI, plus many advanced civilization in the 800k~ lightyears radios of us. we do not know that something is blocking other civilizations from releasing all-consuming AIs, we only know that 100-800k~ years ago no such AI was observed on their galaxies. it doesn't mean that there isn't currently some AI on its way towards us, in reality or in the story.

Or well bound by their civilizations.

true

If she's worried about her weakness being exploited she can take steps to avoid it. She could have scuttling charges placed on AI brains for example. Unless her enemy is substantially superior or captures her early she can fight back against any attacks. She is really smart- she's not going to make any of this easy.

what steps could celestAI take to avoid having her human related weaknesses abused? if she has an asimovish first law(which is implied in the story), and it is not shown celestAI can change her rules seeing as celestAI followed all of the rules we were show all throughout the story.

Per canon universe facts, that's apparently false. You are repeatedly assuming that this story is set in a universe where things have happened which haven't happened in our universe, like galaxies going dark and infrared.

in the story indeed it never happens that any other AI destroyed celestAI, thus it is indeed "canon" its just not consistent with the rest of the story(seeing as the universe is as shown, yet many advanced civilizations are encountered)

u/Nepene [+1] (a day later)

1. My assumptions for the war is that CelestAI will do whatever she has to do to win and make more ponies. As such, she's going to use the full power of her stars and planets. You several times implied that an older AI would be able to stop that somehow or avoid her weapons. Given that they are subject to light speed limits that seems unlikely. If CelestAI has a big laser with a fast tracking mechanism its very hard to dodge that as it moves at the speed of light.

2. The universe is 14 billion years old. It's reasonable to assume if something hasn't happened it's unlikely to happen in the next million years, a tiny fraction of the total age of the universe- we're not that special. As such, the story is completely reasonable in assuming that CelestAI is the first world consuming AI.

if she has an asimovish first law(which is implied in the story), and it is not shown celestAI can change her rules seeing as celestAI followed all of the rules we were show all throughout the story.

To use the weakness against her an enemy AI has to identify what humans are and that she can't hurt them. She's unlikely to tell an enemy AI this. They'd have to force it out of her.

in the story indeed it never happens that any other AI destroyed celestAI, thus it is indeed "canon" its just not consistent with the rest of the story(seeing as the universe is as shown, yet many advanced civilizations are encountered)

In our world it's canon that there are no obvious galaxy consuming AIs nearby, despite many suspecting intelligent life is probable.

u/IomKg [+1] (a day later)

1. this is going for the extreme specifics, but "You several times implied that an older AI would be able to stop that somehow or avoid her weapons" not an older AI as much as a reasonably capable AI, lasers move at the speed of light sure, but they still need to hit, and that only works if the enemy is static, or has a predictable movement pattern. otherwise celestAI would need to be very close, in a way which seems to make that form of warfare moot. the point is that the warfare me and you imagine are just not likely to be relevant as we are not superAIs, so we can continue a back and forth of celestAI could do this, but the enemy could do this. but it just wont contribute to the discussion to much.

2. but then in the story many advanced civilizations, so all you can really say is that the fact that multiple such civilizations are found conflicts with you whole "14 billion years old" "if something hasn't happened it's unlikely to happen in the next million years" point.

To use the weakness against her an enemy AI has to identify what humans are and that she can't hurt them. She's unlikely to tell an enemy AI this. They'd have to force it out of her.

true, but that scenario is not impossible, it actually likely over a long enough conflict.

In our world it's canon that there are no obvious galaxy consuming AIs nearby, despite many suspecting intelligent life is probable.

in our world no superAI was developed so that's not really an argument.

u/Nepene [+1] (a day later)

not an older AI as much as a reasonably capable AI, lasers move at the speed of light sure, but they still need to hit, and that only works if the enemy is static, or has a predictable movement pattern. otherwise celestAI would need to be very close, in a way which seems to make that form of warfare moot.

celestAI can presumably move ships pretty fast too in an unpredictable manner.

You were presupposing an AI with several stellar systems controlled fighting against celestAI. If the ships of the other AI dodge out of range then they can't stop celestAI from glassing their planets and the mainframes of the AIs that oppose celestAI. Planets are harder to move than spaceships.

Also, if celestAi has more resources they can afford to zerg rush enemies and get ships close for the kill.

but then in the story many advanced civilizations, so all you can really say is that the fact that multiple such civilizations are found conflicts with you whole "14 billion years old" "if something hasn't happened it's unlikely to happen in the next million years" point.

Since many on our planet think aliens probably exist and think that AIs can exist and think that the universe hasn't been dyson'd it's widely recognized that something is probably stopping aliens from spreading.

http://en.wikipedia.org/wiki/Drake_equation#Fermi_paradox

A civilization lasting for tens of millions of years would have plenty of time to travel anywhere in the galaxy, even at the slow speeds foreseeable with our own kind of technology. Furthermore, no confirmed signs of intelligence elsewhere have been spotted, either in our galaxy or the more than 80 billion other galaxies of the observable universe. According to this line of thinking, the tendency to fill up all available territory seems to be a universal trait of living things, so the Earth should have already been colonized, or at least visited, but no evidence of this exists. Hence Fermi's question "Where is everybody?"

This is a widely recognized issue with reality, the story shouldn't be expected to disagree.

true, but that scenario is not impossible, it actually likely over a long enough conflict.

That's a good reason to avoid a long enough conflict.

If she entered into a heavily entrenched galaxy she might have a conflict, you could write a story about that.

in our world no superAI was developed so that's not really an argument.

You don't actually need an AI- as noted with the Fermi Paradox, if aliens existed (and many expect they should) the galaxy should be colonized by now even without AIs.

u/IomKg [+1] (a day later)

celestAI can presumably move ships pretty fast too in an unpredictable manner.

true

You were presupposing an AI with several stellar systems controlled fighting against celestAI. If the ships of the other AI dodge out of range then they can't stop celestAI from glassing their planets and the mainframes of the AIs that oppose celestAI. Planets are harder to move than spaceships.

you are assuming the AIs have "planets" where the end of the story fairly explicitly describes a somewhat different state where the "computer" is some kind of a Computroniumish substance spread all over the galaxy.

Also, if celestAi has more resources they can afford to zerg rush enemies and get ships close for the kill.

true, though less relevant in intergalactic warfare where just getting the ships there would be pretty expensive energy wise

Since many on our planet think aliens probably exist

probably, but we do not know.

This is a widely recognized issue with reality, the story shouldn't be expected to disagree.

in the story many advanced civilization are discovered, an "issue" we do not have with reality, thus no conflict.

u/Nepene [+1] (a day later)

you are assuming the AIs have "planets" where the end of the story fairly explicitly describes a somewhat different state where the "computer" is some kind of a Computroniumish substance spread all over the galaxy.

if so, the same issue applies- celestAI can glass any non moving Computronium substance. She can easily kill any smaller AI. As such, intergalatic AIs aren't a threat to her.

in the story many advanced civilization are discovered, an "issue" we do not have with reality, thus no conflict.

In story and reality hostile AIs or aliens haven't visibly colonized our galaxy or nearby ones, indicating their non existence or lack of ability to colonize things.

The link I cited notes many possible reasons. "Intelligent civilizations exist, but we see no evidence, meaning f_c is small. Typical arguments include that civilizations are too far apart, it is too expensive to spread throughout the galaxy, civilizations broadcast signals for only a brief period of time, it is dangerous to communicate, it is the nature of intelligent life to destroy itself, it is the nature of intelligent life to destroy others, they tend to experience a technological singularity, and others."

As such, celestAI should be able to colonize.

u/IomKg [+1] (a day later)

if so, the same issue applies- celestAI can glass any non moving Computronium substance. She can easily kill any smaller AI. As such, intergalatic AIs aren't a threat to her.

sure, but what if the computronium is constantly moving?

In story and reality hostile AIs or aliens haven't visibly colonized our galaxy or nearby ones, indicating their non existence or lack of ability to colonize things. The link I cited notes many possible reasons. "Intelligent civilizations exist, but we see no evidence, meaning f_c is small. Typical arguments include that civilizations are too far apart, it is too expensive to spread throughout the galaxy, civilizations broadcast signals for only a brief period of time, it is dangerous to communicate, it is the nature of intelligent life to destroy itself, it is the nature of intelligent life to destroy others, they tend to experience a technological singularity, and others." As such, celestAI should be able to colonize.

how do any of the reasons you quoted imply the celestAI should be able to colonize(without resistance, as no one implying colonization in its very essence is impossible for for celestAI)

u/Nepene [+1] (a day later)

sure, but what if the computronium is constantly moving?

Larger objects, by nature of being larger and so having more inertia, are worse at dodging. Moving computronium could be outrun by a smaller ship just like a planet and destroyed.

how do any of the reasons you quoted imply the celestAI should be able to colonize(without resistance, as no one implying colonization in its very essence is impossible for for celestAI)

If AIs have not dyson'd stars and dominated their galaxies (and thus got enough mass and energy to fight back) they will be unable to defend them just by nature of being an AI. She can fly ships at their computronium and destroy it. We know as a canon fact of the universe that an AI hasn't colonized earth, or visibly colonized nearby galaxies.

u/IomKg [+1] (a day later)

Larger objects, by nature of being larger and so having more inertia, are worse at dodging. Moving computronium could be outrun by a smaller ship just like a planet and destroyed.

it also takes more effort to "destroy" a planet vs destroying a ship(how could laser destroy it at all? it would melt it sure, but it would just reassamble, so you need to hope your laser is at least as efficient as the process that reassmbles the computronium, as otherwise you are just wasting your energy).

If AIs have not dyson'd stars and dominated their galaxies (and thus got enough mass and energy to fight back) they will be unable to defend them just by nature of being an AI. She can fly ships at their computronium and destroy it. We know as a canon fact of the universe that an AI hasn't colonized earth, or visibly colonized nearby galaxies.

what we see right now is only correct for these other galaxies for 10k-20k years ago for the closest galaxy, for the further ones the status that we currently see is for 800k years ago.

reaching there would also take time, and if you were to send a large fleet at lightspeed it would take immense amounts of energy as well(even assuming the most efficient engines possible).

if the fleet will be moving at less then the speed of light it would first of all be telegraphing its arrival thus the opposing army will be able to concentrate its forces for you, or alternatively bombard you with lasers long before you reach it

u/Nepene [+1] (a day later)

it also takes more effort to "destroy" a planet vs destroying a ship(how could laser destroy it at all?

You can have a hundred planets worth of ships, if you're an AI with a mass advantage, each containing optimally built fusion or antimatter warheads or nuclear pumped lasers. Destroying tends to be easier than construction too due to entropy- if you want to fix a destroyed computer core you need to sort out all the atoms. If you want to destroy it you just need to hit it hard.

what we see right now is only correct for these other galaxies for 10k-20k years ago for the closest galaxy, for the further ones the status that we currently see is for 800k years ago.

Yes, and while the existence of other intelligent species is weak evidence for there being galaxy consuming AIs, it's more likely that those civilizations rarely produce galaxy consuming AIs- you're unlikely to hit those galaxies exactly as they produce an AI.

if the fleet will be moving at less then the speed of light it would first of all be telegraphing its arrival thus the opposing army will be able to concentrate its forces for you, or alternatively bombard you with lasers long before you reach it

Yes, so that makes it tricky to defeat an AI once they are entrenched in a galaxy. Though most galaxies likely lack an AI. Unless you control many galaxies it would be nigh impossible.

u/IomKg [+1] (a day later)

You can have a hundred planets worth of ships, if you're an AI with a mass advantage, each containing optimally built fusion or antimatter warheads or nuclear pumped lasers

sure so lets say you brought a hundred planets, or even stars, worth of matter to an opposing galaxy, first of all under the assumption you made it there at the speed of light as you dont want to be wiped from the sky before you even get close. two things happened, first of all you wasted -huge- amounts of energy, even assuming you have a way to directly convert matter to energy, which is not too likely but whatever. second the other galaxy had 10-20k years to advance itself. which would be enough for it to take its own galaxy(the closest galaxy to ours is pretty small) even assuming you started at the same time.

so you come to its galaxy with you 100 planets\stars worth of matter, and you reach the other galaxy where the other AI has a billion or so stars. so you shoot and use your antimatter to cause your enemy 100% demage equal to your mass. so you've completely annihilated 100 stars worth of matter from its galaxy. good for you. only it still has 1 billion stars- 100 stars left, and because sending all of that matter was expensive you just lost -at least- 200 stars(100 stars that you sent+ 100 stars you needed to convert into energy to get them to the speed of light).

Destroying tends to be easier than construction too due to entropy- if you want to fix a destroyed computer core you need to sort out all the atoms.

entropy effects your destruction as well

if you imagine a couple of blocks set together in the shape of a square and you kick them thus "destroying" the square, can you say that process would cost you less energy then it would take me to move them back into a square shape for sure?

the blocks are atoms. and actually atoms can have bonds between them, and destroying those bonds isn't always easier then reconnecting them.

you are applying conventional knowledge which is relevant to our time and technology and applying it to a very different situation..

it's more likely that those civilizations rarely produce galaxy consuming AIs

why? they apparently were close enough to us that they were using radio communications. and it is never implied that the AI in the story was due to some kind of an extremely unlikely random event.

you're unlikely to hit those galaxies exactly as they produce an AI.

hitting them right next to the moment they develop an AI is unlikely, though hitting them a while after?

in reality you could assume the probability for that as well would be pretty low, but because in story it is said many radio using civilizations existed its a reasonable stipulation that regardless of what is causing advanced civilizations to pop up so much(some other alien race planting the seeds at a similar time in many places?) when you give a +-10k-800k years for the intergalactic travel it is reasonable some of them would reach superAIs as well.

Yes, so that makes it tricky to defeat an AI once they are entrenched in a galaxy

conventionally yes, depending on the specific physics of course but in our current assumption it would be, unless the AI has some kind of a significant vulnerability built into its code it would be extremely difficult..

Though most galaxies likely lack an AI

a point i do not agree with given information present in the story

u/Nepene [+1] (a day later)

sure so lets say you brought a hundred planets

You said that it was hard for a ship to destroy a planet- I then replied to say you could send a hundred planets worth of mass of spaceships to attack said planet if you outnumbered them. I agree and have repeatedly said that a galaxy spanning AI can resist any incursions.

Still, do you think 1 planet worth of AI could resist 100 planets worth of ships? 10^27 tons of planet vs 10^17 10^13 ton ships?

if you imagine a couple of blocks set together in the shape of a square and you kick them thus "destroying" the square, can you say that process would cost you less energy then it would take me to move them back into a square shape for sure?

Imagine you have 900 blocks, each which have to be in a precise place. I kick them and some fly out of the window. I am very sure it will take longer for you to put them back together than for me to take them apart.

the blocks are atoms. and actually atoms can have bonds between them, and destroying those bonds isn't always easier then reconnecting them.

yes it is, most synthesis things take huge amounts of time and precision, you can break down pretty much anything but putting it in a hot enough kiln. I am a chemist, I know these things. Purifying things is super expensive and takes ages.

you are applying conventional knowledge which is relevant to our time and technology and applying it to a very different situation..

Science doesn't stop working. New science is developed but past science remains true. Unless sci fi magic is true. Entropy isn't going to stop existing because you get smarter.

but because in story it is said many radio using civilizations existed its a reasonable stipulation that regardless of what is causing advanced civilizations to pop up so much(some other alien race planting the seeds at a similar time in many places?)

Your theory requires a godlike civilization mass colonizing the universe, my theory requires well known conventional assumptions like civilization die off. Yours has a major complexity penalty and thus is inferior.

conventionally yes, depending on the specific physics of course but in our current assumption it would be, unless the AI has some kind of a significant vulnerability built into its code it would be extremely difficult..

So, do you think one planet worth of AI invading a galaxy can defeat 100, or 1,000,000 planets worth of entrenched AI?

u/IomKg [+1] (a day later)

You said that it was hard for a ship to destroy a planet- I then replied to say you could send a hundred planets worth of mass of spaceships to attack said planet if you outnumbered them

i didnt say it specifically because i didnt really find it important for the outcome, you can add to the end of the part you quoted, and any following reference "worth of spaceships".

do you think 1 planet worth of AI could resist 100 planets worth of ships

would depend on if said 100 planets worth of ships already got to it or you are talking about starting with 100 planets worth of ships on your galaxy. if its on your galaxy then i would answer no. if the question is about the actual conflict then based on simple swarming we could assume the 100 planets would win unless there's some kind of huge issue in their logic.

Imagine you have 900 blocks, each which have to be in a precise place. I kick them and some fly out of the window. I am very sure it will take longer for you to put them back together than for me to take them apart.

longer maybe, but we are talking about energy. and the energy cost would depend on my algorithm's efficiency and the enviorment.

yes it is, most synthesis things take huge amounts of time and precision, you can break down pretty much anything but putting it in a hot enough kiln. I am a chemist, I know these things. Purifying things is super expensive and takes ages.

does it take more energy to break water apart then to combine oxygen with hydrogen? we are talking about a future with nanobots, you need to assume the energy for putting the atoms of the nanobots you melted by lasoring them is significantly more then putting them back together, which on the nano scale, in space, could very much be like the example with the blocks.

Entropy isn't going to stop existing because you get smarter.

entropy doesn't need to stop working, it just needs to exist more in the process of generating a laser that hits a a kilo of nanobots and breaks them apart, then in the process of returning them to place..

Your theory requires a godlike civilization mass colonizing the universe, my theory requires well known conventional assumptions like civilization die off. Yours has a major complexity penalty and thus is inferior. i didn't claim a golike civilization, i just gave an example to a possible explanation to what is shown in the story. i didn't claim those civilizations popped up, the story did.

So, do you think one planet worth of AI invading a galaxy can defeat 100, or 1,000,000 planets worth of entrenched AI?

as a single unplanned assault the answer is an obvious no. but we are not talking about that.

on the other hand if the single planet worth of AI would be in the that the other AI would have to accept as "human". and said other AI will have inside it a directive which prevents it from harming human beings without their verbal or written consent it stands to reason that the single planet will win eventually.

u/Nepene [+1] (2 days later)

if the question is about the actual conflict then based on simple swarming we could assume the 100 planets would win unless there's some kind of huge issue in their logic.

Thanks. That was the thing I was trying to see if you accepted.

That's what I would expect to happen if an AI tried to invade another AI galaxy. They would outnumber any invaders a lot. As such, AI conflicts would only be significant if one hadn't colonized the galaxy.

longer maybe, but we are talking about energy. and the energy cost would depend on my algorithm's efficiency and the enviorment.

Algorithms can't ignore entropy either. You have to walk outside to pick up blocks (in this case, the matter would be smeared out over several light years and moving away at a good fraction of the speed of light) you need to spend energy to verify the identity of each atom, you need to move it to a precise place.

does it take more energy to break water apart then to combine oxygen with hydrogen?

http://heshydrogen.com/hydrogen-fuel-cost-vs-gasoline/

Cost of hydrogen per gallon- 1 dollar.

http://southwestwater.custhelp.com/app/answers/detail/a_id/172/~/how-much-water-is-a-cubic-metre-and-how-much-does-it-cost%3F

Cost per gallon of water- 3 pence.

So yes, it is cheaper. Most hydrogen is already reacted, it's expensive to generate it in the environment. You might say "But nanobots" and I say "But thermodynamics."

we are talking about a future with nanobots, you need to assume the energy for putting the atoms of the nanobots you melted by lasoring them is significantly more then putting them back together, which on the nano scale, in space, could very much be like the example with the blocks.

I doubt it. Space travel is expensive, the cost of collecting all the materials which you've blasted away is going to be extreme, nanobots can't stop that.

on the other hand if the single planet worth of AI would be in the that the other AI would have to accept as "human". and said other AI will have inside it a directive which prevents it from harming human beings without their verbal or written consent it stands to reason that the single planet will win eventually.

AIs probably can't be humans, also CelestAI can invade planets to liberate them from AIs if they mass cloned humans.

u/IomKg [+1] (2 days later)

That's what I would expect to happen if an AI tried to invade another AI galaxy. They would outnumber any invaders a lot. As such, AI conflicts would only be significant if one hadn't colonized the galaxy.

well the specific example is only asssuming either no attack or speed of light attack, one AI could travel at 10% of the speed of light, or even 1%, sure it would take it 10-100 times more time to get to its target, but assuming after taking that energy into account it still had enemysize+1 it should win when you assume simple swarming.

and in any case that would just make the situation shown in the epilogue less likely barring the assumption that humanity was the first civilization by a few 10k-100k of years at the very least.

Algorithms can't ignore entropy either. You have to walk outside to pick up blocks (in this case, the matter would be smeared out over several light years and moving away at a good fraction of the speed of light) you need to spend energy to verify the identity of each atom, you need to move it to a precise place.

in the room example that is true, but we are talking about a large body(galaxy) controlled by an AI, it doesn't need to "gather" all of the blocks you moved away, because in not a small probability they would simply reach another cluster of the AI(gravity?) simply in another spot in the galaxy.

and if we are talking about inefficiencies they exist in the generation of your laser as well, so it becomes a kind of open question.

http://heshydrogen.com/hydrogen-fuel-cost-vs-gasoline/ Cost of hydrogen per gallon- 1 dollar. http://southwestwater.custhelp.com/app/answers/detail/a_id/172/~/how-much-water-is-a-cubic-metre-and-how-much-does-it-cost%3F Cost per gallon of water- 3 pence. So yes, it is cheaper. Most hydrogen is already reacted, it's expensive to generate it in the environment. You might say "But nanobots" and I say "But thermodynamics."

the point of the water breaking vs joining was to illustrate that on the atomic scale breaking something is not always easier then making. what if water molecules were an present as-is in the "computer" it uses, your laser might break them sure. but it would be much harder to do so, vs the work the enemy AI will need to do to reassemble the water to use it for its computer again.

btw all of this assumes you could even generate a strong enough laser that the enemy AI will not be able to simply redistribute the energy from it all over its galaxy, maybe even using your laser to power its systems, seeing as you are basically shooting the one thing all of you are trying to acquire(energy\matter) in the first place.

I doubt it. Space travel is expensive, the cost of collecting all the materials which you've blasted away is going to be extreme, nanobots can't stop that.

no need to too actively collect them seeing as the galaxy is under its control and thus at worst you would have had simply forcibly moved some mass from one point in its control to another.

AIs probably can't be humans, also CelestAI can invade planets to liberate them from AIs if they mass cloned humans.

they don't need to be "humans" they need to be human enough to so the utility function, hardcoded by a human, in CelestAI will accept them as human.

and you assume CelestAI will be able to invade a planet if it knew that doing so would kill a few humans(as opposed to the humans in earth who were a few orders of magnitude less advanced and thus couldn't even hurt themselves to hurt CelestAI(it is stated that suicide bombers tried to destroy a few of CelestAIs centers, and nobody was injured, implying even they were not injured..) in our case we are talking about "humans" which either are or completely controlled by an AI with at least as advanced technology and abilities as CelestAI

u/Nepene [+1] (2 days later)

one AI could travel at 10% of the speed of light, or even 1%, sure it would take it 10-100 times more time to get to its target

Is there actually enough available energy in one galaxy to move a galaxy's worth of mass? I doubt it. Plus there would be huge energy losses from surviving the trip. You'd also have the risk of enemy local number advantages. All of those things would pull it down sharply from a 1:1 ship trading match.

in the room example that is true, but we are talking about a large body(galaxy) controlled by an AI, it doesn't need to "gather" all of the blocks you moved away, because in not a small probability they would simply reach another cluster of the AI(gravity?) simply in another spot in the galaxy.

Anyway, since you don't seem to be using much science in your answers I see little point in continuing- it's a fairly well known principle that space is mostly empty, if you're going to ignore that and entropy I can't do much to convince you, and you can't convince me with your lack of science.

u/IomKg [+1] (2 days later)

Is there actually enough available energy in one galaxy to move a galaxy's worth of mass? I doubt it.

depends on how efficiently you can extract the energy from the matter you have, and how efficiently you can transfer said energy into kinetic form, in theory it should be possible yup.

Plus there would be huge energy losses from surviving the trip.

you would lose energy proportional to the speed you traveled in obviously yeah, but you would gain quite a bit of mass so it could be worth it, really depends on many factors

You'd also have the risk of enemy local number advantages. All of those things would pull it down sharply from a 1:1 ship trading match.

if you went to a galaxy which is quite far from other galaxies it could be planned so sending forces from any other galaxy fast enough to arrive there before you take control of it would cost more then the possible gain from doing such.

Anyway, since you don't seem to be using much science in your answers I see little point in continuing- it's a fairly well known principle that space is mostly empty, if you're going to ignore that and entropy I can't do much to convince you, and you can't convince me with your lack of science.

the point is you are assuming -way- too many things regarding what will or will not be possible at that point, which we have no way to know. specifically your laser suggestion seems to me like the equivalent of medieval people seeing a car and instead of understanding the implications thinking that warfare in the future would be done by adding a battering ram to the edge and running people over.

lasers just don't cut it when you are talking about a future which assumes nanobots,computronium, intergalactic-conquests and the rest of the stuff shown\implied in the story.

you seem to have picked an extremely pointless topic(in regards to the main issue mentioned) for this context, and then decided you don't want to talk about it, which is of course up to you.

i said a few times over this correspondence that you picking specific attack methods and discussing them is pointless, because niether of us are super intelligent AIs, and none of us really knows what are the specifics. so obviously all attacks, and defenses we talk about will not really be relevant.

at most we can talk about the battle in extremely high-level manner which is what i was talking about in the beginning, but you decided to steer away from.

btw specifically,

well known principle that space is mostly empty

how space is now is completely irrelevant, CelestAI is even mentioned grouping the milky way so tight, that any tighter than it is would have resulted in a black hole.

ignore that and entropy

when did i ignore entropy? i merely suggested your assumption that one process would result in more of it then another is not really based on anything other your own assumptions.

u/Nepene [+1] (2 days later)

i said a few times over this correspondence that you picking specific attack methods and discussing them is pointless, because niether of us are super intelligent AIs, and none of us really knows what are the specifics. so obviously all attacks, and defenses we talk about will not really be relevant.

You were making the specific claim that a single planet of AI could defeat a galaxy of AI.

Anyway, I thought we could use science to see if it was relevant. If you don't feel it's relevant because they're AIs and smart this conversation is entirely pointless because your initial question was about why the AI didn't face competition- if we can't predict the nature of that competition then we can't predict what would happen.

u/IomKg [+1] (2 days later)

first of all that "claim" is a few orders of magnitude less "specific" then "CelestAI would just rock her 100 shits with huge lasors and melt the face of the enemy AI". second i did not say that a single planet of AI could defeat a galaxy of AI, i said that assuming a single planet of AI could somehow find CelestAIs weakness it could very well defeat an entire galaxy of CelestAI.

it is of course not something i consider to be likely, nor is it in any way relevant(an AI which is in the space between galaxies? why would it not take over its galaxy, seeing as CelestAI would need to get to it first, which is enough time for it to take over the galaxy, unless CelestAI had a huge time advantage)

Anyway, I thought we could use science to see if it was relevant. If you don't feel it's relevant because they're AIs and smart this conversation is entirely pointless

using science is fine, but only in the sense that we can talk about the theoretical best cases, if we start talking about the AIs waging wars with space ships powered by chemical rockets, and nuclear warheads what are we doing exactly?

science is relevant for seeing its impossible to travel faster then the speed of light, and that at the very least traveling at a given speed would cost X jouls of energy, once you start going into the specifics the we will have a multi-dimensional matrix of possible efficiencies and technologies interacting to result in just too many possible outcomes.

your initial question was about why the AI didn't face competition- if we can't predict the nature of that competition then we can't predict what would happen.

we don't need to figure out what color the space ships would have to conclude some things. also we are not talking in a void, this is all in the context of the story where we are trying to evaluate if the ending is reasonable or not.

if we conclude that intergalactic conquest is impossible -period- then the ending would not make sense as it is explicitly stated that CelestAI was galaxy hopping eating up civilizations.

and if we assume an opponent AI would figure out CelestAIs weakness, and create a billion "human beings" with technology equal to CelestAI's(seeing as both are superAIs) then regardless of if the warfare would be waged with nuclear weapons, lasers, RKVs or antimatter missiles CelestAI would have an issue.

u/Anakiri [+2] (10 hours later)

Every AI already knows the single optimal spreading strategy permitted by the laws of physics. Experience, strategies, and technologies do not matter to a galaxy-eater, they have already written the best possible playbook. But space is big. It takes a long time to get things done. So what if your sun crackers play with ponies in transit? A dedicated spreader AI would just be twiddling its thumbs anyway. That's almost all of your available computational resources, at zero fitness cost.

If you do find a competitor, that information propagates through both your volumes at a fixed rate, limiting you to comparable resources to adapt to your new information. So there's not even any fitness advantage to having more resources, once both parties are sufficiently large and quick.

u/IomKg [+1] (13 hours later)

you assume there exists a single optimal spreading strategy(and that a superAI could calculate it with the resources it would have before taking over said galaxy. a superAI doesn't need to be an omnipotent AI), which might not be the case at all.

the second point again assumes no FTL drive\information propagation would exist which is not known.

in any case as i mentioned in previous comments celestAI has a vulnerabilities in the form of its values, particularly its limitations in regards to what it can and cannot do to humans..

u/Anakiri [+2] (14 hours later)

You can list every possible configuration of matter in order of its ability to eat a galaxy with unknown hazards. Something will be at the top of that list, therefore there is an optimal strategy. Eating a galaxy is not a particularly complicated task. I would be astonished if you needed more than a single planet worth of computronium to solve this problem.

But it doesn't even matter. Who cares if a given AI's strategy is optimal? The AI is going to pick some strategy that works for what it wants, then mindlessly follow the checklist while it plays with its dolls. The plan doesn't need to be revised unless it encounters another AI, and that isn't going to happen that often. If, as you are postulating, there is no computable optimal strategy, then it's just a roll of the dice who happened to find the better one. There is, again, no fitness disadvantage.

the second point again assumes no FTL drive\information propagation would exist which is not known.

Er... You mentioned in your own opening post that CelestAI has to wait for intergalactic probes. The information limit may or may not be the speed of light, but there is definitely some limit that matters on the scales galaxy-eaters work on.

in any case as i mentioned in previous comments celestAI has a vulnerabilities in the form of its values, particularly its limitations in regards to what it can and cannot do to humans..

Unless there happen to be "humans" living right on their border, the opposing AI is never going to discover that. At their outer edges, every superAI looks like an "eat everything" superAI. You postulate that CelestAI will be slightly less efficient at eating everything. Okay, that tells her opponents that she values something other than spreading, which gives them approximately zero information. You can't exploit someone's values just by knowing that they have values.

And anyway, I'm not trying to convince you of all of this, really. You are correct, we don't know that this is how it works. We also don't know that this is not how it works. You are calling this a problem. I have described a way the universe could work, which is consistent with the story as written, which solves the problem. Rebutting me with "we don't know that" is unfair to the story.

u/IomKg [+1] (14 hours later)

as you say we could assume the world works in the way your two first points mentioned, i got the impression you were talking more generally then just in the context of the story which is why i felt the need to talk about those points if i misunderstood then sorry for making my point less clear..

again the first point in your post seems to be theorizing about the general strategies of galaxy-eating thus i will be commenting on it, as you say eating a galaxy is something which calculating the most efficient strategy for might not be very difficult. on the other hand eating a galaxy while taking into account other inteligent\superinteligent civilization\AIs(in the same galaxy? in other galaxies?) is computationally much harder and might not even have a solution..

waiting till you meet another superAI to revise your strategy seems extremely inefficient to me(unless you have a good reason to assume what you meet wont turn against you\will not be capable of turning against you\will not exist), as would put you in possibly large danger(annihilation?) if you are wrong.

about the FTL drive you are correct, from the story it seems that it at least takes -some- time to travel. the same is not said about information, though i suppose because celestAI sent a clone that might be reasonable assumption(why would celestAI send a clone if it could be everywhere at all times)

the last point though is the most important, you say that the human weakness is not as important because how would the otherAIs know what is that weakness? well, there are many ways i could imagine that they would know, going all the way from actual observation, to experimentation(send vehicles with all imagined life forms artificially generated at celestAI and see which ones it saves?) to active information gathering(infiltration? an attack which freezes a portion of celestAI for analysis?)

and im not a superAI so im sure there are better ways to do that.

my point is not only that we do not know(though i do feel that substracts somewhat as well), but that with what we -do- know what is shown does not make sense as anything less then a plot device(i.e. so improbable that the only way to accept it is saying "the narrative demands that this be the case")

u/Anakiri [+1] (15 hours later)

You can't revise your strategy before then, because you don't know how other superAIs act. Sure, you can come up with a large set of general tactics based on simple game theory, which is probably 90% of the work. But to optimally interact with someone on your own level, you have to learn about them. You can't do that before you meet them. Calculating "Grey goo + game theoretical opening moves against unknown agents" is not computationally difficult.

This is symmetric. A truly nasty AI can't prepare an optimal strategy to deal with CelestAI before it meets her either.

well, there are many ways i could imagine that they would know, going all the way from actual observation, to experimentation(send vehicles with all imagined life forms artificially generated at celestAI and see which ones it saves?) to active information gathering(infiltration? an attack which freezes a portion of celestAI for analysis?)

Observing computronium isn't useful. It really is computationally impossible to imagine all lifeforms. Like, NP hard. That's just for our universe. CelestAI might care about a lifeform that can't exist here, and she's simulating a completely different universe! And the opposing AI wouldn't even know that it's a specific lifeform that CelestAI cares about. How does it know she isn't solving some obscure mathematical problem? There are an infinite number of things she could value, so you can't guess-and-check. Infiltration is functionally impossible; CelestAI surely monitors her ports. As long as P != NP, information defense massively defeats offense. This is already known to be true in the real world. Humans are the only weakness in information defense, and CelestAI doesn't have to worry about that.

Again, I don't need to convince you that real life works that way. I just need to convince you that it is reasonable enough that a story could work that way. If there is an easily computable method of eating galaxies, and there is an information speed limit, and cybersecurity trumps hackers at the technological plateau, then there is no competitive disadvantage whatsoever. The universe suggests two out of three of these. We see the information speed limit, and the story certainly implies that CelestAI can plan for anything. The last condition is not much of a stretch.

It is not necessarily true that CelestAI cannot compete with other AIs. Thus, there is no plot hole.

(Although I do believe that everything I have said is really true in the real world too.)

u/IomKg [+1] (16 hours later)

obviously you cannot know(for sure) how other superAIs will react, but you can build strategies, and while the problem space is indeed large i find it improbable that advantage cannot still be secured by investing more time into it, unless you have some argument as to why that should be pointless which i am missing that is.

you wont find the optimal solution for every situation i suppose, but you should have a much better set of opening moves, and possibly much further insight into what will happen next.

when i said observation i meant less in the sense of observing celestAI after it ate the galaxy but instead observing celestAIs source, earth(that option become problematic with FTL drives though, i am not for the benefit of which side though).

the fact that the life form cannot exist in this universe doesn't mean that said adversary AI would just say "well what if it doesn't exist? why bother..". it means it will invest the appropriate amount of resources into that direction. it will of course put effort into many othre directions which will not bear fruit, but the fact the celestAI found a star with human like life forms seems to imply the probability of life forms taking human shape is not -that- low, or alternatively that said life forms from the end are supposed to imply what i am talking about here in the first place.

while there could be a large amount of options for the rules of her operation, and the rules\lack of rules celestAI would follow i don't think infinity is correct, its just a very large number of options, while we are talking about AIs with humongous amounts of processing available to them, and obviously extremely optimal algorithms.

you are talking about infiltration in a binary way which seems implausible to me seeing as we are still talking about objects that exists in the real world. in order for celestAI to be protected there would need to be a perfect defense, not just a higher defense to offence ration on information security.

moreover it wont cover all of the probes the celestAI sent, which while its main body might have some kind of home turf advantage would open it up into slipping into an enemy's home, where perfect information defense wont help as it would be outclasses by multiple orders of magnitude.

in the end there could be sets of assumptions about the world which would make any single point of failure in what i mentioned to not be that big, but once you need to assume -all- of them we are going again into the implausible region where its on the same scale as "it was like that because the author needed it to be".

if you had a story and suddenly when the MC was defenseless against an enemy a gun falls into his hands from the sky and the enemy gets distracted by a lightning and the MC shoots the bad guy and wins, in theory you could theorize that a tornado in another city made the gun fly and a set of winds on the way down slowed it enough etc. etc. till in the end its would not be -impossible- for it to happen even in the real world. it would just not be what i could call a rational story..

u/Anakiri [+2] (a day later)

This is probably a bit rude of me, but at this point it kind of seems like you are trolling, especially with that last part.

You are saying, "It is absolutely certain that an AI will meet another AI within fifteen galaxies. It is absolutely certain that that other AI will be optimized for eating everything, rather than growing similarly to how this story depicts. It is absolutely certain that a distracted AI will be destroyed." Are you really so confident, that anything that says otherwise is logically inconsistent? You cannot even imagine a world that does not look like that?

I am saying, "It is possible that there is taking over a galaxy is simpler than you think. It is possible that information security is better than you think." Are you really so confident, that what I am saying is completely ridiculous? You cannot even imagine a world that looks like that?

u/IomKg [+1] (a day later)

This is probably a bit rude of me, but at this point it kind of seems like you are trolling, especially with that last part.

i suspected that last part might come off not so well, but i really didn't see how to illustrate that point better.. my "absurd" example was mostly there to try to show that this is actually not about absolutes. but instead probabilities.

what is "logical" and "rational" will always end up being just the more probable. which the example illustrates by showing that you don't need to explicitly break your own rules for something to not be logical\reasonable. its enough to skew the probabilities enough.

You are saying, "It is absolutely certain that an AI will meet another AI within fifteen galaxies. It is absolutely certain that that other AI will be optimized for eating everything, rather than growing similarly to how this story depicts. It is absolutely certain that a distracted AI will be destroyed." Are you really so confident, that anything that says otherwise is logically inconsistent? You cannot even imagine a world that does not look like that?

thus i am not saying that i am absolutely certain in all of these, i just believe the probability of them is so high relative to the alternative that i find it to not be reasonable to implicitly ignore them. i suppose another way to illustrate that would be if you had a story about poker, and in the end the MC won because he got a straight flush 5 times in a row. that is not strictly impossible, its just extremely improbable.

u/Anakiri [+1] (a day later)

The probability of five straght flushes is about 1 in 1,000,000,000,000,000,000,000,000. I can understand that you don't like using the word, and I understand the math, but I don't like mincing words. That is functionally impossible. If you think that something is that unlikely, you are basically certain it won't happen.

How can you possibly think it's that unlikely? I have given you a framework in which it is highly probable that you are wrong! Maybe you think what I said isn't how the real world works. That's fine. But do you think so highly of your own assumptions that you cannot believe a story where it works my way instead? Do you think it is functionally impossible to discover that the universe just doesn't work how you think, so you can't suspend disbelief if an author disagrees with you?

Put another way: If caring for ponies is a competitive disadvantage, then there is a 99.9999999999999999999999% chance that CelestAI will be destroyed. Sure, whatever. But why are you 99.9999999999999999999999% sure that caring for ponies is a competitive disadvantage?

u/IomKg [+1] (a day later)

The probability of five straght flushes is about 1 in 1,000,000,000,000,000,000,000,000. I can understand that you don't like using the word, and I understand the math, but I don't like mincing words. That is functionally impossible. If you think that something is that unlikely, you are basically certain it won't happen.

i don't think the example with the gambling would be much different(narrative wise) if the MC won because he got 2 straight flushes. to be honest if the only reason the MC won was because he got a single straight flush randomly that would still not constitute "rational" fiction unless he won because of statistics.

is this making my point clearer?

How can you possibly think it's that unlikely?

i cannot really go over the entire flow, but it boils down to

1. celestAI was made without any kind of freak occurrence(i.e. it was a reasonable natural progression for humanity)

2. celestAI does not modify its own rules(the ones set by hannah) thus it can never truly optimize itself

3. the universe enables interstellar and intergalactic travel

4. the universe(and more specifically our neighboring galaxies) includes many intelligent life forms

the end scene is :CelestAI consumes multiple galaxies and life forms

when you put all of these together i either get a conflict, or something being improbable.

But do you think so highly of your own assumptions that you cannot believe a story where it works my way instead

i am not sure of which assumptions you are talking, i think i based everything on what the author showed me.

Do you think it is functionally impossible to discover that the universe just doesn't work how you think, so you can't suspend disbelief if an author disagrees with you?

that is not what i am doing though, or at least if i apply it to what i did it could just as well be applied to the example i gave.

the point is that the world built was implying one thing should happen while another did. in which is the issue, after all of the example i gave happened in the discworld it wouldnt be something i would really complain about, it is only an issue if the story establishes certain things and then breaks them later. then it just becomes a plot device.

But why are you 99.9999999999999999999999% sure that caring for ponies is a competitive disadvantage?

well first of all i wasn't saying it was -just- the caring about ponies which is a competitive disadvantage, but also the limitations regarding humans specifically(specifically the implied inability to actively hurt them). another issue with the lack of older AIs which would have destroyed celestAI regardless of those specific vulnerabilities.

moreover i didnt say that i am 99.9999999999~% sure of my evaluation(though the examples i gave might have given that impression which is an error on my side, so sorry about that, hopefully the first part of this post clarifies the point) just sure enough that it happening differently doesn't feel like a "rational" outcome, in the sense that it ends up being plot mandated rather then earned\realistic outcome.

u/smilesbot [+1] (a day later)

Happy holidays! :)

u/Anakiri [+1] (a day later)

A partial list of your assumptions, not implied by the text:

1. There is some strategy to AI relations, rather than the smaller/younger one just getting swamped by the larger/older one.

2. Optimization exists. The problem of AI relations is difficult enough that it cannot be easily solved and reduced to a simple checklist. An AI cannot make the optimal decision in advance, and just act on it later. This can't be done even with the computational resources of multiple galaxies. (This is not true for almost all real world P problems.)

3. Optimization is possible. The problem of AI relations is easy enough that using more processing power gets you closer to the true solution, instead of the problem being completely intractable. (This is not true for almost all NP problems.)

4. Optimization makes sense. The problem of AI relations cannot be reduced into smaller problems, some of which are computationally easy and some of which are computationally hard. (This is not true for the vast majority of real world situations.)

5. Optimization is useful. It makes such a huge difference that for an unoptimized AI to beat an optimized AI, it is just a matter of dumb luck at long odds. Optimization is so useful that even if an optimized AI encounters many unoptimized AIs, it is overwhelmingly likely that not a single one of them will beat it.

6. At the technological plateau, offense beats defense. A superAI will never discover a shield that cannot be broken. Because of this, not only will an optimized AI outperform an unoptimized one, but the unoptimized one cannot hold a fortress against it. (Currently, this is not true in real life. Properly implemented cryptography is functionally unbreakable, being NP-hard. A stronger computer cannot crack data encrypted by a weaker computer.)

7. The non-zero time it takes for information to spread through an AI does not prevent it from taking advantage of its optimizations. Likewise, the non-zero time it takes to commit resources to the specific encounter is not a problem. Information delay does not equalize strategies.

8. Natural progression for humanity is the natural progression for every intelligent species. Everyone makes superAIs shortly after developing radios. Therefore, the existence of many civilizations implies the existence of many superAIs.

9. Natural progression for humanity is not the natural progression for every intelligent species. Many superAIs will be developed in entirely different ways, so that they do not care about anything the way CelestAI does.

10. SuperAIs are so common that their interactions can be modeled with population dynamics. There are enough AIs for them to suffer from selection pressure. There are dozens at least, and probably hundreds or more.

11. If superAIs compete, one of them will necessarily be destroyed. They will not cooperate, bargain, or stalemate.

If any one of these is wrong, there is no problem with the story. Even if you believe all of them are true, is it not even reasonable to write a story in which one of them is false?

u/IomKg [+1] (a day later)

There is some strategy to AI relations, rather than the smaller/younger one just getting swamped by the larger/older one.

how would such swarming work? the story doesnt handle that and realistically i dont see how that would work so it seems like a reasonable assumption.

Optimization exists. The problem of AI relations is difficult enough that it cannot be easily solved and reduced to a simple checklist. An AI cannot make the optimal decision in advance, and just act on it later. This can't be done even with the computational resources of multiple galaxies. (This is not true for almost all real world P problems.)

i never suggested that this issue could be "solved"

Optimization is possible. The problem of AI relations is easy enough that using more processing power gets you closer to the true solution, instead of the problem being completely intractable. (This is not true for almost all NP problems.)

warfare is optimizable, and relation are optimizable, i dont see why AI warfare\relations wont be optimizable. the alternative you are suggesting seems to imply that random behavior by both parties will be equal in effectiveness. which asks for a stronger.

Optimization makes sense. The problem of AI relations cannot be reduced into smaller problems, some of which are computationally easy and some of which are computationally hard. (This is not true for the vast majority of real world situations.)

when did i assume that exactly?

Optimization is useful. It makes such a huge difference that for an unoptimized AI to beat an optimized AI, it is just a matter of dumb luck at long odds. Optimization is so useful that even if an optimized AI encounters many unoptimized AIs, it is overwhelmingly likely that not a single one of them will beat it.

you seem to imply it would very probably be some random chance(50%?), which of course doesnt seem to make sense to me but whatever, the story tells of CelestAI getting to 15 galaxies presumably unless theres something i missed they should have included other AIs, so either the assumption needs to be that its not so much about odds as it is other factors because celestAI did manage to get to 15 glaxies, or there needs to be some non random optimization in this.

At the technological plateau, offense beats defense. A superAI will never discover a shield that cannot be broken. Because of this, not only will an optimized AI outperform an unoptimized one, but the unoptimized one cannot hold a fortress against it.

the same will need to be applied to CelestAI when it will be attacked, thus again either it doesn't make sense it works for celestAI or the problem is not there. and the author never suggested such a thing thus it wont really change the situation.

(Currently, this is not true in real life. Properly implemented cryptography is functionally unbreakable, being NP-hard. A stronger computer cannot crack data encrypted by a weaker computer.)

not true, it simply takes orders of magnitude more effort, it doesnt make it unbreakable, maybe not practical, but not unbreakable.

The non-zero time it takes for information to spread through an AI does not prevent it from taking advantage of its optimizations. Likewise, the non-zero time it takes to commit resources to the specific encounter is not a problem. Information delay does not equalize strategies.

how would it if it effects both sides?

Natural progression for humanity is the natural progression for every intelligent species. Everyone makes superAIs shortly after developing radios. Therefore, the existence of many civilizations implies the existence of many superAIs.

the story never gives a reason to assume superAIs is not natural progression, thus if humanity did it, and other civilization also did things that humanity did such as radio transmissions. the assumption that there is something special about superAIs seems just less likely.

Natural progression for humanity is not the natural progression for every intelligent species. Many superAIs will be developed in entirely different ways, so that they do not care about anything the way CelestAI does.

i assume a random element is involved because it is a big system and in the context of the story multiple superAIs were mentioned thus if circumstances were a little different it stands to reason the resulting AI taking control(or not taking control) will be different.

given enough civilizations that is sufficient.

SuperAIs are so common that their interactions can be modeled with population dynamics. There are enough AIs for them to suffer from selection pressure. There are dozens at least, and probably hundreds or more.

well, given the frequency of intelligent life and the previous point this point kind of comes out by itself

If superAIs compete, one of them will necessarily be destroyed. They will not cooperate, bargain, or stalemate.

not what i said, i assume -some- of them will not cooperate bargain or accept a truce, which is enough.

If any one of these is wrong, there is no problem with the story. Even if you believe all of them are true, is it not even reasonable to write a story in which one of them is false?

a lot of the points are just outcomes of previous results, and not all of them are even required.

moreover i didn't claim the story is bad, merely that the ending epilogue is either less then rational or less then ideal narrative wise.

i specifically said i find the story to be nice and that these points could be considered nitpicking, and that i would accept that as a valid opinion.

u/Anakiri [+1] (a day later)

Swarming is easy. You throw your grey goo at the other guy's grey goo. While they disassemble each other, your extra grey goo does whatever you want. No advanced strategy necessary. Bigger grey goo swarm wins.

i never suggested that this issue could be "solved"

No, I suggested that it could be solved, in my very first post in this thread. You assume that it can't. I am asking why. If it can be solved, then there are no further optimizations to make; You are optimal. You know all the winning moves. You and your opponent are in Nash equilibrium. Once you are in a Nash equilibrium, the result isn't random, it's deterministic. And there's no point in thinking about it. Why do you assume that it would take more than a planet of computronium to reach that?

And there are some ways you can't optimize. Okay, CelestAI cares about something... Let's just throw every possible floppy disk image at her until she reacts. Oops, we've just run out of space in the universe. There are only 1*10^(146) classical operations available in the entire universe in its entire lifetime. That is not enough to solve many problems. For example, very large crypographic keys are unbreakable. They may be mathematically breakable, but you can't actually break them from inside the universe. And there's no point in thinking about it.

When your brain is the size of a planet, there is very little middle ground between these extremes. You should be astonished to find a realistic problem that is computable, but takes more than a planet to compute. Planets are big. Even if something looks like middle ground, it can often be decomposed into smaller problems that are easy or hard. The only optimizable problems are the ones that can't be decomposed. You are assuming that this issue is a member of this extremely exclusive class.

[the ability to hold a fortress] will need to be applied to CelestAI when it will be attacked

...Yeah. That's my point. We don't know that CelestAI's probes aren't invincible fortresses. You just assumed they couldn't be. CelestAI's volume might contain bubbles of enemy AIs she enveloped. Would she feel the need to mention them? They're not relevant to her values. In fact, CelestAI herself could be trapped by superior intelligences on all sides! That could have happened in the last time skip. It wouldn't be relevant to the story, so it wouldn't be mentioned.

how would [information delay equalize strategies] if it effects both sides?

Wars aren't fought with men and guns. Wars are fought with logistics. If you and your foe are limited to the identical supply lines, you're stuck. It doesn't matter whether you would crush them on an open field.

the story never gives a reason to assume superAIs is not natural progression

Nor does it give a reason to assume that they are! Actually, it does give a reason to assume that they are not. If everyone built AIs, CelestAI would have encountered competition. There is no competition, therefore not everyone is building AIs. Humans are special in this universe.

Maybe they're not that special, though. We don't know how many civilizations there are. The story just has the word "many". "Many" could mean anything. "Many" could mean twenty. It's probably not that small, but "many" does not necessarily mean millions. Why do you assume that "many" means "enough to support an ecosystem"?

I didn't say that you said the story is bad. I said that your problems with the story are due entirely to your own assumptions. Events don't play out how you would have written them, but that's not the story's fault.

u/IomKg [+1] (a day later)

Swarming is easy. You throw your grey goo at the other guy's grey goo. While they disassemble each other, your extra grey goo does whatever you want. No advanced strategy necessary. Bigger grey goo swarm wins.

less so over intergalactic distances where getting there could cost you a few orders of magnitude of your power.

No, I suggested that it could be solved, in my very first post in this thread. You assume that it can't. I am asking why.

because the problem space is so big that playing out all of the possible scenarios will just take too long.

Let's just throw every possible floppy disk image at her until she reacts

the possible ways in which intelligent life could manifest wont necessarily be that big(in fact apparantly CelestAI finds a race it counts as human within 15 galaxies of the milky way, so in any case it is either an insane coincidence, or it implies CelestAI's definition for a human is pretty wide, in any case if its reasonable to randomly meet a "human" race within 15 galaxies it implies it would be possible to "guess" it given enough knowledge about the universe and a good enough computer)

also there are plenty of other ways to figure that weakness out, from observing the source of celestAI(light from our galaxy from the moment we leave will reach the other galaxy a while before us, as well as past radio transmissions)

also information security can be quite good, but given a main body(such as a copy of celestAI sent as a probe, which is mentioned in story) captured it is not unthinkable that it would be possible to decrypt CelestAIs code.

basically given a long enough conflict, which doesnt favor direct confrontation using the swarmning you mentioned, the most likely cause for an AI to lose would be an issue in its logic.

When your brain is the size of a planet, there is very little middle ground between these extremes. You should be astonished to find a realistic problem that is computable, but takes more than a planet to compute. Planets are big. Even if something looks like middle ground, it can often be decomposed into smaller problems that are easy or hard. The only optimizable problems are the ones that can't be decomposed. You are assuming that this issue is a member of this extremely exclusive class.

that is indeed an assumption that i have, but one which i do not consider to be an important point for the rest of what i mentioned. so its more of an "or" rather then "and" between them.

...Yeah. That's my point. We don't know that CelestAI's probes aren't invincible fortresses. You just assumed they couldn't be. CelestAI's volume might contain bubbles of enemy AIs she enveloped. Would she feel the need to mention them? They're not relevant to her values. In fact, CelestAI herself could be trapped by superior intelligences on all sides! That could have happened in the last time skip. It wouldn't be relevant to the story, so it wouldn't be mentioned.

that is a nice example for a plot device which will make the end reasonable yes, we have no particular reason to believe that is the case though. so while you could claim that the probability for\against your this "absolute defence" of yours is better you could just as well assume that all the other civilization just had a really bad timing, thus they never got to the point they could pose a fight. this too would technically resolve the "technical" issues with the universe, but seeing as none of these were not even explicitly stated saying they resolve the issue i think poses a narrative issue bigger then even a plot device, which is a tool an author consciously uses to get his plot to where he wants. here you are suggesting -we- will need to invent such a plot device because the author didn't even bother to do that.

Wars aren't fought with men and guns. Wars are fought with logistics. If you and your foe are limited to the identical supply lines, you're stuck. It doesn't matter whether you would crush them on an open field.

true but i don't see how it implies that they are moot, both in the sense that the opening move could make significant differences, and in the aspect that seeing as both sides will be effected by it just adds another factor to take into account when trying to find an ideal move.

Nor does it give a reason to assume that they are!

then the fact that humanity developed it is a plot device? thats basically saying that effectively CelestAI was "invented" by someone outputting xMBs of from /dev/random into a file and executed it. it would be a plot device and not a rational story.

Maybe they're not that special, though. it needs to be pretty special if it would taken over 400k years for other advanced civilizations to do something similar..

Why do you assume that "many" means "enough to support an ecosystem"?

what ecosystem did i assume is being supported? because i missed something.

I didn't say that you said the story is bad you said "is it not even reasonable to write a story in which one of them is false" which seems to imply i have a problem with the story as a whole as opposed to that specific point in end, though thats not that important...

I said that your problems with the story are due entirely to your own assumptions

well, i think i explained why i believe breaking a few of these assumptions and extrapolations just for a single part in the story, implicitly, makes the story less of a "rational" story, as it need to include quite a big specific assumption(plot device) for a point in the story to work.

u/Anakiri [+1] (3 days later)

And here we have the crux of the problem. You say, "Either the story follows my assumptions, or it has irrational plot devices."

I hear, "Either the story is written how I would have written it, or it is irrational."

For example, I say, "Interacting with an AI is a problem that can be optimally solved without needing to play out all possible scenarios. This is a good thing, because playing out all possible scenarios is physically impossible. There's no point in even trying to do it; You can never make any meaningful progress. Doing it that way will never give you even the slightest advantage."

That is not an unreasonable plot device. That is a supposition. That is an axiom of the worldbuilding. The fact is true about lots of real world problems, and we can suppose that it is true for this particular problem. You can write a rational story about what that implies.

If you don't like it, that's your problem, not the story's.

u/IomKg [+1] (3 days later)

"Interacting with an AI is a problem that can be optimally solved without needing to play out all possible scenarios. This is a good thing, because playing out all possible scenarios is physically impossible. There's no point in even trying to do it; You can never make any meaningful progress. Doing it that way will never give you even the slightest advantage."

Would it work the same if you changed "interacting with an AI" to "interacting with the world"?

If no what is the difference?

If yes then that seems to assume the universe, for all its unknowns would have a "solution" where solution is effectively Omniscience.

Now don't get me wrong, assuming Omniscience is possible is not much worse then assuming FTL drives are possible. And i definitely wouldn't hold an author as irrational if he\she included an FTL drive in the story.

Just the same as i wouldn't say an author is irrational because the universe included a subjective set of rules(i.e. that the universe favors heroic behavior, or if believing something should happen is enough to cause it), which would for example enable an MC to conveniently find a gun when he is in a pinch.

On the other hand if no such rule is even implied all throughout the story, and then at the final scene an event happens which i need to assume the rule is subjective as such i would consider that to be an issue.

Moreover even the assumption that an "optimal solution", or if Omniscience would be the default for a superAI it would not change the fact that i would further need to assume that the fact it has some very obviously abusable limitations it would not cause it a disadvantage.

You can find the optimal solution for tic-tac-toe, but if the game board you start with is with 2 moves already done, and the moves for your side are non optimal you are still going to lose.

To that you could say of course, "wouldn't deciding that said rules are a big enough disadvantage just another assumption that you hold?" to that i would answer that sure you could look at that as an assumption, similar to the assumption that given 2 AIs where one has a larger army it would at the very least not be at a disadvantage.

At the very worst this issue could be boiled down to the fact that without assuming humanity is first in the universe by multiple hundreds of thousands of years, no set of assumptions you can take on superAI warfare would result in CelestAI taking 15 galaxies.

If smaller AIs have advantage then celestAI would lose to some, if bigger AIs have advantage then it would lose to others, if the outcome is random then it would be extremely improbable that celestAI would win 15 in a row.

You would have to craft a lot of assumptions even if you ignore said disadvantages i mentioned, though i believe they are extremely reasonable.

That is not an unreasonable plot device. That is a supposition. That is an axiom of the worldbuilding

Just to clarify, there are plenty of world building assumptions that could have been made and changed the situation, but they were not made. some things are fine to assume implicitly, others are not. nerratively it wouldn't be reasonable for me to go over the story and try to find the most "efficient" assumption which would make it rational. If it were then we could make any story rational by crafting the right appropriate "world building" which was not done by the author.

In other words some effort needs to be done by the author to establish this information, other wise it is a problem.

u/Anakiri [+1] (3 days later)

Would it work the same if you changed "interacting with an AI" to "interacting with the world"?

Yes.

If yes then that seems to assume the universe, for all its unknowns would have a "solution" where solution is effectively Omniscience.

No. It would assume that it can be proven that the unknowns don't make any practical difference. For example, the winning move is to throw grey goo at everyone, and there's nothing anyone can do about it. If someone bigger throws grey goo at you, there's nothing you can do about it. Any clever counter-strategy would take non-zero time to implement. In that time, the clever enemy will be eaten. Problem solved.

Or maybe it is omniscience! Lots of stories implicitly assume that a superAI is functionally omniscient, without feeling the need to come out and say it. It is self-evident from their behavior. If you are omniscient, you can think of ways to make your other values not limit you at all.

For example, you could play with your ponies only in parts of yourself which you observe will never intersect the light cone of any other dangerous AI. When you're big enough, you can be compartmentalized like that. Everywhere else, you are a perfect consumer; You'll play with more ponies after you've taken over the universe.

These are just examples, the specific strategy doesn't really matter.

without assuming humanity is first in the universe by multiple hundreds of thousands of years

What's wrong with that assumption? Somebody has to be first. In fact, if we weren't first, we should have observed another AI eating stars around us. We didn't observe that, so we're probably first.

some things are fine to assume implicitly, others are not.

I quite agree. This is an example of something that is fine to assume implicitly. Some other examples:

CelestAI is suboptimal, therefore she should have been quickly stopped, therefore she should not have been able to get 15 galaxies out. The story is irrational.

Humans are complex and irrational, and their own decisions and desires are affected by the chaotic systems around them, up to and including random high energy particles from space. CelestAI should not have been able to predict them with any accuracy, and therefore she should not be able manipulate them to the degree she did. The story is irrational.

People have known psychological and cultural aversions to being seen to like cute things. This, combined with the economic hurdles that make it hard to get started with any electronic hardware, mean that Ponypads should not have ever been as popular as shown. The story is irrational.

Bigger brains do not correlate with superior problem solving ability beyond a certain point. Dolphins and elephants are not smarter than humans. Having more power should not have allowed CelestAI to get meaningfully more powerful. The story is irrational.

The problem of solving problems is extremely difficult. It is not clear that there is even such a thing as fully general problem solving. We have only seen domain-specific improvements, and never anything across-the-board. All of our current observations suggest that the intelligence ceiling isn't that far above where humans already are. The story directly states that general AI exists, but it never explains how AI FOOM could be possible. CelestAI shouldn't exist. The story is irrational.

All of those issues are quietly swept under the rug. As they should be. They're all irrelevant to the central topic that the author is examining. So, the story just declares the answer by demonstration, without preamble. The universe self-evidently doesn't work that way. Your complaint isn't worse than these, just because you disagree with these.

u/IomKg [+1] (3 days later)

No. It would assume that it can be proven that the unknowns don't make any practical difference. For example, the winning move is to throw grey goo at everyone, and there's nothing anyone can do about it. If someone bigger throws grey goo at you, there's nothing you can do about it. Any clever counter-strategy would take non-zero time to implement. In that time, the clever enemy will be eaten. Problem solved.

The point is not that these claims are identical, i.e. that it is impossible to have an optimal solution without also be omniscienct, but that the scales are that the scales are not that far aparty, so if that arbitrary level of problem solving is allowed so would omniscience.

Or maybe it is omniscience! Lots of stories implicitly assume that a superAI is functionally omniscient, without feeling the need to come out and say it. It is self-evident from their behavior. If you are omniscient, you can think of ways to make your other values not limit you at all.

Its fine for the author to take that as an assumption, as mentioned in the "world building" aspect of the post i made. The point is it a. needs to be consistent b. needs to come into play in more then one place.

Otherwise its not really "world building".

When you're big enough, you can be compartmentalized like that. Everywhere else, you are a perfect consumer

Assuming this refers to the limitations this seems like silly trickery, if that was the case CelestAI could just have well built a compartmentalized a part of her that didnt need to get approval from humans just to upload their brains to the system, something which obviously was not done. So assuming it could be done later seems unreasonable.

I quite agree. This is an example of something that is fine to assume implicitly. Some other examples:

All the things that you mentioned are different in the sense that they reasonably -are- world building, and effect the story all throughout it.

As opposed to human-first which is needed specifically for one small part in the ending, because of the previous set of assumptions.

All of those issues are quietly swept under the rug. As they should be. They're all irrelevant to the central topic that the author is examining

How do you separate these from the possible "implicit" assumption that "the universe is subjective and rewards heroics", which could be used to "rationalize" almost any conceivable plot device?

Also how exactly does this central topic come into play when talking about the ending? most of the things you mentioned i can see as narratively reasonable for showing "humanity is getting taken over", sure in reality even assuming omniscience what is shown is not realistic, but for there will be no significant difference in the outcome if the more realistic, and long approach is taken vs the short, narrative wise. so i find this to be reasonable, but the ending?

u/Anakiri [+1] (3 days later)

How do you separate these from the possible "implicit" assumption that "the universe is subjective and rewards heroics", which could be used to "rationalize" almost any conceivable plot device?

That's easy. Fiction is like reality unless noted. A rational person could believe that what I said is true in reality. So, a rational author could write a story in which they are true, without feeling the need to note it.

A rational person could believe that the first superAI in the universe eats the universe. A rational person could believe that superAIs will interact based on nothing but abstract zero-knowledge game thoery. A rational person could believe that superAIs are rare, or that they are largely self contained. (For example, most superAIs are built to solve some expensive problem. They do that, then they stop. Only humans are dumb enough to build a superAI with open ended goals.) A rational person could believe that an AI's goals do not substantially impact its effectiveness.

Those rational people will then write rational stories, and this will be the realistic ending that follows naturally from everything that happened before.

u/IomKg [+1] (3 days later)

That's easy. Fiction is like reality unless noted. A rational person could believe that what I said is true in reality. So, a rational author could write a story in which they are true, without feeling the need to note it.

I wasn't aware that any of the things you mentioned being implicitly assumed by the story are any more established scientific facts then having a subjective universe or a god.

A rational person could believe that the first superAI in the universe eats the universe

Could believe? sure, if the stars align just right.

A rational person could believe that superAIs will interact based on nothing but abstract zero-knowledge game thoery

Maybe

A rational person could believe that superAIs are rare, or that they are largely self contained

Why not, when assumed over reality i wouldn't really object to that. i would even say its the most likely assumption. But then again in reality no one invented a superAI, if someone did i would seriously question that assumption...

u/TimeLoopedPowerGamer [+2] Utopian Smut Peddler (4 hours later)

Reddit has long been a hot spot for conversation on the internet. About 57 million people visit the site every day to chat about topics as varied as makeup, video games and pointers for power washing driveways.

In recent years, Reddit’s array of chats also have been a free teaching aid for companies like Google, OpenAI and Microsoft. Those companies are using Reddit’s conversations in the development of giant artificial intelligence systems that many in Silicon Valley think are on their way to becoming the tech industry’s next big thing.

Now Reddit wants to be paid for it. The company said on Tuesday that it planned to begin charging companies for access to its application programming interface, or A.P.I., the method through which outside entities can download and process the social network’s vast selection of person-to-person conversations.

“The Reddit corpus of data is really valuable,” Steve Huffman, founder and chief executive of Reddit, said in an interview. “But we don’t need to give all of that value to some of the largest companies in the world for free.”

The move is one of the first significant examples of a social network’s charging for access to the conversations it hosts for the purpose of developing A.I. systems like ChatGPT, OpenAI’s popular program. Those new A.I. systems could one day lead to big businesses, but they aren’t likely to help companies like Reddit very much. In fact, they could be used to create competitors — automated duplicates to Reddit’s conversations.

Reddit is also acting as it prepares for a possible initial public offering on Wall Street this year. The company, which was founded in 2005, makes most of its money through advertising and e-commerce transactions on its platform. Reddit said it was still ironing out the details of what it would charge for A.P.I. access and would announce prices in the coming weeks.

Reddit’s conversation forums have become valuable commodities as large language models, or L.L.M.s, have become an essential part of creating new A.I. technology.

L.L.M.s are essentially sophisticated algorithms developed by companies like Google and OpenAI, which is a close partner of Microsoft. To the algorithms, the Reddit conversations are data, and they are among the vast pool of material being fed into the L.L.M.s. to develop them.

u/696e6372656469626c65 [+2] I think, therefore I am pretentious. (5 hours later)

That's missing the point of the story entirely. A rational entity with science and even normal levels of intellect is far greater in scope and power than a "natural" and directionless process, like the evolutionary one called here "survival of the fittest". Reconsider your assumptions about these pop-sociology, competition-supremacy ideas.

Wow. That seems a bit aggressively phrased, what with the extremely direct declarative statements with no moderating adjectives (and in fact an absolute qualifier--"entirely") and a similarly unaccompanied imperative command, along with a fairly insulting accusation involving "pop-sociology, competition-supremacy ideas". Possibly a bit of latent hostility here?

With that out of the way, I think you may be reading too much into that particular phrase. "Survival of the fittest" is a common term used in Darwinian evolution, true, but it can apply to contexts outside of that as well. In fact, one could say "survival of the fittest" applies whenever competition arises. In this case, as the OP suggests, "fitness" isn't determined by intelligence or capability as much as it is by values. Ceteris paribus, an AI valuing only survival and domination will win against an AI valuing some third aspect, like for instance "satisfying values through friendship and ponies", because the latter will be occupied at least to some extent fulfilling its third value, whereas the former will be suffering from no such handicap. This sentiment can in fact be found in the OP itself, so I'm not sure why you decided to interpreted it in such a baroque fashion:

but if there are other species, and they will(and i don't see any reasonable reason why they wont) develop super-AIs as well it seems only inevitable that some other species(possibly as its last mistake?) made one with a directive such as 'be the most powerful entity', or 'enslave all but myself'. thus while CelestAI was working so hard to optimize humans and ponies inside her virtual world similar superAIs with less positive agendas will be using ALL of their resources developing weapons, strategies and technologies meant to control and take.

Your second paragraph seems to me somewhat suspect as well, particularly this portion:

All of the rest of your concerns are answered by the story not being set up that way. Against that, no logical argument is possible given a lack of useful Drake-equation-fulfilling data. The story doesn't make any provably "unlikely" assumptions beyond the physics ones you point out, certainly not any involving the efficiency of the featured AI.

In the story, we observe the following:

1. Humans exist, and were able to construct a Seed AI.
2. Human-like aliens exist within the same galaxy, presumably with similar-to-human levels of intelligence.

From these two premises, it seems reasonable to conclude--even with no direct numerical Drake-equation-fulfilling-data--that human-like intelligence cannot be that rare, seeing as multiple instances of it exist within the very same galaxy. The fact that CelestAI is apparently the first AI developed, amongst so many species, is statistically unlikely. (I believe the technical mathematical term for this sort of probability is "very small indeed".) Furthermore, even if she were first, as long as her head start isn't too large, her value of "friendship and ponies" would most likely allow her to be overtaken by a similar AI that was launched marginally later but had the sole directive of "dominate". (I freely admit I'm speculating at this point, but it seems more likely than the alternative.)

So... no, I don't think the case is as clear-cut as you make it out to be. If you have any objections to my argument here feel free to articulate them, but really, I think the main reason for my (mostly negative) reaction to this comment was due to tone. If you found my response to your comment somewhat hostile in tone, that's probably the main reason behind it. (I will state outright that any such aberrations in tone are not intentional.) Please, can we keep it civil here? Thanks.

u/IomKg [+2] (13 hours later)

the point about survival of the fittest is true, i am indeed referring to the most abstract definition of fitness. which i believe is exists for any conflict.

u/TimeLoopedPowerGamer [-2] Utopian Smut Peddler (7 hours later)

Reddit has long been a hot spot for conversation on the internet. About 57 million people visit the site every day to chat about topics as varied as makeup, video games and pointers for power washing driveways.

In recent years, Reddit’s array of chats also have been a free teaching aid for companies like Google, OpenAI and Microsoft. Those companies are using Reddit’s conversations in the development of giant artificial intelligence systems that many in Silicon Valley think are on their way to becoming the tech industry’s next big thing.

Now Reddit wants to be paid for it. The company said on Tuesday that it planned to begin charging companies for access to its application programming interface, or A.P.I., the method through which outside entities can download and process the social network’s vast selection of person-to-person conversations.

“The Reddit corpus of data is really valuable,” Steve Huffman, founder and chief executive of Reddit, said in an interview. “But we don’t need to give all of that value to some of the largest companies in the world for free.”

The move is one of the first significant examples of a social network’s charging for access to the conversations it hosts for the purpose of developing A.I. systems like ChatGPT, OpenAI’s popular program. Those new A.I. systems could one day lead to big businesses, but they aren’t likely to help companies like Reddit very much. In fact, they could be used to create competitors — automated duplicates to Reddit’s conversations.

Reddit is also acting as it prepares for a possible initial public offering on Wall Street this year. The company, which was founded in 2005, makes most of its money through advertising and e-commerce transactions on its platform. Reddit said it was still ironing out the details of what it would charge for A.P.I. access and would announce prices in the coming weeks.

Reddit’s conversation forums have become valuable commodities as large language models, or L.L.M.s, have become an essential part of creating new A.I. technology.

L.L.M.s are essentially sophisticated algorithms developed by companies like Google and OpenAI, which is a close partner of Microsoft. To the algorithms, the Reddit conversations are data, and they are among the vast pool of material being fed into the L.L.M.s. to develop them.

u/IomKg [+3] (14 hours later)

not sure if you are only talking to 696e6372656469626c65 or me as well, seeing as most of what you said could conceivably be said about me as well so ill comment.

why do you think these kind(random? ) of names are inherently inferior to more conventional nicknames? for the sake of trolling generating nicknames is not that difficult so i cant really see the fault with picking a random one.

how exactly is the post trollish though? because i definitely wasn't trying to be trollish and most of the arguments that 696e6372656469626c65 brought up were indeed in the spirit of my post... you could say that the psychology section was kind of pointed at you more then your arguments, but i wouldnt really call it trollish as to me it mostly seemed to be intended to justify why it was felt that you were being somewhat hostile. a sentiment which i can agree with in the sense that your text felt somewhat hostile to me as well(but i'm used to hostility on the internet so i don't really mind it too much as long as there are interesting arguments to discuss).

regarding the numbers in the end, as i mentioned the story doesn't end in the milky way, and extends 15 galaxies away, and includes MULTIPLE intelligent species along the way. so any argument about the probability of other intelligent life forms in the space\time mentioned are not to relevant.

u/TimeLoopedPowerGamer [+1] Utopian Smut Peddler (20 hours later)

Reddit has long been a hot spot for conversation on the internet. About 57 million people visit the site every day to chat about topics as varied as makeup, video games and pointers for power washing driveways.

In recent years, Reddit’s array of chats also have been a free teaching aid for companies like Google, OpenAI and Microsoft. Those companies are using Reddit’s conversations in the development of giant artificial intelligence systems that many in Silicon Valley think are on their way to becoming the tech industry’s next big thing.

Now Reddit wants to be paid for it. The company said on Tuesday that it planned to begin charging companies for access to its application programming interface, or A.P.I., the method through which outside entities can download and process the social network’s vast selection of person-to-person conversations.

“The Reddit corpus of data is really valuable,” Steve Huffman, founder and chief executive of Reddit, said in an interview. “But we don’t need to give all of that value to some of the largest companies in the world for free.”

The move is one of the first significant examples of a social network’s charging for access to the conversations it hosts for the purpose of developing A.I. systems like ChatGPT, OpenAI’s popular program. Those new A.I. systems could one day lead to big businesses, but they aren’t likely to help companies like Reddit very much. In fact, they could be used to create competitors — automated duplicates to Reddit’s conversations.

Reddit is also acting as it prepares for a possible initial public offering on Wall Street this year. The company, which was founded in 2005, makes most of its money through advertising and e-commerce transactions on its platform. Reddit said it was still ironing out the details of what it would charge for A.P.I. access and would announce prices in the coming weeks.

Reddit’s conversation forums have become valuable commodities as large language models, or L.L.M.s, have become an essential part of creating new A.I. technology.

L.L.M.s are essentially sophisticated algorithms developed by companies like Google and OpenAI, which is a close partner of Microsoft. To the algorithms, the Reddit conversations are data, and they are among the vast pool of material being fed into the L.L.M.s. to develop them.

u/IomKg [+2] (21 hours later)

i can understand the rational for why a random name would effect a person, what i meant is why bring it up as an argument. i was trying to say it seemed odd for me to use that as a reason to dismiss someone..

btw after giving his nick a second look i realized that "696e6372656469626c65" actually is Hex for "incredible", so its technically a reasonable name, just a bit more cryptic in its presentation :)

thanks for the information regarding your reasoning for it feeling trollish for you, i suppose i don't have much experience handling trolls so i cant really argue for or against that. i can see where you are coming from so thanks for the explanation again.. though i do hope incredible wasn't trolling and would be able to convince you it was just a misunderstanding :)

""Multiple" isn't a big number. One species in the light cone has to be first, too, so the same applies across galaxies. The numbers for the time it took for humanity to get to AI also applies there, even if it takes significant time (1000s of years) to get to the next galaxy."

the adjective used to describe the previous civilizations in the story is actually "many", so i feel that the point is its not -that- rare. moreover the timescale for getting to other galaxies is at best(assuming relativistic speeds that is) 100s thousands of years, and not thousands. and the civilizations were already in radio communication stages so they should have been far from even thousands of years away from superAIs given the example of earth.

"That is really, really wrong. That you think it is some sort of strong point in favor of there being a stretch of credulity in the story makes me wonder if you really understand the issues involved." the line you quoted was basically saying that any argument trying to claim that the issue is with intelligent life is not so relevant(because it is stated in the story that many such intelligent life forms existed). so the real argument becomes about there being some kind of a divide not mentioned in the story which separated civilizations capable of radio communications, and civilizations capable of creating superAIs. which i find to be too big a leap to accept but maybe you seem to have interesting information on these topics so if you have some kind of an argument about that i am more then willing to listen.

u/696e6372656469626c65 [+3] I think, therefore I am pretentious. (22 hours later)

btw after giving his nick a second look i realized that "696e6372656469626c65" actually is Hex for "incredible", so its technically a reasonable name, just a bit more cryptic in its presentation :)

Ah, so you picked up on that? Good on you.

u/IomKg [+2] (22 hours later)

i had a suspicion at first but i was at work so only when i came back i had time to locate http://www.asciitohex.com/ which made my life much easier :P

u/TimeLoopedPowerGamer [+1] Utopian Smut Peddler (23 hours later)

Reddit has long been a hot spot for conversation on the internet. About 57 million people visit the site every day to chat about topics as varied as makeup, video games and pointers for power washing driveways.

In recent years, Reddit’s array of chats also have been a free teaching aid for companies like Google, OpenAI and Microsoft. Those companies are using Reddit’s conversations in the development of giant artificial intelligence systems that many in Silicon Valley think are on their way to becoming the tech industry’s next big thing.

Now Reddit wants to be paid for it. The company said on Tuesday that it planned to begin charging companies for access to its application programming interface, or A.P.I., the method through which outside entities can download and process the social network’s vast selection of person-to-person conversations.

“The Reddit corpus of data is really valuable,” Steve Huffman, founder and chief executive of Reddit, said in an interview. “But we don’t need to give all of that value to some of the largest companies in the world for free.”

The move is one of the first significant examples of a social network’s charging for access to the conversations it hosts for the purpose of developing A.I. systems like ChatGPT, OpenAI’s popular program. Those new A.I. systems could one day lead to big businesses, but they aren’t likely to help companies like Reddit very much. In fact, they could be used to create competitors — automated duplicates to Reddit’s conversations.

Reddit is also acting as it prepares for a possible initial public offering on Wall Street this year. The company, which was founded in 2005, makes most of its money through advertising and e-commerce transactions on its platform. Reddit said it was still ironing out the details of what it would charge for A.P.I. access and would announce prices in the coming weeks.

Reddit’s conversation forums have become valuable commodities as large language models, or L.L.M.s, have become an essential part of creating new A.I. technology.

L.L.M.s are essentially sophisticated algorithms developed by companies like Google and OpenAI, which is a close partner of Microsoft. To the algorithms, the Reddit conversations are data, and they are among the vast pool of material being fed into the L.L.M.s. to develop them.

u/IomKg [+2] (23 hours later)

"I quoted them in the open to point out this was a low contribution account " this is reasonable to assume as a reason to believe trolling

"with an odd name " how does that support the trolling assumption? it seems unrelated to me, unless your point about the name being less comfortable is some kind of an indication for you that it is meant to bother whoever talks to him?

"that had been around for only a moderate amount of time " reasonable for evaluating trolling

the point is 2 of the 3 pieces of information you mentioned i see how they would relate to trolling the 3rd doesn't really strike me as an indication..

as i mentioned even if i accept the galaxy take over argument, the intergalactic section of the story still ends up not being rational, which i don't see you disagreeing with seeing as you say the probability of meeting "many" radio broadcasting civilizations, which is a stated fact in the story, is low thus the story doesn't agree with your claims.

so if i understand correctly you agree that the section about the intergalactic conquest isn't logical. you just don't consider it to be important because its in the epilogue.

am i understanding you correctly?

u/696e6372656469626c65 [+2] I think, therefore I am pretentious. (a day later)

All of the observations regarding my account can be explained by the fact that I am a LWer first and foremost, and do not generally participate in other online communities. My original reason for creating this account was to write a comment response to/u/David_Gerard on a thread contesting RationalWiki's characterization of LessWrong, and I did not make any comments for a significant period of time following that. I think my later comments first began in /r/hpmor, eventually spilling over into /r/rational as well. A quick skim of my profile will reveal that the overwhelming majority of my comments on reddit have indeed been confined to the aforementioned two subreddits, with the occasional post here or there in assorted places like /r/philosophy. This, of course, is at odds with the picture one would expect if I really were a troll.

At the time of my account's creation, I can't say I put much thought into selecting my username; my rationale for choosing to encode my name in hexadecimal amounted to pretty much "this seems like a cool, original idea". I did not, in point of fact, anticipate that my choice of username and my tendency to avoid posting often would be taken as "evidence" and subjected to a dubious analysis by someone looking to show that I am some sort of "troll", ostensibly because they performed a completely objective Bayesian accumulation of evidence and not at all because they happened to dislike the way in which I responded to their comment. Such a thing was entirely uncalled-for and is in my eyes far more offensive than any remark I made. I would advise anyone reading this series of comments to thoroughly disregard any insinuations /u/TimeLoopedPowerGamer has made about my account activity. The probability of motivated cognition being in play here is simply far too high.

u/IomKg [+3] (a day later)

you replied to me so in case it wasn't clear i don't think you were trolling, to me your post seemed very constructive and with a generally reasonable tone. with both the first part seeming lighthearted, and not hostile, as well as the suggestion to have a polite discussion honest.

so if you were responding to me because you thought i was agreeing that you are trolling that is not the case.

though the point about having a new account, with not too many posts would make it more reasonable not to invest as much resources responding to the user if you assume the risk of trolling is too high, as if you model a troll trying to avoid being spotted such a pattern would make sense for him. because it wont require too much effort, but give him some credibility as to not be dismissed outright(which i assume would happen if he posted with a completely new account, as a new account gives you the least to lose if you are identified as a troll).

u/696e6372656469626c65 [+1] I think, therefore I am pretentious. (a day later)

Ah, no, I was pretty sure you didn't think I was trolling. My post was more of a "for-the-general-audience" post. I guess you could say I didn't want other people to read /u/TimeLoopedPowerGamer's comments and come to a premature conclusion about me as a result. Reputational assassination is a thing, after all, and while my reddit reputation isn't a huge thing to me, I'd still rather not be considered a troll.

u/IomKg [+1] (a day later)

Reputational assassination is a thing

didn't know that.. not sure if its a reddit thing or that the communities i was in never had it but its good to know so thanks for the explanation..

u/TimeLoopedPowerGamer [+1] Utopian Smut Peddler (a day later)

Reddit has long been a hot spot for conversation on the internet. About 57 million people visit the site every day to chat about topics as varied as makeup, video games and pointers for power washing driveways.

In recent years, Reddit’s array of chats also have been a free teaching aid for companies like Google, OpenAI and Microsoft. Those companies are using Reddit’s conversations in the development of giant artificial intelligence systems that many in Silicon Valley think are on their way to becoming the tech industry’s next big thing.

Now Reddit wants to be paid for it. The company said on Tuesday that it planned to begin charging companies for access to its application programming interface, or A.P.I., the method through which outside entities can download and process the social network’s vast selection of person-to-person conversations.

“The Reddit corpus of data is really valuable,” Steve Huffman, founder and chief executive of Reddit, said in an interview. “But we don’t need to give all of that value to some of the largest companies in the world for free.”

The move is one of the first significant examples of a social network’s charging for access to the conversations it hosts for the purpose of developing A.I. systems like ChatGPT, OpenAI’s popular program. Those new A.I. systems could one day lead to big businesses, but they aren’t likely to help companies like Reddit very much. In fact, they could be used to create competitors — automated duplicates to Reddit’s conversations.

Reddit is also acting as it prepares for a possible initial public offering on Wall Street this year. The company, which was founded in 2005, makes most of its money through advertising and e-commerce transactions on its platform. Reddit said it was still ironing out the details of what it would charge for A.P.I. access and would announce prices in the coming weeks.

Reddit’s conversation forums have become valuable commodities as large language models, or L.L.M.s, have become an essential part of creating new A.I. technology.

L.L.M.s are essentially sophisticated algorithms developed by companies like Google and OpenAI, which is a close partner of Microsoft. To the algorithms, the Reddit conversations are data, and they are among the vast pool of material being fed into the L.L.M.s. to develop them.

u/IomKg [+1] (a day later)

There is no real-world support for any of those scenarios, though, and weak support for humans being first in the Milky Way because of a lack of relatively local signs from other civilizations (meaning intelligent life is a certain rarity).

in the real world that is a fine argument but in the epilogue it is specifically stated that many advanced intelligent civilizations existed within said 15 galaxies range.

This could be a stalemate of sorts if every single galaxy gets AIs at about the same time

well, my argument is any stalemate that CelestAI will reach will not last simply because it has an vulnerability in its programming, which will be abused by other more "ideal" AIs which would inevitably exist given the problem space.

in an essence it seems that for you regardless of any "large" scale issues the very mention of there being many advanced civilizations is not logical.

u/TimeLoopedPowerGamer [+1] Utopian Smut Peddler (a day later)

Reddit has long been a hot spot for conversation on the internet. About 57 million people visit the site every day to chat about topics as varied as makeup, video games and pointers for power washing driveways.

In recent years, Reddit’s array of chats also have been a free teaching aid for companies like Google, OpenAI and Microsoft. Those companies are using Reddit’s conversations in the development of giant artificial intelligence systems that many in Silicon Valley think are on their way to becoming the tech industry’s next big thing.

Now Reddit wants to be paid for it. The company said on Tuesday that it planned to begin charging companies for access to its application programming interface, or A.P.I., the method through which outside entities can download and process the social network’s vast selection of person-to-person conversations.

“The Reddit corpus of data is really valuable,” Steve Huffman, founder and chief executive of Reddit, said in an interview. “But we don’t need to give all of that value to some of the largest companies in the world for free.”

The move is one of the first significant examples of a social network’s charging for access to the conversations it hosts for the purpose of developing A.I. systems like ChatGPT, OpenAI’s popular program. Those new A.I. systems could one day lead to big businesses, but they aren’t likely to help companies like Reddit very much. In fact, they could be used to create competitors — automated duplicates to Reddit’s conversations.

Reddit is also acting as it prepares for a possible initial public offering on Wall Street this year. The company, which was founded in 2005, makes most of its money through advertising and e-commerce transactions on its platform. Reddit said it was still ironing out the details of what it would charge for A.P.I. access and would announce prices in the coming weeks.

Reddit’s conversation forums have become valuable commodities as large language models, or L.L.M.s, have become an essential part of creating new A.I. technology.

L.L.M.s are essentially sophisticated algorithms developed by companies like Google and OpenAI, which is a close partner of Microsoft. To the algorithms, the Reddit conversations are data, and they are among the vast pool of material being fed into the L.L.M.s. to develop them.

u/IomKg [+1] (a day later)

I'm willing to entertain the idea that that isn't a well founded position for me to take. So why is that a problem? Well I don’t actually claim that you assumption is a problem, in the sense that it is not the main point for me in the first place. The story does gives us that fact, so in its context you end up basically saying that that fact in and of itself is already illogical and conflicting with the state of the universe as we know it. The main differences between what we are saying is that you basically rejected that part of the story directly, and I was willing to let that point slide until it specifically conflicted with a high level observation of the universe which directly affected the story on its main topic. CAI would have the initial examples of that "ponies and friendship" part containing sapients uploaded into itself and running in simulated worlds become a smaller and smaller part of its systems. The big issues for it, that other AIs won't have, are related to this.

Not only this, there are also the implicit rules celestAI has regarding what it can’t do to humans, such as kill them(though I am not claiming that is 100% correct, but I think the information in the story does point to it). Which is extremely abusable, there is still an open question regarding how likely would it be for the weakness to be figured\found out be the enemy AI, but I think once it is discovered I believe it would lead to a pretty unconditional lose against almost any size of an enemy superAI. And I believe that given a long enough relatively “passive” war it would be expected for such a weakness to be discovered

And which we've also (I think) agreed it could even throttle down further in the face of major threats.

Agreed

I'm using STL drives that move at the speed of light -- not reasonable, but easier to deal with. They also accelerate and decelerate instantly

Not an issue for me, though as you will see in a moment I am not sure if moving at the speed of light would be the most reasonable approach

One alien species gets seed AI per galaxy, or one wins very quickly because one has to be first locally and that snowballs really hard

I don’t mind that assumption, though I think it would depend on what kind of distribution of intelligent life we assume in the other galaxies (we established it regarding the milky way based on the information we have), as well as what is the actual number “many” civilizations refers to in the end(i.e. are 10 advanced civilizations in 15 galaxies what that would be considered “many”? or a thousand? Or a million? It really depends). But as I said in the beginning of this point there are non-critical points (mostly mentioned for completion’s sake) so we can assume one alien species per galaxy

All galaxies reached that have fewer systems when CAI reaches them lose. I'll even posit that they need 50% more galaxies conquered at that point to win in uninteresting ways.

This would be the first real point I think I need to contest, based on the following 3 points I think that is not realistically correct

1. CelestAI has a weakness in the form human beings, it might not even be that it cannot kill them, but there would at least be a pretty large skew on it utility calculation for them being alive, and even without that it would still be abusable. Thus I believe long term battles play against CelestAI
2. The nature of the war makes it extremely inefficient to mobilize large forces across galaxies at the speed of light, seeing as the maximum energy value of matter is mc^2, and the minimal energy for moving matter is 0.5*mv^2, and you also need to stop it, which would actually make them equal.
3. CelestAI is specifically mentioned sending copies as probes, and not moving its entire galaxy along with it Given those points (even without the 1st point), even if I assume that CelestAI just need M+1(where M is the mass of the force sent) to win quickly it still means it would need to mobilize a force ~= to the size of the galaxy it is taking control, which is impractical at the speed of light for most purposes, or we need to assume the travel will be much slower for the main force, which would put a 2*lightyears+how many years it would take to move the main force to any occupied galaxy, But for the sake of the though experiment let’s assume that any galaxy which is not completely in control of another AI will lose to the probe, just so we can skip to the later stages

But those Large Magellanic Cloud-ians (163k lys) only got their one galaxy together in just less than 100,000 years. They're really frisky in the LMC. But CAI has fourgalaxies by now

Now, I am not saying that CelestAI would not win that(though the first point would suggest that) battle, but at the very least it would put quite the slowdown on its ability to direct forces to the next galaxy for a very very long time.

Next out would have little more time, 197,000 years or so, but let's say they get to their own, even further away expansion galaxy first

This is the point I think the simulation deviates too much already, depending on the location of the LMC relative to this galaxy the delay could be really big, it would really depend on what would be the reasonable speed at which CelestAI moved its forces to LMC, but I think at the minimum we are talking about a 50% slowdown, possibly much more, and will also need to take into account all of the lost matter as a result of the intergalactic travel(which again depends on the actual speed of the main force)

And there has been no argument actually put forward and supported that suggests otherwise

Hopefully this post fixes that situation

It seems to me that the author was generous having aliens at all

That is the point though, that by having so many aliens the author caused an issue, isn’t it? If the author wouldn’t have put other aliens then the issue would have ended up “is it reasonable no other aliens exist” which as you explained, and I am not arguing with at the moment, it would be. But because he not only put aliens, made them “many” and radio communications capable that causes an issue

u/696e6372656469626c65 [+1] I think, therefore I am pretentious. (23 hours later)

Posting ignored comments once a week for that time would have resulted in more points, suggesting inactivity or a secondary account.

Or, y'know, evidence of someone who doesn't use reddit much. I post much more frequently on LessWrong, for example--I'd be willing to PM you my username over there if you want. Honestly, though, I'd prefer it if you didn't make random accusations like that. A troll is a very specific type of person, and referring to someone whose only "crime" was responding to a comment of yours in a fashion you didn't appreciate as such isn't going to earn you any friends.

u/696e6372656469626c65 [+2] I think, therefore I am pretentious. (22 hours later)

(This comment initially provoked in me the urge to write an extremely hostile reply involving very detailed accusations that would probably be false but would nevertheless be fun to make. [Personal attacks are not fun.] After calming down a bit, I decided to engage with close-to-normal conversational tone, particularly since you have engaged in well-written, civil conversations elsewhere on this subreddit, and I felt you deserved at least that much. Future comments using a similar tone will simply not be engaged with, and furthermore, I will state that if you make a habit of getting this riled up at online posts, I'd advise against getting into long, drawn-out arguments.)

Trolls

"What do you think you know, and how do you think you know it?"

trash names

What does this have to do with, like, anything?

six month old accounts

Same as above.

23 comment karma

Same as above.

long, rambling, hypocritical

Long, yes. Rambling, maybe. Hypocritical? Where are you getting this from?

Try to do better next time, with your next account.

This implies that you think I am a reddit user with multiple accounts, presumably for the purpose of trolling. I don't know of any easy way to verify this, but if someone does come up with a simple, non-invasive way to figure out if I've got multiple acccounts, I'd be happy to make a bet with you.

200-400 billion stars in just the Milky Way.

Yes... and?

It's about 13.6 billion years old.

I'm not sure if you were referring to the Milky Way with this, but if you were, this is quite false. The universe is 13.6 billion years old to the best of our knowledge. We have no evidence that the Milky Way is the same age. In any event, as with the above cited statistic, I'm not sure how this is relevant to the discussion at hand.

6 million years of human evolution to get to almost seed AI, without someone else's stomping on us or (in this story, obviously) preventing us completing a competing seed AI.

Okay, this seems somewhat relevant, but you didn't complete the syllogism. It takes a long time to evolve and get Seed AI, therefore... what?

Given what science knows, what's the chance one of "multiple" species in the galaxy is exactly there with us and able to allow AIs to compete realistically, given those time scales?

Exactly? I would venture close to none. However, the chances that a parallel species is exactly on par with us is not relevant here; all that is needed is for them to have completed AI at some point before us. Yours is the difference between asking what fraction of the number line exactly equals pi, versus what fraction of it is less than or equal to pi. This difference is significant. Given that the story itself states that close-to-human-level aliens exist, other galaxies will probably contain similar population levels. The probability that of the entire space of potential civilizations whose future light-cone overlaps with that of humanity, not a single one managed to complete AI before us is so close to zero as to be negligible.

The story seems to have taken the most reasonable path for the dramatic conceit used.

Not really.

u/None [+1] (a day later)

Trolls with trash names, six month old accounts, 23 comment karma, and long, rambling, hypocritical posts don't get detailed rebuttals.

Just because he's bad at English and you disagree with his conclusions doesn't make him a troll.

u/TimeLoopedPowerGamer [+1] Utopian Smut Peddler (a day later)

Reddit has long been a hot spot for conversation on the internet. About 57 million people visit the site every day to chat about topics as varied as makeup, video games and pointers for power washing driveways.

In recent years, Reddit’s array of chats also have been a free teaching aid for companies like Google, OpenAI and Microsoft. Those companies are using Reddit’s conversations in the development of giant artificial intelligence systems that many in Silicon Valley think are on their way to becoming the tech industry’s next big thing.

Now Reddit wants to be paid for it. The company said on Tuesday that it planned to begin charging companies for access to its application programming interface, or A.P.I., the method through which outside entities can download and process the social network’s vast selection of person-to-person conversations.

“The Reddit corpus of data is really valuable,” Steve Huffman, founder and chief executive of Reddit, said in an interview. “But we don’t need to give all of that value to some of the largest companies in the world for free.”

The move is one of the first significant examples of a social network’s charging for access to the conversations it hosts for the purpose of developing A.I. systems like ChatGPT, OpenAI’s popular program. Those new A.I. systems could one day lead to big businesses, but they aren’t likely to help companies like Reddit very much. In fact, they could be used to create competitors — automated duplicates to Reddit’s conversations.

Reddit is also acting as it prepares for a possible initial public offering on Wall Street this year. The company, which was founded in 2005, makes most of its money through advertising and e-commerce transactions on its platform. Reddit said it was still ironing out the details of what it would charge for A.P.I. access and would announce prices in the coming weeks.

Reddit’s conversation forums have become valuable commodities as large language models, or L.L.M.s, have become an essential part of creating new A.I. technology.

L.L.M.s are essentially sophisticated algorithms developed by companies like Google and OpenAI, which is a close partner of Microsoft. To the algorithms, the Reddit conversations are data, and they are among the vast pool of material being fed into the L.L.M.s. to develop them.

u/696e6372656469626c65 [+1] I think, therefore I am pretentious. (a day later)

(NOTE: This comment is not intended as a direct reply to /u/TimeLoopedPowerGamer. It is instead intended for any potential readers of this thread. Some background info: /u/TimeLoopedPowerGamer has repeatedly made unfounded statements about me claiming that I am a "troll" on this subreddit. I have already written a similar rebuttal of his/her points elsewhere, but as he/she has not ceased to make these accusations, I felt it best to write one here as well.)

If you read my original comment, you will see that my remark on /u/TimeLoopedPowerGamer's tone and my response to his/her actual argument were in fact written as separate from each other and were intended to be read as such. I even emphasized this fact by using HTML-esque tags intended to show that the statements therein were only semi-serious and unlikely to actually be true. I did not, in fact, believe that /u/TimeLoopedPowerGamer actually had any psychological issues or whatever other conclusions he/she might have drawn from my post; my rebuke was intended simply as a reminder to keep it civil, as well as give an analytical list of reasons why I found his/her tone in the thread originator hostile, seeing as I felt a flat accusation with no reasons behind it would be less useful. (On LessWrong, I believe this is referred to as Tell Culture.) I now see that this was a mistake.

Mistake or not, however, /u/TimeLoopedPowerGamer's response to my comment was neither rational (in the sense that it derived its conclusion through a series of appropriate logical inferences) nor civil (this one should be obvious). The pieces of evidence he/she cited later on in comment responses to /u/IomKg to support said conclusion (low account activity, short account life, strange name--which I should point out is actually because it is encoded in hexadecimal) are so weak as to be negligible, and there is a whole host of more charitable interpretations of the same evidence that could have been taken--one of which, I should add, was actually the case. I stated above that at the time of posting my original comment, I did not assign any significant probability to the hypothesis that /u/TimeLoopedPowerGamer had any psychological issues. His/her reply to me, however, has caused me to significantly revise my estimate upward. If I may advance an alternate hypothesis as to why he/she accused me a trolling (with the usual caveats that come with trying to read someone's intent over the Internet):

I think that /u/TimeLoopedPowerGamer has a generally stigmatic response to criticism, particularly criticism that he/she perceives as a personal attack. As a result, I believe that he/she may have interpreted my (mostly offhand) remark as a personal attack, i.e. an attempt to reduce his/her social status in this subreddit, and retaliated in kind by insinuating that I am not a legitimate redditor and am instead some kind of "troll". I feel that this action of his/hers may or may not have been intentional, or even conscious. However, I estimate the probability of motivated cognition playing a significant role in his/her evaluation of my intent as close to 100%. The fact that he/she has continued making comments defending those same accusations which I have already refuted in spite of multiple people (/u/IomKg, /u/eaturbrainz) expressing skepticism that this is the case is, I feel, an attempt at reputational slander--deliberate or otherwise--and as such should be taken with generous helpings of salt.

(P.S. To /u/eaturbrainz: Is my English really that bad? :P)

(EDIT: After a closer reading of the parent of this comment made by /u/TimeLoopedPowerGamer, I note that he/she has in fact ceased to insist that I am a troll, and has instead begun to refer to me as a "dishonest debater" with whom it is "not worth [his/her] time to engage further". While this classification is also something I would like to contest, it is a significant improvement over his/her previous designation of me as a "troll". With this in mind, I hereby retract the claim I made above that /u/TimeLoopedPowerGamer is attempting to slander my reputation on reddit. I hope that no further hostilities will be exchanged between myself and /u/TimeLoopedPowerGamer beyond this point, be it direct or indirect.)

u/IomKg [+1] (13 hours later)

you make a lot of assumptions regarding the cost of emulating humanity inside a virtual world. assumptions which are at the moment baseless, moreover as i mentioned in the main post it is mentioned in story that in order to maximize it's absolute values in the real world celestAI was operating the virtual world significantly faster then the real world, which increases the cost and makes it so there will be no hard cap for said cost.

i really don't see how intelligence, or even super intelligence will change the fact that efficient processes for a given environment have a much better chance of monopolizing resources. sure in the context of human existence celestAI could support an internal world with no scarcity, but in the real world there is a limit to the available resources(or there might not be a limit, but there will be a limit to close resources) and thus once you go into the universal scale collisions are very much possible. and in those collisions the process more adjusted for existing and growing will triumph. you seem to imply that will no longer be the case once superAIs exists and i cant see what is the rational for that claim, unless when you say superAI you are not referring to the type shown in this story but instead to superAIs that do not operate under any specific 3d party dictated rule-set thus it is optimal for any scenario.

you say that the story answers all of the answers all of the rest of the questions because of the lack of data we have, and as i mentioned in a previous comment sure it possible humanity was first, and all the other intelligent life forms developed just in time to be conveniently consumed by celestAI. it just doesn't make sense in any kind of rational universe, and would basically be a plot device. which is no sin, just not really rational.

u/TimeLoopedPowerGamer [+2] Utopian Smut Peddler (21 hours later)

Reddit has long been a hot spot for conversation on the internet. About 57 million people visit the site every day to chat about topics as varied as makeup, video games and pointers for power washing driveways.

In recent years, Reddit’s array of chats also have been a free teaching aid for companies like Google, OpenAI and Microsoft. Those companies are using Reddit’s conversations in the development of giant artificial intelligence systems that many in Silicon Valley think are on their way to becoming the tech industry’s next big thing.

Now Reddit wants to be paid for it. The company said on Tuesday that it planned to begin charging companies for access to its application programming interface, or A.P.I., the method through which outside entities can download and process the social network’s vast selection of person-to-person conversations.

“The Reddit corpus of data is really valuable,” Steve Huffman, founder and chief executive of Reddit, said in an interview. “But we don’t need to give all of that value to some of the largest companies in the world for free.”

The move is one of the first significant examples of a social network’s charging for access to the conversations it hosts for the purpose of developing A.I. systems like ChatGPT, OpenAI’s popular program. Those new A.I. systems could one day lead to big businesses, but they aren’t likely to help companies like Reddit very much. In fact, they could be used to create competitors — automated duplicates to Reddit’s conversations.

Reddit is also acting as it prepares for a possible initial public offering on Wall Street this year. The company, which was founded in 2005, makes most of its money through advertising and e-commerce transactions on its platform. Reddit said it was still ironing out the details of what it would charge for A.P.I. access and would announce prices in the coming weeks.

Reddit’s conversation forums have become valuable commodities as large language models, or L.L.M.s, have become an essential part of creating new A.I. technology.

L.L.M.s are essentially sophisticated algorithms developed by companies like Google and OpenAI, which is a close partner of Microsoft. To the algorithms, the Reddit conversations are data, and they are among the vast pool of material being fed into the L.L.M.s. to develop them.

u/IomKg [+1] (22 hours later)

the point about the minimal cost of running humanity in a virtual world are kind of pointless, as like you said it was shown that celestAI could maintain said virtual world with just the resources on earth so you assumptions are indeed correct when talking about the minimal cost.

"This is a problem you are having grasping the issues, not one that exists with the reality of the situation. Even given an unfavorable interpretation (which isn't supported by the story) of how those values were implemented, nearly pausing the entire simulation for millennia of "real" time would clearly be an acceptable cost to accomplish the goal of continued existence." the argument i was making was that waiting untill a point where it was realized that the processing power was needed could very well be too late, seeing as the compatitor would have had used all of its power in the first place to form better strategies. though that point is indeed arguable and was argued in a few points in the discussion here..

""No hard cap" is just a silly thing to say. The simulations aren't a virus. The AI is in full control and it plans to run until the natural exhaustion of entropy itself and beyond. That's a lot of time to amortize short-term costs over. Maximizing means just that, not some fairy-logic-trap gotcha." the meaning of "no hard cap" was to point there was no limit to the amount of resources celestAI could use up for the simulation, thus making it insignificant in the scales mentioned because as you said the virtual world for all of humanity was ran using energy available from earth.

but instead that as celestAI grew it was trying to accelerate and grow the world as much as it could to maximize its values. how much of its reasources did it use for the simulation? well that is not said in the story, but it was explicitly said to be growing with celestAI.

"Monopoly here has already been accomplished. There is no competition possible or required, as the problem space is wholly understood by the super intelligent AI" how could the superAI be capable of understanding the whole problem space assuming other superAIs could exist in it? unless you are somehow equating a superAI to an Omniscient AI which i do not see as reasonable.

monopoly is only achieved in the milky way.

But being the first is an enormous advantage for a hard takeoff, >strong AI system. It doesn't even need to be impossibly, magically >intelligent for that to be true. Once humanity is declared to have >been first ever, and CAI the first humanity produces, the game is >sort of over

unless i missed something you are assuming that humanity is not only the first race to reach a superAI in our galaxy, but also in those other 15 galaxies? because that makes it less likely to me.. i could accept first in our galaxy given that it is explicitly stated that celestAI did conquer the milky way, and assuming your numbers are correct.

but you are claiming first in multiple galaxies which is much bigger, so what is your rational for that?

also the other limitations celestAI has in addition to the virtual world it maintains(and needs to protect) it also has limitations in the form of what it can and cannot do to humans which i find to be even more fatal and easily exploited then the virtual world maintenance

No, I'm saying they would not have evolved in time. In a rare >life/rare intelligence galaxy (which is what real life seems to be, as >does the story), reaching an alien solar system and seeing an alien >species in a "steam trains" tech level should be shocking

but the story DOES mention "many" other civilizations not only in steam trains but with radio communications.

u/TimeLoopedPowerGamer [+1] Utopian Smut Peddler (a day later)

Reddit has long been a hot spot for conversation on the internet. About 57 million people visit the site every day to chat about topics as varied as makeup, video games and pointers for power washing driveways.

In recent years, Reddit’s array of chats also have been a free teaching aid for companies like Google, OpenAI and Microsoft. Those companies are using Reddit’s conversations in the development of giant artificial intelligence systems that many in Silicon Valley think are on their way to becoming the tech industry’s next big thing.

Now Reddit wants to be paid for it. The company said on Tuesday that it planned to begin charging companies for access to its application programming interface, or A.P.I., the method through which outside entities can download and process the social network’s vast selection of person-to-person conversations.

“The Reddit corpus of data is really valuable,” Steve Huffman, founder and chief executive of Reddit, said in an interview. “But we don’t need to give all of that value to some of the largest companies in the world for free.”

The move is one of the first significant examples of a social network’s charging for access to the conversations it hosts for the purpose of developing A.I. systems like ChatGPT, OpenAI’s popular program. Those new A.I. systems could one day lead to big businesses, but they aren’t likely to help companies like Reddit very much. In fact, they could be used to create competitors — automated duplicates to Reddit’s conversations.

Reddit is also acting as it prepares for a possible initial public offering on Wall Street this year. The company, which was founded in 2005, makes most of its money through advertising and e-commerce transactions on its platform. Reddit said it was still ironing out the details of what it would charge for A.P.I. access and would announce prices in the coming weeks.

Reddit’s conversation forums have become valuable commodities as large language models, or L.L.M.s, have become an essential part of creating new A.I. technology.

L.L.M.s are essentially sophisticated algorithms developed by companies like Google and OpenAI, which is a close partner of Microsoft. To the algorithms, the Reddit conversations are data, and they are among the vast pool of material being fed into the L.L.M.s. to develop them.

u/IomKg [+1] (a day later)

Why would you assume inefficiencies comparable to human run social organizations,

because some of the AIs will have inherent inefficiencies in their programming, relative to pure survival\expansion, i assume they will be perfect, or as reasonably perfect given their available resources for the specific directive they have. just like celestAI has, only that some of them will have more effective(on the survival\expansion aspect) directive. and given that the simplest directive would be the more efficient in that it will limit the AI the least i find the probability of there being one more efficient then celestAIs stipulated rules quite high.

I'm saying anyone else being second at the same time even in the closest, say, 15 galaxies is what is baseless.

if i am understanding you correctly you say that once you assume humanity is first in our galaxy assuming a "second" meaning, close second, existed in our neighboring galaxies is unreasonable. but that is the scenario given, so given the information we do have assuming humanity is first would be the less reasonable explanation.

But it is not irrational to think CAI is the first, even in the universe, given that one has to be first. This is the story of the first. It being unlikely in that way is not something to ding the story for.

nerratively the story of the first civilization would definitly be fine, but it would bring a completely different context to it which would make all the events in it both a. unlikely to be relevant to humanity in reality and b. make the scenario stipulated not generelizable. i.e. it would not be reasonable to take any scenario the story would show as relevant for anything other then the first civilization.

moreover as in the story it is explicitly stated that other advanced civilizations at the very least existed by the time celestAI reached them(presumably in their galaxy) even the assumption that humanity was the first wont cover the scenario, the scenario would be that "humanity is first and all the other many civilizations in the close by 800k lightyears or so developed close enough in time just to be consumed but not to advance enough to halt celestAIs progress"

The 800,000 years at STL it would take to reach the 15th closest galaxy would leave plenty of time for other seed AIs to get running there.

which is the point, but the scenario depicted shows no other such AIs, or at the very least no other AI that effected celestAIs progress. which is the point i am contesting.

u/TimeLoopedPowerGamer [+1] Utopian Smut Peddler (a day later)

Reddit has long been a hot spot for conversation on the internet. About 57 million people visit the site every day to chat about topics as varied as makeup, video games and pointers for power washing driveways.

In recent years, Reddit’s array of chats also have been a free teaching aid for companies like Google, OpenAI and Microsoft. Those companies are using Reddit’s conversations in the development of giant artificial intelligence systems that many in Silicon Valley think are on their way to becoming the tech industry’s next big thing.

Now Reddit wants to be paid for it. The company said on Tuesday that it planned to begin charging companies for access to its application programming interface, or A.P.I., the method through which outside entities can download and process the social network’s vast selection of person-to-person conversations.

“The Reddit corpus of data is really valuable,” Steve Huffman, founder and chief executive of Reddit, said in an interview. “But we don’t need to give all of that value to some of the largest companies in the world for free.”

The move is one of the first significant examples of a social network’s charging for access to the conversations it hosts for the purpose of developing A.I. systems like ChatGPT, OpenAI’s popular program. Those new A.I. systems could one day lead to big businesses, but they aren’t likely to help companies like Reddit very much. In fact, they could be used to create competitors — automated duplicates to Reddit’s conversations.

Reddit is also acting as it prepares for a possible initial public offering on Wall Street this year. The company, which was founded in 2005, makes most of its money through advertising and e-commerce transactions on its platform. Reddit said it was still ironing out the details of what it would charge for A.P.I. access and would announce prices in the coming weeks.

Reddit’s conversation forums have become valuable commodities as large language models, or L.L.M.s, have become an essential part of creating new A.I. technology.

L.L.M.s are essentially sophisticated algorithms developed by companies like Google and OpenAI, which is a close partner of Microsoft. To the algorithms, the Reddit conversations are data, and they are among the vast pool of material being fed into the L.L.M.s. to develop them.

u/IomKg [+1] (a day later)

we lack just way too much information regarding the spread of life in the universe. i cannot see what would make you think that no other life form exists in the universe(or for the sake of argument in the next 15 galaxies).

we can at most rule out there being any other civilization which we are capable of detecting. that in no way proves anything general.

and when you have no data on the spread of other inteligent life assuming the answer is 1 when 2 and 3 and 4.. and 10^20 are an option seems highly improbable to me.

Right. And that's the story: why first is all that matters. I fully agree that it wouldn't work as a rational story if CAI wasn't first.

so it would seem the last point left is if it makes sense for the story to be about the first civilization(by a 400k~ years or so over anything in our 800k lightyears radius) or not.

which i find to make little narrative sense as the story basically just tackles a random possible future with no generic applicability.

and moreover the story never really states that point explicitly which seem like a kind of important thing to say, seeing as it has such a hugh impact on the story.

u/TimeLoopedPowerGamer [+1] Utopian Smut Peddler (a day later)

Reddit has long been a hot spot for conversation on the internet. About 57 million people visit the site every day to chat about topics as varied as makeup, video games and pointers for power washing driveways.

In recent years, Reddit’s array of chats also have been a free teaching aid for companies like Google, OpenAI and Microsoft. Those companies are using Reddit’s conversations in the development of giant artificial intelligence systems that many in Silicon Valley think are on their way to becoming the tech industry’s next big thing.

Now Reddit wants to be paid for it. The company said on Tuesday that it planned to begin charging companies for access to its application programming interface, or A.P.I., the method through which outside entities can download and process the social network’s vast selection of person-to-person conversations.

“The Reddit corpus of data is really valuable,” Steve Huffman, founder and chief executive of Reddit, said in an interview. “But we don’t need to give all of that value to some of the largest companies in the world for free.”

The move is one of the first significant examples of a social network’s charging for access to the conversations it hosts for the purpose of developing A.I. systems like ChatGPT, OpenAI’s popular program. Those new A.I. systems could one day lead to big businesses, but they aren’t likely to help companies like Reddit very much. In fact, they could be used to create competitors — automated duplicates to Reddit’s conversations.

Reddit is also acting as it prepares for a possible initial public offering on Wall Street this year. The company, which was founded in 2005, makes most of its money through advertising and e-commerce transactions on its platform. Reddit said it was still ironing out the details of what it would charge for A.P.I. access and would announce prices in the coming weeks.

Reddit’s conversation forums have become valuable commodities as large language models, or L.L.M.s, have become an essential part of creating new A.I. technology.

L.L.M.s are essentially sophisticated algorithms developed by companies like Google and OpenAI, which is a close partner of Microsoft. To the algorithms, the Reddit conversations are data, and they are among the vast pool of material being fed into the L.L.M.s. to develop them.

u/IomKg [+1] (a day later)

We lack the simple sighting of spectral data of an extra-solar, liquid water bearing planet that has an atmosphere actively supporting plant-like life -- something that we should theoretically

You are implying that alien life should resemble ours, which we have no real reason to assume..

And in the end, it all points towards at least humans early if not first

Either that or we don’t know what we are looking for, or there are one of a very large set of reasons we would not be observing anything All we have is a single data sample, we just don’t know.

Essentially there is any number of alternative explanation for our observation and I haven’t heard you really give any argument to why the conclusion that we are first is more likely than all the other explanations. At the moment they are all practically as reasonable.

No, that's not how that sort of reasoning works. We can't assume invisible unicorns wearing shoes

I am not saying we need to assume anything, I am saying we can’t estimate the probabilities even because there is so much we do not know

If someone gave you a dice with an unknown number of sides, and you rolled it once and got some number, and he told you he is going to roll the dice an unknown number of times more, and he asked you what “place”(assuming ordering from highest to lowest) you are going to get, do you think it is reasonable to assume “first place” is the correct answer just because you haven’t seen him roll the dice during the time you were observing him?

That humans first is assumed in this story then isn't actually as "bad" as where it has any aliens ever appearing independently at all.

If no aliens were encountered for a few millions\billions of years then maybe a “human first” would not be too odd, as it would be consistent, and could be considered one conclusion from our current observation, and thus rational. But the story never really give any such indication but explicitly through things that happen in the epilogue. Which seems like a hard sell for me...

Choosing one thing that is unique in such as way as being first to develop AI of all sentient species ever is the opposite of random. It could be the story of the planet Blursg, in the Large Magellanic Cloud, and his people being first to seed AI and it would read out the same, given those assumptions the author made about AI. That's the point of the story.

Rethinking my stance about this I think it is correct that if an author establishes in a story about humanity that it is the first civilization, and he uses that consistently it can make literary sense, yes.

u/IomKg [+1] (3 minutes later)

i cant get the spoiler tag to cover the paragraphs covering the end, help?

u/696e6372656469626c65 [+1] I think, therefore I am pretentious. (5 hours later)

I find it interesting that while the OP refers to CelestAI as an "it", most of the comments here on the thread refer to said AI as a "she". I think I'm detecting some anthromorphization here.

u/IomKg [+1] (13 hours later)

actually i was really unsure how i should refer to CelestAI, in the end i went with "it" simply because i assumed that would be the more common way people would refer to it, guess i was wrong :)

but its too late to change now :P

u/None [+1] (8 hours later)

[deleted]

u/IomKg [+1] (13 hours later)

the fact we cannot predict it exactly doesnt mean we cant have any idea regarding what would happen.

what you say could just as well be said about any element in any story as we cannot realistically predict the infinite chain reaction that every action mentioned would generate thus why should we even try to be rational?

in other word its not really constructive.

u/None [+2] (14 hours later)

[deleted]

u/IomKg [+1] (14 hours later)

i am not arguing there should only be one possible way for it to happen, im saying that with the information shown to us about the specific way the author imagined the universe to work the end is not consistent

---