How big is the stack? Too often large data will blow the top and destroy adjacent stacks in multi-thread environments. Are memory barrier fences used to check against overflow?
I think it is supposed to be stacks in the general sense of the data structure, not the literal `sp` register. I imagine they could be arbitrarily sized up to physical limits if you do some mmap magic. I'm still a bit fuzzy about how you could make useful programs with that, but it seems interesting.
If I am reading it correctly, the compiler creates N bump allocators per function (or possibly globally) - where N is (I'm guessing at this point) determined by liveness or similar.
Goose looks familiar like C or Rust, and is built on one idea: there is no heap
So it looks like it restricts the memory management to 100% scope based. I expect that makes a lot of designs for programs not translate to it as well as they fit in Rust or Java (for example). There are a bunch more constraining design choices they list further down:
- Nothing ever moves
- ... A string, an array of strings, a record with variable-size fields and an array of those records are each one contiguous block with no pointer in it
I'll have to look a bit deeper to decide if it's feasible to write many things in this language.
Say you wanted to make a Node.js framework with callbacks that react to an event. The callback might be a closure that captured some of its surrounding variables. At that point, any of the captured variables are not trivially stack-allocated.
You might be able to do something similar to what Rust does with moving though.
You could predefine your event handlers within your context, then call 'enter framework' and pass your event handlers as arguments. this is continuation passing style
Typically you would construct this state in statically allocated memory. A lot of systems work this way. Dynamic heap allocation isn’t the only alternative to stack allocation.
Say you wanted a map of some key to growable arrays (or maps), where the number of keys isn't known until runtime. It I understand correctly, you can't really do that because the number of growable stacks nees to be known at compile time.
Might as well go back to ye olden days and put 100% of your memory in a giant preallocated block and instead of stack locals you just use the block. No allocation cost at runtime, cheese benchmarks by making the super arena.
Might as well go back to ye olden days and put 100% of your memory in a giant preallocated block
I do that for the Playdate: statically allocate most of the large structures and tables in global variables, and don't call malloc or free after initialization is done.
I also put some things on the stack, not so much because I need to dynamically allocate or free something, often it's because I get lower access latency to the stack compared to main memory (because stack lives inside ARM's tightly coupled memory).
Dynamic memory management is a performance enhancement. It lets you more efficiently utilize memory vs static allocation at the cost of, well, lots of types of bugs.
You can always preallocate MAX_NUM_ELEMENTS * SIZE_OF_OBJECT for all your arrays. If someone tries to send you more objects then you have buffer space for you just reject the request.
Latency sensitive programs already do this, since memory allocation is rarely deterministic (although it can be in .NET and other similar languages). Likewise a bunch of destructors going off when an object is freed in C++ also takes a, practically, non-deterministic amount of time. (Really well profiles and controlled programs can make this deterministic if allocation patterns are always identical, but that is rarely the case.)
Of course having fixed sized buffers means you have to have protocols that are aware of size limitations. Most protocols now days assume infinite memory. Everything just fails, badly, when memory does run out.
After having worked in embedded for awhile I grew to deeply appreciate planning around memory limits. Really everyone should be doing it but almost nobody is.
The goose language has dynamic allocation via builtin types, like most systems for the past 40 years (Folks were using dynamic allocation on systems back when 640k was a lot of memory). Most embedded systems these days far exceed the capabilities of 40 year old desk top computers and also use extensive dynamic allocation.
The distinction for goose is that it has fixed locations where the free must occur (function return, effectively), not that it doesn't have dynamic allocation (because it does have dynamic allocation)
People working on cortex m series chips still static alloc though. :D
Them and game programmers. Also the HFT people from my understanding.
The latter two are due to latency concerns.
Still though, it is something that I think more engineers should try out once or twice. Thinking about how buffers actually need to be is a useful exercise.
I am researching a similar idea but which allowed moves if the compiler was able to fix the resulting structure, but mine will probably stay a small side project for a long time.
If it's just strict aliasing then you can't be as fast as C++ in all cases. So you have to either provide an escape hatch like Rust (making the memory safety claim only mostly true), or accept worse performance in some cases (making the performance claim only mostly true).
So much Claude text. I'm sure this is great (I did a bunch of stuff with/to aardappel's lobster, years ago, and it was pretty tidy, and quite easy to work with), but: Claude's writing makes my brain melt.
Yeah it's very clear at the bottom that it's created by AI. I think this is an interesting case where someone with some great ideas they never got to can actually implement them. I personally hate the Claude style, but just looking at the samples was enough to get a feel for the language.
Lest it seem like I'm patting myself on the back for my ability to detect Claude's writing style, as if this instance would be evidence of any particular skill in that department: I did skim the README enough to note that bit.
116% would be 2x which will break the laws of physics. 16% is possible if you're not allocating heap memory, which I believe is the main selling point here.
I genuinely wonder how this style minimized the loss function or got the most upvotes in RLHF and yet is so universally hated that it gets flagged to death almost every time, and similar to Reddit. If I were to describe it, it's "snappy" and information-dense, without fillers. I dislike it too of course.
This style is deliberately trained by the frontier providers, not something that occurs accidentally. It is a manipulative style that is extremely effective against the general population. It utilises countless dopamine-inducing techniques used in clickbait headlines and Youtube thumbnails, and barrages the user with a wall of text that obfuscates everything it attempts to say, which is extremely useful for giving the appearance of intelligence; when you use a lot of sophisticated language and technical jargon, people won't understand you, but rather than assuming that you're stupid for writing something incomprehensible, they will instead give you credit and assume they can't comprehend it because it's too advanced for them, even if actually is incoherent.
It is only flagged on HN because it's been made against the rules, giving the minority who hate it the power to retaliate. It was only some months ago I was routinely getting downvoted every time I pointed out obvious bot accounts spamming a post per minute in blatant LLM-speak. Even now, LLM articles are still allowed and people upvote them to the top all the time.
information-dense
Err, no. It absolutely is not. You could say that it's dense in technical language, but the style has mastered the art of saying a lot without saying anything at all.
Your comment sparked an idea - it’s like the bike shedding of written text. If you write clean, concise English, it’s easy to parse and then inject your thoughts. If you instead (like an llm) throw a human a large wall of jargon heavy text, they’ll just give up and agree with you.
Nim defaults to this kind of stack management and value semantics, except the `ref` and `ptr` trapdoors are there whenever you need them. So like this, Nim requires no memory annotations or semantics for good, safe default behavior.
Goose is a straightjacket by comparison. I've never enjoyed languages that plant a flag on one mechanism and force users to adapt.
I've never enjoyed languages that plant a flag on one mechanism and force users to adapt.
Well, the hardware designers have chosen one (or at most a small number of) mechanism[s] and implemented in silicon. The farther you diverge from their implementations, in terms of abstractions, language features, and the like, the more you will pay in performance. Your choice.
It appears they got around that by making expanding and contracting data arrays builtin types the compiler impls. And it does this by creating what they call a "data stack" for each thing that needs to grow. To me it reads like "we have a builtin type that hides the heap allocation and does the free at the scope exit ", which is fine I guess as it keeps the nature of the language in tying lifetimes strictly to function scopes.
I just can't understand why you'd let an LLM write your README. The code, the implementation? Sure! That's the purpose of a coding agent.
The README, though? That's the first thing I read about your project. Anybody familiar with LLMs is going to pick up that one wrote it in an instant. Surely you understand your own project well enough to write it in your own words, right? If not, what's even the purpose of the project in the first place?
Yeah I hate the Claude writing style. It would be better with a cut down README. I suppose the point is that this was a latent idea that they got out there using these new tools, and that this would probably have not been done otherwise.
Well it's their original idea and it's based on the Lobster compiler they wrote, so it isn't just a random person doing it. I'd say it's more like an expert language designer using a tool to explore an idea they otherwise wouldn't have time for.
What does Goose change about memory management ?
A lot - title could probably use editing. The language has no heap, only stack memory, so the only deallocation is returning from a call stack frame.
How big is the stack? Too often large data will blow the top and destroy adjacent stacks in multi-thread environments. Are memory barrier fences used to check against overflow?
I think it is supposed to be stacks in the general sense of the data structure, not the literal `sp` register. I imagine they could be arbitrarily sized up to physical limits if you do some mmap magic. I'm still a bit fuzzy about how you could make useful programs with that, but it seems interesting.
Stack allocations will be lost when the enclosing call exits.
How does this help for long term values ?
If I am reading it correctly, the compiler creates N bump allocators per function (or possibly globally) - where N is (I'm guessing at this point) determined by liveness or similar.
So it looks like it restricts the memory management to 100% scope based. I expect that makes a lot of designs for programs not translate to it as well as they fit in Rust or Java (for example). There are a bunch more constraining design choices they list further down:
I'll have to look a bit deeper to decide if it's feasible to write many things in this language.
What kinds of things do you think might not translate well?
Not the OP, but I'm thinking about like closures?
Say you wanted to make a Node.js framework with callbacks that react to an event. The callback might be a closure that captured some of its surrounding variables. At that point, any of the captured variables are not trivially stack-allocated.
You might be able to do something similar to what Rust does with moving though.
You could predefine your event handlers within your context, then call 'enter framework' and pass your event handlers as arguments. this is continuation passing style
it is, but more specifically it’s an eliminator for a coinductive step.
Nah we did something similar on Microsoft Band. I wrote about how - https://meanderingthoughts.hashnode.dev/cooperative-multitas...
The tldr is you had to preallocate a struct with anything you wanted captured ahead of time.
Typically you would construct this state in statically allocated memory. A lot of systems work this way. Dynamic heap allocation isn’t the only alternative to stack allocation.
Say you wanted a map of some key to growable arrays (or maps), where the number of keys isn't known until runtime. It I understand correctly, you can't really do that because the number of growable stacks nees to be known at compile time.
You could write everything if you refactor to continuation passing style
Might as well go back to ye olden days and put 100% of your memory in a giant preallocated block and instead of stack locals you just use the block. No allocation cost at runtime, cheese benchmarks by making the super arena.
I do that for the Playdate: statically allocate most of the large structures and tables in global variables, and don't call malloc or free after initialization is done.
I also put some things on the stack, not so much because I need to dynamically allocate or free something, often it's because I get lower access latency to the stack compared to main memory (because stack lives inside ARM's tightly coupled memory).
CPS is an intermediate representation, humans shouldn’t have to tolerate it.
Embedded people are well used to this.
Dynamic memory management is a performance enhancement. It lets you more efficiently utilize memory vs static allocation at the cost of, well, lots of types of bugs.
You can always preallocate MAX_NUM_ELEMENTS * SIZE_OF_OBJECT for all your arrays. If someone tries to send you more objects then you have buffer space for you just reject the request.
Latency sensitive programs already do this, since memory allocation is rarely deterministic (although it can be in .NET and other similar languages). Likewise a bunch of destructors going off when an object is freed in C++ also takes a, practically, non-deterministic amount of time. (Really well profiles and controlled programs can make this deterministic if allocation patterns are always identical, but that is rarely the case.)
Of course having fixed sized buffers means you have to have protocols that are aware of size limitations. Most protocols now days assume infinite memory. Everything just fails, badly, when memory does run out.
After having worked in embedded for awhile I grew to deeply appreciate planning around memory limits. Really everyone should be doing it but almost nobody is.
The goose language has dynamic allocation via builtin types, like most systems for the past 40 years (Folks were using dynamic allocation on systems back when 640k was a lot of memory). Most embedded systems these days far exceed the capabilities of 40 year old desk top computers and also use extensive dynamic allocation.
The distinction for goose is that it has fixed locations where the free must occur (function return, effectively), not that it doesn't have dynamic allocation (because it does have dynamic allocation)
Yeah embedded means a lot of things.
People working on cortex m series chips still static alloc though. :D
Them and game programmers. Also the HFT people from my understanding.
The latter two are due to latency concerns.
Still though, it is something that I think more engineers should try out once or twice. Thinking about how buffers actually need to be is a useful exercise.
u end up with stack based heap likes anyways
I am researching a similar idea but which allowed moves if the compiler was able to fix the resulting structure, but mine will probably stay a small side project for a long time.
Faster than C++ is always a head-turner. Curious what "magic" enables that with memory safety.
Usually strict aliasing.
If it's just strict aliasing then you can't be as fast as C++ in all cases. So you have to either provide an escape hatch like Rust (making the memory safety claim only mostly true), or accept worse performance in some cases (making the performance claim only mostly true).
So much Claude text. I'm sure this is great (I did a bunch of stuff with/to aardappel's lobster, years ago, and it was pretty tidy, and quite easy to work with), but: Claude's writing makes my brain melt.
It's a no from me. I'm sorry.
Yeah it's very clear at the bottom that it's created by AI. I think this is an interesting case where someone with some great ideas they never got to can actually implement them. I personally hate the Claude style, but just looking at the samples was enough to get a feel for the language.
Lest it seem like I'm patting myself on the back for my ability to detect Claude's writing style, as if this instance would be evidence of any particular skill in that department: I did skim the README enough to note that bit.
I'm always fuzzy on this, so 116% faster or 16% faster? The benchmarks suggest 16%.
116% would be 2x which will break the laws of physics. 16% is possible if you're not allocating heap memory, which I believe is the main selling point here.
Evergreen: https://randomascii.wordpress.com/2018/02/04/what-we-talk-ab...
(Feels like it should probably say "1.16x as fast" or "1.16x the throughput" - or something like that.)
All-stack-no-heap
Isn't this kind of the point of Java's Project Valhalla or am I just confused???
Project valhalla allows more things to be on the stack but doesn't get rid of the heap.
Great launch!
I was thinking about making a language with same thoughts: Safer than C++ and faster than rust (and a 3rd thing: optimized for AI)
and you actually did it for me. Hooray!
Just the AI language optimization thing is missing..
The AI optimization is to put it in distribution. So... make it look like python or js?
a flagged-dead comment in this thread:
https://news.ycombinator.com/item?id=49749113
I genuinely wonder how this style minimized the loss function or got the most upvotes in RLHF and yet is so universally hated that it gets flagged to death almost every time, and similar to Reddit. If I were to describe it, it's "snappy" and information-dense, without fillers. I dislike it too of course.
My read: it's not the style per se, it's the association. People are just sick of AI slop and react badly to anything that smells like it.
The dead summary itself was reasonably informative, I'd say.
This style is deliberately trained by the frontier providers, not something that occurs accidentally. It is a manipulative style that is extremely effective against the general population. It utilises countless dopamine-inducing techniques used in clickbait headlines and Youtube thumbnails, and barrages the user with a wall of text that obfuscates everything it attempts to say, which is extremely useful for giving the appearance of intelligence; when you use a lot of sophisticated language and technical jargon, people won't understand you, but rather than assuming that you're stupid for writing something incomprehensible, they will instead give you credit and assume they can't comprehend it because it's too advanced for them, even if actually is incoherent.
It is only flagged on HN because it's been made against the rules, giving the minority who hate it the power to retaliate. It was only some months ago I was routinely getting downvoted every time I pointed out obvious bot accounts spamming a post per minute in blatant LLM-speak. Even now, LLM articles are still allowed and people upvote them to the top all the time.
Err, no. It absolutely is not. You could say that it's dense in technical language, but the style has mastered the art of saying a lot without saying anything at all.
Your comment sparked an idea - it’s like the bike shedding of written text. If you write clean, concise English, it’s easy to parse and then inject your thoughts. If you instead (like an llm) throw a human a large wall of jargon heavy text, they’ll just give up and agree with you.
i agree, it's essentially copywriting
Cluould be viewed like a fancy evoultion of CHICKEN (Cheney on the MTA) that used stack for everything.
Unfortunate language/protocol name reusal. Watch out for these.
https://www.se.com/ww/en/download/document/998-2095-18-12-19...
Nim defaults to this kind of stack management and value semantics, except the `ref` and `ptr` trapdoors are there whenever you need them. So like this, Nim requires no memory annotations or semantics for good, safe default behavior.
Goose is a straightjacket by comparison. I've never enjoyed languages that plant a flag on one mechanism and force users to adapt.
Well, the hardware designers have chosen one (or at most a small number of) mechanism[s] and implemented in silicon. The farther you diverge from their implementations, in terms of abstractions, language features, and the like, the more you will pay in performance. Your choice.
I suppose to be distinctive and have a "new idea", you sort of have to do that. Otherwise there's really no point in making a new language.
This is neat.
But the benchmarks are tiny, and it’s likely that Goose was tuned on them.
So, I think I would read this as: Goose has competitive performance to C and Rust and I’ll take them at their word that it’s as memory safe as Rust
So if nothing moves, you can't make an array that grows?
https://github.com/aardappel/goose/blob/master/samples/02_me...
It appears they got around that by making expanding and contracting data arrays builtin types the compiler impls. And it does this by creating what they call a "data stack" for each thing that needs to grow. To me it reads like "we have a builtin type that hides the heap allocation and does the free at the scope exit ", which is fine I guess as it keeps the nature of the language in tying lifetimes strictly to function scopes.
I just can't understand why you'd let an LLM write your README. The code, the implementation? Sure! That's the purpose of a coding agent.
The README, though? That's the first thing I read about your project. Anybody familiar with LLMs is going to pick up that one wrote it in an instant. Surely you understand your own project well enough to write it in your own words, right? If not, what's even the purpose of the project in the first place?
Yeah I hate the Claude writing style. It would be better with a cut down README. I suppose the point is that this was a latent idea that they got out there using these new tools, and that this would probably have not been done otherwise.
Am I the only one with a deep aversion to the keyword "let"?
This is an interesting idea. It’s a pity it was implemented by LLM instead of explored by someone with curiosity.
Well it's their original idea and it's based on the Lobster compiler they wrote, so it isn't just a random person doing it. I'd say it's more like an expert language designer using a tool to explore an idea they otherwise wouldn't have time for.
Some of us may never adjust to a world without sweat equity.