Hacker News

Top stories

Live mirror
30 storiesupdated just nowView source snapshot
  1. Ember-1 (fireworks.ai)
    38comments
  2. In an $80 motel room, a discovery to shed light on the origins of life (nytimes.com)
    46comments
  3. Writing Efficient C++ Code (asawicki.info)
    45comments
  4. Replacing the old battery on rechargeable bike lights (jvns.ca)
    46comments
  5. Show HN: TinyAIArena watch AI agents battle it out (tinyaiarena.com)
    31comments
  6. The Normalization of Inexplicable Failures (ihatethefuture.com)
    62comments
  7. There are no "rogue" AI agents (eoinhiggins.substack.com)
    144comments
  8. SNL Weekend Update: Anthropic CEO Dario Amodei on A.I.'S Threat to Humanity [video] (youtube.com)
    30comments
  9. John Coltrane Centenary's – Impulse Records Release the Legendary Tiberi Tapes (jazzwise.com)
    4comments
  10. Walgit: A Git server that is one binary in front of an object store (github.com/rgodha24)
    6comments
  11. Fragment of oldest known peace treaty found in Turkey (livescience.com)
    3comments
  12. Oral history of John Chowning, inventor of FM synthesis [video] (youtube.com)
    —discuss
  13. Flip Fluid on Flip Dots (mitxela.com)
    21comments
  14. Fakecloud: Local AWS cloud emulator for integration tests (fakecloud.dev)
    40comments
  15. The Cartesian Hand: In-Hand Manipulation with All-Linear Fingers (generalroboticslab.com)
    4comments
  16. Faster prompt lookup drafting in llama.cpp (jadidbourbaki.github.io)
    2comments
  17. Show HN: Building a Markdown editor for Mac, iOS and web (markdown.beauty)
    24comments
  18. Video CDs Break Windows Explorer (clydesnotes.blogspot.com)
    6comments
  19. PostmarketOS is rebranding as Nura (nura.eco)
    15comments
  20. Show HN: Trail – new kind of logic game (franzai.com)
    6comments
  21. Show HN: A CC0 museum of retro 3D tricks you can paste into a page (3d-retro.com)
    10comments
  22. Ten lines of code that changed my world (pixelambacht.nl)
    22comments
  23. Go Concurrency Distilled (antonz.org)
    145comments
  24. C's Flexible Integer Sizes Were Not a Design Mistake (pikuma.com)
    67comments
  25. Finally, A True Blue Rose Exists (sciencenews.org)
    31comments
  26. PipePipe: NewPipe hard fork implementing SponsorBlock (github.com/infinityloop1308)
    262comments
  27. Rusty thoughts on "Parse, don't validate" (thegreenplace.net)
    24comments
  28. Reading’s Bayeux Tapestry (diamondgeezer.blogspot.com)
    6comments
  29. Show HN: Reladraw – A diagram language where you decide where to place things (github.com/reladraw)
    105comments
  30. The Greatest Pun in JavaScript (shukla.io)
    4comments

Rusty thoughts on "Parse, don't validate"

57 pointsby 10h agoeli.thegreenplace.net
24 comments
4h agoHN ↗

Good article on Rust's strengths with types. If you like this, you may be curious how you might build your own parse capabilities. I like the Rust crates Winnow and Nom, and also the Rust traits From and Into.

4h agoHN ↗

Nonempty type wrappers are a stark reminder that we are missing out on refinement types.

4h agoHN ↗

I was thinking about its implementation, too, and ended up deciding that it was probably a performance optimization.

`Vec<T>` stores all data on the heap, so getting anything out of it involves a pointer deref and possibly also an array bounds check. This `NonEmpty<T>` type keeps the first element of the list in a location that supports some low-level optimization that might make a significant difference in situations where accessing the first element is much more common than accessing subsequent elements.

4h agoHN ↗

Rust has a proposal for this, in the form of pattern types. It's on nightly, but quite far from being ready for prime time.

4h agoHN ↗

This is great. Alexis King actually stated that, had she known how popular “Parse, Don’t Validate” had been, she would have written it in a language more widely used than Haskell.

3h agoHN ↗

With its rich type system, Haskell is the perfect language to demonstrate the dictum.

3h agoHN ↗

With its crap type system Python might be even better, in a strange way.

Haskell's strong, static, non-reflective type system tends to make "parse, don't validate" produce code that also looks nicer. Which is great. So great that it steals a bit of the main message's valor.

In Python, though, it's really easy to just let your data be a dynamically typed list of dicts forever. So easy that parsing into something more strongly typed looks like a whole lot of extra effort. Upon looking at that sort of thing many a working Python programmer, myself included, hears the voice of GvR murmuring disparaging things about "academic" programmers down in the pit of their brain.

Which creates an opportunity to demonstrate all the ways the (arguably) more Pythonic way is actually a royal PITA when you try to make your code robust. Handling and reporting data validity errors gets scattered all over the code, which makes it annoying to maintain. Unit test suites get bloated because it's not obvious what inputs a function should be able to handle. Comments and docstrings to help keep track of this stuff begin to proliferate.

1h agoHN ↗

I write Python. While I read the original post and nodded along, nothing changed about how I worked until I started playing with Rust. Seeing a strongly typed language force the constraints and stop stupid errors was when the rationale finally clicked for me. Knowing that this function will never blow-up in an unexpected way changes how you think about problems.

You otherwise get accustomed to some wishy-washy blobs of data that get passed around and find it normal. Maybe you include some ad hoc guard rails here and there, which catches the egregious errors, but there is always some lingering uncertainty. Some string that should have been an int, missing key here, the object which never had the validation check, etc

It is like unit-testing - more-up front work, but I could never go back to a world where I did not get these automated assurances. Sadly, I am a grug-brain which could only feel the lesson from personal experience.

1h agoHN ↗

I’m a big fan of “Parse dont validate” and I feel like injecting that approach into python has so many benefits.

there’s all the code correctness stuff that we all love. But the biggest benefit is making it so much easier to maintain other parts of the stack.

Following a stack trace into somebody else’s functions and you see that the args are completely untyped or just a bunch of ‘dict[str, Any]’ is the worst. But if you see those inputs as more narrowly typed data classes it makes it so much easier to grok what the function is supposed to do

41m agoHN ↗

I'm the author of typedload. For json-type of data that is loaded in dictionaries you can do A LOT of validation at runtime automatically if you define the types.

3h agoHN ↗

Another take, from the primary example: This is what `unwrap()` is for. I understand that the author is looking at this from a correctness and safety(?) perspective. For practical purposes, I would unwrap here. If it's less trivial than the example, unwrap with a comment explaining why it's fine.

Another angle: Unfortunately, the `first()` method being fallible here is just an issue of using an imperfect method/datatype here. This is where the author gets in to a non-empty-vec custom type. Then you are balancing using a more correct type that takes custom wiring vs a std lib thing everyone understands and takes no setup. I would lean towards this setup if I were using this non_empty_vec.first() unwrap pattern a number of times in the code base; then the setup would be worth it, at least for my own code bases. If I were exposing this in a lib others would use, I would keep the standard Vec so as to be more transparent for others.

In both views: "This is what unwrap is for" does it for me in all cases I've encountered to date. Maybe for aerospace or safety critical systems, I would have a different take.

A third take: I notice this trend in the rust community. It's not my cup of tea. Keep things simple, easy to maintain, and don't let "correctness" get in the way. In this example, I don't think it gets in the way, but I have seen this mindset lead to it getting in the way, especially in embedded, where mapping the Owernership model to hardware ends up in messy patterns and surprising assertions about embedded-101 concepts like DMA being "unsolved", "no good way", "difficult" etc.

Rust provides tools to make sure specific logic is correct if it passes the compiler. People sometimes go overboard and assume you have to type-maxx your code, regardless of complexity added by doing so.

3h agoHN ↗

...I'm so happy that I write in untyped languages so I don't have to deal with any of this.

3h agoHN ↗

But you kind of do.

For a untype language like js array, since it can be empty, you have to either always check the length, the item returned, or have a precondition to know the array is not empty.

All three of those cases is either code or context you’re holding in your head.

All that stuff is equivalent to a type system

2h agoHN ↗

Congratulations on having to implement your checks at runtime, but stupid, repetitive, boring work is kind of why we invented compilers.

57m agoHN ↗

I lamented patterns of what I consider over-use of the type system in Rust (This applies to other languages as well). There is a balance: I prefer this over-use/strict use (e.g. implementing a custom non-empty Vec type) over not using typing at all, but a big margin!

Balance in all things?

3h agoHN ↗

Not sure that type is good advice:

    pub struct NonEmpty<T> {
        pub head: T,
        pub tail: Vec<T>,
    }

You'd have to manually implement the traits to support the ergonomics of slices and iteration and costly reallocation if you need to pass ownership as a Vec:

I'd expect:

    pub struct NonEmpty<T> {
        v: Vec<T>,
    }

The constructor would enforce the invariant and then you'd impl Deref and DerefMut for [T] to gain normal len/is_empty/indexing/iteration, passing as &[T] to other funcs and mutating values (which can't break the invariant).

To mutate length while preserving the invariant it's dealers choice e.g.

- add .into_vec() for unwrap/mutate/rewrap

- add invariant preserving mutators of your choice

2h agoHN ↗

Which deref must I use to get most of the existing interface sans `retain()`?

1h agoHN ↗

Deref/DerefMut enables implicit type coercion rather than exposing an interface. You can choose the target type and immutable/mutable but not parts of the target type.

You can use all the slice reference methods (that do not require ownership) with:

    impl<T> Deref for NonEmpty<T> {
        type Target = [T];

        fn deref(&self) -> &Self::Target {
            &self.v
        }
    }

    impl<T> DerefMut for NonEmpty<T> {
        fn deref_mut(&mut self) -> &mut [T] {
            &mut self.v
        }
    }

https://doc.rust-lang.org/std/primitive.slice.html

If you DerefMut to a Vec then you won't be able to preserve the invariant.

If you want control over methods to expose then you need wrapper methods for those you want. If you want to expose some of the traits the inner type implements then there are likely derive macros available e.g. with derive_more you could expose just indexing as:

    #[derive(Index, IndexMut)]
    struct MyVec(Vec<i32>);
1h agoHN ↗

Not sure switching between languages makes for a compelling argument:

"Look how easy it is to accidentally bypass the invariant of a rust newtype by transliterating the data shape into Haskell and deriving a new type". Uh, ok.

If comparing the "risk of accident" between a newtype wrapper whose only role is enforcing the the invariant versus manually reimplementing vector and iterator semantics to use a different layout... I'd say the newtype wins that.

It would be good advice to keep a newtype that enforces an invariant as a single purpose primitive type. A building block and not a place to add other features.

There might be times I'd prefer structural enforcement e.g. something serialisation related. Converting into a non-rust format is what they are doing in their "accident"!

1h agoHN ↗

I prefer a slightly more general rule: Make Illegal States Unrepresentable. The "Parse, don't validate" rule is a special case of MISU.

What's the difference? MISU applies even when there's no parsing-like transformation happening. For example, if you have a variable that represents the current state of a network connection, and let's say it can be Disconnected, or Connected to some IP address (this is an oversimplification).

Then one way to do it would be

  struct {
    connected: bool,
    peer_ip: int32
  }

The trouble is that this allows us to represent an illegal/meaningless state: we're disconnected but there's still some junk old peer_ip hanging in there. Even worse, we might have written

  struct {
    connected: bool,
    peer_ip: Option<int32>
  }

Now we could have connected = true but peer_ip = None.

The solution is to use a sum type:

  type connection = 
     Disconnected
   | Connected of int32

(sorry for using made-up syntax; I hope it's clear to anyone familiar with Rust.)

"Make Illegal States Unrepresentable" applies throughout your program, at every interface between modules or functions in the program, including but not limited to parsing input.

47m agoHN ↗

In rust this would simply be

   enum ConnectionStste {
        Connected(u32),
        Disconnected
    }
18m agoHN ↗

or

   struct Connected { peer_ip: u32 }
   struct Disconnected;
   struct Connection<State> { state: State }
3m agoHN ↗

ie "Expect type validation from a parser."

This is delegating to a library or moving the problem around.