Crafting a good prompt is what differentiates an expert's output from a beginner's. Of course you need good constraints too. But those good constraints are created precisely through good prompts.

This is an amazing article. The problems it describes are exactly what we found when building a production system that used LLMs to (most of the time) produce reliable results. Extensive tests are necessary, and stakeholders have no idea how their suggestions fail in production. They just see the handful of times they tried something and had it work, not the long tail of cursed results. (“How hard can it be? Why don’t you just…”)

We didn’t get to the point of self-built prompts, as the article suggests, but it’s an intriguing idea.

They can want anything they like, if customers want to use agents and they don't provide APIs they will lose out.

You prevent abuse of the API by models the same way you prevent abuse by humans: you have server side checks.

have a tool call that gives the customer a discount, without it getting used when it shouldn’t?

without comment on the rest of your post, a lot of discounts are just dark patterns, buy 2 for $10 or one for $6 is one of the oldest dark patterns around, and a lot of people fall for it, when they really didn't need that much whatever it was they were buying. In fact there are laws outlawing these practices coming into force for various types of products (alcohol, sugar) .

Other types of discounts like loyalty are more vague personally I think are a minor dark pattern. Negotiated discount on bulk purchases b2b are a different class, but it could be argued that you should only get the discount that the seller saves on logistics.

I'm just pointing this out because its very normalized, 'signup and get 3 months free', 'buy 2 get one free' etc, people dont bat an eyelid at, the same as advertising , but they are all just the earliest versions of dopamine hacking and dark patterns that consensus now is coming around to say probably isn't the best thing when scaled beyond a simple one to one interaction. Why are snapchat dopamine hacking with streaks but your local coffee shop stamping your card for your 10th coffee free different?

from the company perspective how do they, for example, have a tool call that gives the customer a discount, without it getting used when it shouldn’t?

I might be misunderstanding your point but I think you're in agreement with the gp, as in they are arguing for a locked down interface behind which the AI sits, narrowed to use within whatever particular operations the user can access, as opposed to a wide open AI interface with access to any operation (in principle).

So in your example there's no way to ask the AI for any old discount because the interface for doing so is locked down to just the discounts you can access, although the AI may be able to apply a discount you didn't ask for dynamically, if allowed, to give the customer a better experience.

Of course both versions can be implemented securely, it's just probably in general a smaller attack surface if you provide a tighter interface first.

how do they, for example, have a tool call that gives the customer a discount, without it getting used when it shouldn’t?

One possibility is that the company and the user both have AI agents, and that these are able to negotiate with each other. Then I talk to my local customized agent, which knows my preferences, and it explains the required context if the company’s agent does something that won’t make sense to me. But the company’s agent is still privy to the details required to offer discounts, say.

Companies will do whatever creates the most lock-in for the user and generates the most profit for themselves.

Ask yourself what's in their best interest as a business? That's probably what they'll do.

All of those browser and computer use agents (available from OpenAI, Anthropic, Meta, and X.ai already) represent an interoperability workaround already.

provide the user/customer with an interface to an AI that can do things for the user

This is exactly what MCP is. But the reality is it will likely be about as popular as browser extensions and most normies will avoid.

Simplifying UX at the cost of personal control is inevitable because not everyone wants to think about tool selection and coordination. But maybe we can angle the future toward "tool bundles" that interoperate well or (if we are dreaming) mandate models remain accessible by any harness, which none of the players want but would be best for users and ecosystem development.

IMO it's a fundamental tension. AI does not fit well inside a product. It works best outside of it. But that means the product - not its functionality, but the business around it - becomes irrelevant. Which none of the product vendors want.

Also the motivation behind CLIs and SDKs, right? And the way agents write Python regularly, these days, I am hopeful what can be offloaded to deterministic systems will be.

So I actually struggled with this recently (2 months ago). My "vision" was to have customers BYO agent and provide a secure MCP with an in-app (forced) approval gate for sensitive operations. But everytime I described to one of my clients, how they could bring their own ChatGPT, Claude, Gemini, w/e, I got blank stares.

So now I'm building (already have done so mostly) cheap ZDR models into the application. I'm trying to build them in a way that I would want to use them, so a less "I'll be your AI today!" help bubble, but who's kidding... At least they aren't just customer pacifiers that use the help system, but actually can act on behalf of the user.

After which I'll end up building the MCP, but for power users, hopefully leveraging much of the same work. It's worth noting that MCPs have way too much friction still. It's gotten a lot easier recently (very recently) to bring your own custom MCP with Claude, but ChatGPT is more work, and restrictive (specific account types or submitted MCP apps) for the end user and I don't even know what Google wants at this point... It's a bit messy. IMO they are all dropping the ball (except Claude, which has taken the best approach).

I think we might be in this interim state that requires us to build ALL of the UIs, to address ALL of the users (Human responsively and Clankers), and it's a bit painful. If I was "the user" myself, then I would just want a CLI w/ OAUTH with clear documentation, that an agent could use (how is this worse than MCP?). But I'm not my own customer. That being said... I probably WILL build this version to scratch my own itch and bet on the future.

Imagine if companies exposed access to their services via a discoverable uniform interface.

Each company would determine the range of actions a customer could perform in response to their request and current circumstances. A representation of state could be transferred between the company's server and whatever client the customer chooses to use. The customer would be free to interact with any of these services in a way that suited them via their chosen client.

Either that or we could kludge something together with MCP.

100% agree, and I predict this is where the actual nearest big battle in our industry will occur.

AI subsumes products. Users want that. Vendors, do not. AI does not work well when shackled within confines of a product - it works better from outside, where it can treat slices of products as tools, and mash them together into ad-hoc solutions. Alas, products is how our industry makes money. Take arbitrary slice of problem space, slap a trademark on it, and shill to people (or VCs for funding). This disconnect makes AI an existential threat to a good chunk of software industry, and you can bet that companies (possibly including your own employers) won't go gently into the night.

MCPs are an aberration, the early stage of "AI adoption" where no one knew what they're doing but they knew they "have to do something with AI!". This age is now ending, and I expect the tensions will go high, as most vendors will get desperate to avoid their products getting obsoleted overnight.

Not sure if companies would ever want this. They rely on their dark patterns and intentional manipulations to get customers to do what they want.

[delayed]

This is like asking LG to make TVs where you can bring your own video input. It just doesn't make economic sense when they can extract so much money by owning the glass.

TL;DR: you need to do... essentially supervised learning on your prompts? I mean, if I wanted to do ML, I'd already have been doing it ten years ago.

the textual nature of prompts leads us to take the intentional stance towards systems which aren’t conscious, and thus miss the essential nature of their non-meaning

I see LLMs as being capable of making useful distinctions and having a rich action space. They are widely used because their operation is useful, and that can only happen when semantics work well in practice. But useful things that pay for themselves don't need our "essential nature" blessing, they already have persistence by mutual entanglement with us.

No idea how effective this is, but it sure looks a lot more like engineering than most of the 'prompt engineering' things I've seen in the past years. Kudos to the author for writing this concisely without aggrandizing his work.

Isn't the goal of the author to get reproducible behavior out of the agent though? I would thinking turning the temperature down would serve that production goal too.

It’s a talk, not an article.

Ohhh, i didn't realize there was more, from mobile it looks as if the article ends on the first image, and the images are like other articles (the first being a self link)

RLHF might help, you don't have to train a model from scratch.

Am I the only one who waited to the end for, and was disappointed not to see, a peek into these optimized prompts? I so want to take a peek into that abyss, even if that risks the abyss looking back at me. I'm curious just how twisted they get relative to the original, in what alien ways.