There’s a Reason that AI Talk at (Procure)Tech Events is All Talk and No Substance
A recent rant on LinkedIn noted that, at DPW (and other events), there was “Tons of talk about using AI, not so much about how to procure it, scope it, contract it, measure it, commercial agreements, which suppliers are best for which usage.”
Well, there’s a simple reason for that. And it goes as follows.
It’s the tech-du-jour. More specifically, the hype-du-jour. As a result you can forget any advice on how to:
–> Procure It
No one really knows who has what, or what it actually does, what it’s really worth, how to properly cost it, how to compare offerings and offers. So how can they tell you how to procure it?
–> Scope It
According to the hype it’s your new employee that does everything for every one, despite the fact that it hallucinates more times per second than an LSD junkie does in a lifetime. Without knowing what it does or where it goes, they can’t tell you how to scope it!
–> Contract It / Commercial Agreements
Even the vendors wrapping someone else’s model don’t really know what it does, what they can guarantee, what they can’t, or who is really liable (although courts are starting to make those decisions). So no one knows how to contract it.
–> Measure It
Whereas we have tried-and-true mathematically sound measures for traditional deep neural networks where you can get accuracy ranges with confidence ranges, when it comes to LLMs, no one has a f*ck1ng clue how to measure them. Random tests by random humans judged by random people with random definitions of accuracy is not a measure. And I’d have more confidence in a decision made by a Koala. At least it’s cute!
–> Assign It
If we can’t even measure what it does, do you think we can measure how well the vendor pushing it really understands it? Definitely NOT!
And that, in a nutshell, is why there’s a lot of hot air and nothing of actual substance in all the AI discussions. Which you should avoid anyway. Because you don’t want tech with a 6% success rate [McKinsey, MIT], which is half the general success rate of new tech installations (now that tech failure rates have reached an all time of 88% [Bain]).
You cannot export-control math!
A truly brilliant observation by Mr. Stephen Klein in a recent post on how China May Be The Only One With Mythos because they may have saved its responses (and thus figured out how to replicate it).
(Gen-) AI LLMs are just mega math models. Really big mega math models with probabilities being computed on top of probabilities being computed on top of probabilities in force-feedback loops that reinforce its computations (which could be brilliant deductions or hallucinations that equal the acid high of the most LSD addicted junkie on the planet).
And since the outputs are dependent on the equations that define the model and the training data, if someone can recreate the training data they can reverse engineer the equations from the output if they are sufficiently adept at mathematics, or at least a close proximity.
Which means that blocking off access to those who can evaluate and improve the model will not help if those you don’t want to have the model already have it.
But this isn’t a post about the blocking of AI models (because I personally think that’s great), but a post about what happens if you try to hide your capabilities behind “proprietary algorithms based on math” assuming that no on else can recreate it if you don’t talk about it and that the IP alone justifies an unreasonable price for your product or valuation for your company.
Every country has mathematical geniuses, and more than one can come up with the next iteration of a mathematical theory at about the same time. Maybe only one gets remembered (Newton vs. Leibniz), but it doesn’t mean they didn’t both invent the same concepts at about the same time (in calculus).
And the more you trump it up, the more it entices someone to tear it down. (And figure out how you built it.)
On its own, the math alone is not the advantage you think it is. There are a lot of free scientific papers with great math. In fact, more than enough to create your own Claude, DeepSeek, Gemini, Grok, etc. But most people can’t because it’s not just the equations, it’s the parameters, the training data, and the implementation.
With regards to the implementation, just because you have server racks that can do trillions of calculations per second, that doesn’t mean you can code inefficiently. For example, an average token output by Claude requires billions of operations, and an average question posed to Claude will require 1,000 to 10,000 tokens to answer, for 1 trillion to 1 quadrillion calculations for an output. A poorly designed model could require 10, 100, or 10,000 times that.
The same goes for classical machine learning or optimization algorithms. Good implementations will require billions of calculations. Bad, trillions to quadrillions to quintillions. Responses go from real-time to hours to days to the computations never end.
Then there is the training data. You can’t judge a model implementation unless you have good training sets that are representative of real-world problems. Optimizing for theoretical problems that don’t exist in the real world isn’t helpful and may, in fact, lead to a worse solution that not even testing it at all!
The real differentiator is the expertise both in the implementation of solutions based on math and deep knowledge of the domain the solution is for. That can’t be recreated by mathematical ability alone and requires experience. And that can be export controlled (and represents the real value).
What Data Do You Need For Successful Procurement?
In a comment to a recent post over on Linked in, Mr. Buckingham asked Do you think that data driven decisions are clearly the correct thing to do, but that they tend to maintain the status quo and can be restrictive to innovation?
I couldn’t leave this one alone and responded that:
“They only maintain the status quo IF the data collected and constraints created are limited to those that support the status quo … which, sadly, they usually are …
As the Sourcing Optimization Grand Master Paul Martyn recently pointed out — the real context never gets mentioned, stakeholders just add “necessary” constraints, costs, and weightings that they know will heavily favour the incumbent
And as the Sourcing Simplifier Garry Mansell regularly points out, organizations fail because they only include the best lagging indicators in board presentations to ensure they can keep doing the same old, same old
But if you instead collect [only] external data on the market, and not internal data that supports the status quo, the tendency will be to focus heavily on innovation and change.
Data is the way to go, but it has to be evenly distributed across internal and external so you can get the full picture.”
Seemed like this is something that should be elaborated on.
For example, let’s say Procurement is trying to gauge the effectiveness of their category sourcing in a key category. They could demonstrate this through internal or external metrics. Internally they could show the average price per unit decreased year-over-year by a significant percentage. Externally, they could retrieve the average market price for the primary products and show they are paying less. If they only ever use one of these metrics, and it stays good, as Mr. Buckingham notes, it maintains the status quo, even if it shouldn’t. In order to truly gauge effectiveness it has to, at the very least, measure both. Just reducing costs doesn’t mean you are getting the best price, and just beating the market doesn’t either. In some categories, market quotes / GPO rates / etc. are never the best price, which is dependent on what you actually need, who, and where, you are getting it from.
Digging deeper, if the specs are over engineered, you restrict to a suppliers in a certain geography, or only deal with incumbents, you’re not getting the best price. So you not only need to take internal and external measures and benchmarks, but ensure you are taking the right internal and external measures and benchmarks.
And then, if there are hidden costs (from increased risk, revised shipping / order lead times, quality reductions, etc.), take this into account as well. (Which is why you need strategic sourcing decision optimization with what-if scenarios, as we’ve been telling you for the past 20 years.)
Only when you identify the right data and collect the right data can you make the right decision, which might reinforce the status quo, might tell you do do something completely different, or tell you to split your bets and do both!
The reality is, you should always make data-driven decisions, but you can only do so if you are collecting the right data for your needs.
Software Acquisition Insider Tips 2026 Part VI
It’s been 17 years since SI published its first major series on generic insider tips back in 2009 where we gave you a lot of advice that more-or-less still stands today if you want to safely acquire software. In our preamble, we overviewed what those 11 pieces of advice were then, and summarized the 6 major difference that affect how you apply that advice today so you can continue to make the right decisions when acquiring software in the age of AI Hype and exaggerated I2O claims. In the last four parts we addressed the first eight pieces of advice and how they have evolved over the years. Today, we conclude.
Separate software from service
Seventeen years ago we wrote:
Many software products aren’t really software products at all. In other words, there is some incantation that has to be performed by the software vendor, in the form of services, configuration, or other magic, on a regular basis, to keep the software running. In this case, you haven’t bought software, you’ve bought software plus services. What’s even worse is that you’ve single-sourced it. If the vendor goes broke, or can’t deliver, you have no options.
This is still a reality with many products, and, even worse, in the age of AI-based “agentic” offerings, they only keep working if the vendor is constantly monitoring, maintaining, upgrading, retraining, correcting exceptions and errors behind the scenes that you don’t see, etc. There’s more hidden services than ever. That, combined with the escalating token costs, is why they have to sell you on “outcomes” because they can’t charge SaaS fees and survive. (And that’s why outcomes is a dirty word. Just remember freedom of speech means that they can say anything they want, even if it’s not true, but that you have the right to ascertain the truth and demand that word never be used again in their RFP responses! [As well as “AI”.])
Good software is very affordable today. If it’s too expensive, it’s either relying too much on experimental AI that doesn’t work, stuffed full of features you don’t need, or hiding services behind the scenes you don’t want to be paying for.
Monitor the market
This isn’t 20 years ago where there were very few price benchmarks beyond what you could collect from an RFP and a few general price ranges from your favourite consulting firm that were wider than an average football field, this is now where there are dozens of platforms that monitor SaaS spending and a few big consultancies that specialize in not only monitoring and pricing how low a vendor will go, but breaking apart their combined bids into individual SKUS because they have hundreds of data points to compare against as they have been collecting that data for years as they negotiated on behalf of their clients. If you do the research, you should know exactly how much each vendor will charge, on average, for the solution you are looking at for an organization of your size, what the price trends are, when they make the best deals, and what the best pressure points are. If you don’t, you shouldn’t be making major 6, 7, and even 8 (when you consider lifetime) figure software acquisitions. Period. Market monitoring is a must!
Skip the mind games
Abruptly end a meeting mid-stream to make a point? Make the salesman sweat at quarter-end to squeeze a few extra discount points? Scream emotionally that they are price gouging and you’re going to not only report them to the Better Business Bureau but tell all your friends in the local Procurement Association? Threaten to use Klod or Chat J’ai Pété to build it yourself? Lie and say you can make do with your current system for another quarter if you can’t come to a deal or just select any random competitor at DPW?
Sure you could use these techniques to try to get a better deal using an antagonistic wild west negotiation philosophy, but at the end of the day, it will cost you more than you save because even if the rep caves (because the rep knows if he doesn’t close something, he’s the next rep walked out the door by the investors to make a point), how incentivized is the rep, and the vendor as a whole, going to be in helping you to succeed? Especially if their profit margin is close to 0 in a best case situation? Answer: not very. They will do the absolute minimum to meet their contractual requirement, and then take the phone off the hook, tell the chatbot AI to infinitely loop you when you try to contact support, and, when there is an actual bug, make sure you are last in the queue to get serviced, waiting until the last minute of the SLA to do it
Focus on the value you are getting, a price point based on real market data and intelligence that they should be able to match (and make a fair margin while actively supporting you to meet the ROI metrics they promise), and delayed payments until the module is actually live and users actually onboarded before you start paying for it. (In other words, if you buy six modules, but only two are available day one, two more won’t be available for 90 days, and two more for 180 days, you don’t pay the full license fee until all modules are live AND all users set up. And implementation/integration payments are tied to milestone completion.) That’s way more effective.
Now, as always, there are a lot more tips and advice we could give, but these are the biggies. If you want more details, dig deep in the archives. Or, you can contact <font=black>the doctor for an engagement.