upvote
People underestimate just how much effort goes into minimizing food waste and shortages in these types of industrial systems. The entire supply chain coordinates using these embedded sensors and control systems to make sure you always have safe food available at the lowest possible cost.

Connectivity in these environments is often challenging, it isn't like connecting a consumer fridge to wifi. Sometimes you are bridging weird networks that multiple parties need access to. It should be secure but that isn't as straightforward as one might assume. Many of these systems have very long lives, so there are many limitations that reflect the technology that existed when they were built.

All of this is invisible to the average person but it is what makes the world work.

reply
It looks more like they set a target level for shortages and then try to minimize waste without exceeding that level. Almost every time I go to Safeway, they have run out of some items I intended to buy. Trader Joe's is more reliable, probably because they have fewer items in stock.
reply
No, this “user story” does not mean the customer actually wants or uses the feature. You do not need remote command and control to tell if your truck chiller is working.
reply
You don't need remote command and control to tell you if your single truck chiller is working, you can just go look at the thermostat. But a company needs it to confirm that the chillers are working and at a safe temperature across all 1,000 trucks in their fleet.
reply
That's remote monitoring, though, not remote operation. Very different threat vector.
reply
What am I missing? Of course the user cares if their food has been stored at the proper temperature from production to transit to the shelf. They don't care that this specific implementation clearly had a security vulnerability, they just want to check out and get back to their lives (and not get sick), but there's no way this whole setup was completely unnecessary to produce that desired outcome, flawed or not.

Maybe we throw it all away and go to 1990s-style grocery chain management, hire a bunch of people, and costs go up while reliability and monitoring shifts to humans. But that doesn't mesh with consumer expectations on price, which is probably how the whole system originated.

reply
But you’re going to adopt a platform that meets the needs across your enterprise, so the capability is often there even if one situation may only call for monitoring.

Further, once you have hundreds or thousands of devices, “just call someone to turn it up/down/off” doesn’t scale.

reply
Further, once you have hundreds or thousands of devices, “just call someone to turn it up/down/off” doesn’t scale.

And yet, somehow it did scale just fine for a hundred years before we hooked everything up to the internet.

reply
And yet, progress marches forward and people want great efficiency and flexibility.
reply
And you know what the customer needs? Or is this just a personal feeling. There’s multiple people from the industry chiming in on these threads as to why this is needed.
reply
And yet, supermarkets existed in the year 2000, selling products at comparable or lower inflation-adjusted prices, without any of this connectivity.

Yes, a lot of it exists to squeeze a bit more of a margin out of the business, but come on. The alternative isn't going back to the Middle Ages, it's shaving 0.1% off the profit margin. It's OK to wonder if the risk is worth it.

reply
Grocery stores infamously have some of the lowest margins in all of retail and typically hover in the 1-3% range. As a sector they have about zero room for inefficiency and have to be this aggressive about chasing operating cost reductions.

The increased profits that have occurred in recent decades have landed with the big food companies, not with the retail chains selling their products at very little margin.

reply
Wonder the margins on those $9 boxes of Cheerios at major California grocers.

They did promise to increase the cost of breakfast a couple years ago.

reply
[dead]
reply
Yeah, I worked at one during that time. The amount of food we threw away on a regular basis was incredible.
reply
Except that in this case, the possibility of something like this probably is worth it, because system vulnerabilities are solvable problems, while ignoring the ability to improve system efficiency is a static stance. A lot of things have driven up the price of groceries since that time. You mention "comparable or lower inflation adjusted prices", but this hides the fact that the inflation adjusted basket of groceries in the US is near inflation (105% vs a baseline of 94%), while the input costs to that supply chain (like labor, and diesel) have gone up faster. The efficiency isn't doing nothing, grocery prices would almost definitely be higher without it.
reply
> because system vulnerabilities are solvable problems

If only…

reply
That's why I spoke of them in the plural. Obviously the possibility of a system hack (presumably) won't ever completely go away, but individual vulnerabilities as they appear in specific systems are usually tractable problems, especially if you narrow it to things that one wants solved rather than all possible vulnerabilities regardless of severity or accessibility.
reply
> system vulnerabilities are solvable problems

Which apparently why after >2000yrs of effort from both sides we now, finally!, have locks that no lockpicker can compromise.

reply
That's not what "vulnerabilities are solvable problems" means, except maybe for cybersecurity maximalists who believe that, "bro, add passkeys, and just one more thing, attestation, and W^X, and ..., and there will never be vulnerabilities".

Vulnerabilities will always exist. Systems will always be hackable. You solve the problem by patching what you can, and cleaning up after what gets exploited, if and when it gets exploited.

reply
> the inflation adjusted basket of groceries in the US is near inflation

You must not mean what you have said here. If so, you are double-dipping.

reply
I think it's an interesting question.

I would be interested to know what the damage caused by this attack is compared to savings over a system where thermostats need to be adjusted on-site.

reply
That's the work that goes into hiding the "other half" of the inflation (i.e. the 2x diff between reported inflation and what you can actually see in your life).
reply
How would you stay in business if your competitor was offering 20% off your prices because they had better logistics and tracking? Ask consumers to eat the cost due to their latent demand for, uh, inefficient delivery and food spoilage?
reply
Are you saying that stores in the 2000s were throwing away 20% more of their total revenue than they are today? That sounds crazy, especially given that the prices - again - seem pretty much the same or higher if you correct for inflation.

But more to the point, there's plenty of grocery stores that ask 20% more than their competitors and are doing just fine; in fact, constant price changes and promotions make it impossible for consumers to really compare if Safeway is cheaper than Whole Foods, so people usually go to the place that's physically closer, has better selection, or better vibes. I can't even know if a 12-pack of Coca-Cola is going to be $6.99 or $9.99 today.

reply
Coca cola is a bad example as it does not require refrigeration during transport and does not spoil fast (at all?).
reply
Choose your own number, but it has to be large enough that supermarkets decided the CapEx was worth it, and also large enough that once one did it, the rest followed suit.
reply
It's not even significant cost. The main cost is still the hardware, not electronics driving it, and especially not a tiny module that talks with electronics to send/receive commands/metrics
reply
There are numbers between 0.1% and 20%
reply
This is already the case in the grocery market. There are various leaner and more efficient operators offering the same goods for lower prices, sometimes a lot lower than just 20%. However most people don't think to shop at the various Grocery Outlet's or Food4Less's. Working class is wise to it but not really further up the class ladder. They don't think much at all and reflexively go to their ralphs, which plays the opposite game where they have higher prices such that they might occasionally mark them lower and make you think you are getting a deal. Some people are even going to erewhon.
reply
There’s a whole massive industry out there of companies looking to make 2% efficiency savings on everything, and VCs looking for this. The glossy presentation might have even said 2FA somewhere so yes it’s definitely safe.

The perspective of resiliency and how much will you lose if things go wrong seems kinda… new?

Funny that it should happen in France which has been at the forefront of European self resilience.

reply
Good find.

The biggest issue is that many of these IoT devices are badly secured. I spent years trying to convince people to build more secure IoT systems, but they always preferred cheap and easy solutions.

reply
> I spent years trying to convince people to build more secure IoT systems, but they always preferred cheap and easy solutions.

For me this has always been true of systems in general all across the decades that I've worked with technology. It's like pulling teeth to convince anyone in charge that backups or security are even a little bit important until after there's been a costly disaster, then it's all about "Why didn't you warn me this could happen?" - Umm... You mean like every single day for many months until you told me to shut up about it because my paranoia doesn't justify the tiny expense of ensuring you don't lose many tens of thousands of dollars or years of irreplaceable work or important records?

reply
I went through a manual you mentioned in another comment. It listed "High level of security against undesired user access" as one of the features but it wasn't expanded upon. So I did more searching and found cybersecurity best practices from Danfoss.

https://www.danfoss.com/en/service-and-support/fix-and-troub...

They highlight connection through a firewall, mention secure remote access methods, monitoring for data breaches, strong access controls, etc. Reasonable, but these are just recommendations, so the stores could have followed all of them or none at all.

An article about Danfoss vulnerabilities mentioned in a different comment (https://claroty.com/team82/research/freeze-the-controller-de...) says

> we searched publicly available internet-wide scanning platforms, including Shodan and Censys, for signatures associated with its management interface. It didn't take long to find what we were looking for. Numerous controllers [2765] were directly accessible from the public Internet.

So none at all is a possibility.

reply
I get that having networked devices is a very nice to have feature. However, putting them directly to the open internet is bewildering. Any person worth their salt in IT would know that's a bad idea. Were these people never brought into the conversation? Do they connect to the store's WiFi so that IT isn't actually needed to install once the WiFi creds are obtained?
reply
I imagine a very conscientious vendor could easily set up a system that detects if the device is trivially exposed to the internet, causing warning messages and beeps and general pestering.

... And then the customer demands it be turned off, and the customer is always right...

reply
IIoT (Industrial IoT) doesn't have time to deal with the load of bullcrap that cybersecurity is. Half of it is not even about security, but setting up moats and keeping control over users, which is very much not what you want to prioritize when setting up industrial infrastructure, and is rightfully rejected by engineers working there, who are concerned with safety and reliability first and foremost.

Though I'm afraid things are getting worse, or at least it looked so a few years ago, back when I worked in this space. The hot thing in IIoT / Industry 4.0 was basically taking all webshit technologies and deploying them on industrial hardware, to "leverage mature innovation and OSS ecosystem" or such nonsense. The problem (one of many) being, once the engineers were done trimming excessive security measures that negatively impacted safety, they were still left with node.js running on industrial controllers, now even less secure in general than it was originally.

reply
Yes, there is little demand for this sort of thing when the potential loss, should it even come, is already insured. Your customer might actually not be users themselves but their insurance companies. It may be a better model to say pitch to them and then have them offer discounts to their clients who are using your tech that mitigates whatever % risk. Someone pitched that car probe to Progressive after all.
reply
They're on Shodan and there's an admin account with a password that apparently *cannot* be changed.

So you can find one, log in, and have a look for yourself.

You shouldn't.

But you can.

But you shouldn't. Okay? Don't.

reply
I would have said let them be doomed. But due to late stage capitalism so much has been centralized that everybody is doomed in many cases...
reply
I look forward to the safe diversity provided by early stage socialism!
reply
Why are technicians (service engineers!) coming out for a simple defrost in the first place? The normal defrost cycle should be automated. For anything else you need someone to come out anyway because something is wrong - there is no point in starting a defrost cycle if the root cause is leaking refrigerant - fix the leak and refill the system both of which need to be done in person.
reply
Because in some ticket tracking system, some customer with the most tickets was nearly always calling for a tech to physically start a defrost. Maybe solving that accounts for 2% of support overhead. It’s best to lose that customer. Second best is telling them defrost is remotely-triggered, and then not doing it. Third best is actually triggering it.
reply
Thankfully these kind of attacks make pressure to finally have liability in software like in other industries.
reply
The people who should be held liable never will be. Their money is like an impenetrable shield to any sort of accountability. They'll just scapegoat the guy that warned them about the risk repeatedly for not doing more to prevent it.
reply
It has to start somewhere, even if imperfect.

I have been in meetings were the Data Security Officer refused to sign off on specific projects, without having a proper pentesting report in place, for example.

Because legally it would be their responsibility, if something would happen.

reply
I wonder if, following the US army story in August, someone might have bet on a prediction market “Danfoss refrigeration systems will defrost after suffering an already-disclosed vulnerability”.
reply
Seems like two totally different issues. One is cost and one is efficiency gains.

For the latter I would imagine you could have a short (maybe a month) logging period where you connect the fridges and collect data and then comeup with some offline system to manage them generally for better efficiency. You could even have a test group or single fridge that you keep connected to the internet for future data points (like in case of a giant heatwave or something)

reply
And I would bet dollars to donuts that someone at that company would respond "who's going to hack a refrigerator? Heh"
reply
Send those asking that question the scene from Silicon Valley
reply
We all keep a shotgun next to the printer in case it makes an unexpected noise here because the average HN reader knows a fridge manufacturer knows nothing about infosec and the lowest-bid contractor they farm the connection does the minimum to check the boxes. So yeah, we've been rightfully complaining about why does my icemaker need an internet connection for years.

Lets combine some testy topics: will LLMs make this any better? On the one hand, if it's easy for fridge manufacturers to churn out code, will the slop create enough cracks in the icemaker to sail the titanic through? Or will the lower barrier to pretty good security agents get more security into the IoT field (assuming it's taken advantage of?) Or will it not matter because the store manager will just set the web portal password to 123456?

reply
That could be done by the servicemen or managers on-site. There is no reason at all to have remote control fridges. All this really does is make some higher up feel more importantl because they have a centralised controller, while providing very little real benefit.

Anything you might want to change with a freezer already requires someone physically there. It's like making a socket ratchet have a remote controlled direction toggle. Or a remote control button to light the inside of your money box on fire.

reply
The benefit is the same as making websites remote controllable. A person then doesn't have to drive in to flip a switch, which means fewer people are needed to operate more systems, saving the company money.
reply
Switch to do what? If food is in it you can't change anything. If food isn't in it then why is it running or why wasn't it defrosted when it was emptied?

If you home refrigerator could be defrosted remotely, in what scenario would you ever do that?

reply
Presumably the grocery stores are already staffed by at least one employee capable of flipping a switch, no?
reply
Stores are now installing e-ink price tags to save labor updating price labels. They're not going to turn back from centralized HVAC/fridge monitoring and control.
reply
Unlike fridge remote control (note, not remote monitoring but remote control) e-ink price tags actually make a good deal of sense. The prices are already managed centrally so it reduces the labor to needing to accurately document which piece of hardware is at which shelf location as opposed to needing to manually check that each tag has the up to date price.
reply
Agreed, logging temperatures without the ability to actively control the fridge seems a much better approach.
reply