undefined

points

[-]

It's true that markets are more _adversarial_. But there's still a lot of trouble with distribution shifts even in server metrics. As an example, our SRE team got paged a few times in the past month for traffic drops due to the World Cup. This stresses the nowcasting alert in several dimensions:

- there's no seasonal pattern to the matches, they happen sorta randomly.

- they drive increased query traffic in the hour or so before the game

- then during the game usage drops, sometimes to below "normal" depending on time of day and who's playing

So... now the accuracy of your forecasting tool depends on correctly predicting when world cup matches happen, and also who wins them!

edit: and this is just one recent example. others involve severe weather, national gameshows, earthquakes, and when you celebrate christmas.

by kjellsbells2 hours ago|

parent|

[-]

The UK power grid operators famously plan for massive demand surges at eg the end of major football matches. I cant imagine what their forecasters thought of the England-Mexico nail biter. Half the nation heading to put the kettle on, half glued to their seats. Would love to see the charts of demand now that the World Cup is done...

by IneffablePigeon2 hours ago|

parent|

[-]

They used to plan not just for major live events but for the end of popular soaps like Eastenders. With live TV becoming less important culturally and renewables eating the world most of their forecasting these days goes into weather predictions I think. Interestingly even other forms of power generation are affected by the heatwaves we have seen recently - it can reduce gas turbine efficiency by 10% or so and can even reduce the safe output of nuclear if their cooling water comes from a source that gets too warm.

But I’m sure the World Cup is still pretty relevant for operations.

by pishpash3 hours ago|

parent|

prev|

[-]

Unless disconnected from the world, or very thinly coupled, most processes become hard to predict in a generalizable way for this reason.

by gregw25 hours ago|

prev|

[-]

Between the difficulty and value propositions for forecasting timeseries "disk usage" versus "financial markets" there are some rather relevant time series such as "company sales" or "demand for company product" that come up again, and again, and again but are neither "easy" nor "predicting-the-stock-market-hard".

by 4 hours ago|

parent|

[-]

deleted

by qurren55 minutes ago|

prev|

[-]

Exactly. A lot of academics seem to not get the fundamental problem: You don't know the future.

You don't know what Trump is going to say 2 hours from now. You don't know what natural disaster is going to happen tomorrow. You don't know what war is going to break out next month.

NOTHING in your past data contains anything that can tell you these events are going to happen.

Now markets may have idiosyncratic residuals from momentum and reversion effects that you can quantitatively model and profit from, and that's a tradeable signal, but the way you do that is realizing that a certain coin is slightly biased and trade it a million times, averaging out the news shocks and recovering the residual idiosyncratic bias that you found.

Trying to forecast actual prices beyond ultra-short horizons is trying to predict those shocks, which is a fool's errand. You have a system with a signal to noise ratio of 1:100, and you're effectively trying to predict the noise instead of the signal.