upvote

points

by mannyv10 hours ago |
comments
upvoteby bastawhiz10 hours ago|
next
[-]
Maybe I'm misunderstanding, but this powers 1.1.1.1, it doesn't front an internal dataset. A cache miss hits a nameserver. Which is to say, the dataset is "every DNS record in the world"
reply
upvoteby auspiv10 hours ago|
parent|
[-]
I think the question is probably more along the lines of - why not do a database with 100 TB of storage/records instead of a cache? tomato / tomato.. especially with smart caching in front of database. 100TB of flash is a good bit cheaper than 100TB of memory
reply
upvoteby ecnahc5159 hours ago|
parent|
next
[-]
Because it would be slower and have different scaling requirements than the ones they want.
reply
upvoteby pocksuppet7 hours ago|
parent|
prev|
next
[-]
It's not 100TB of data. It's probably 50 GB of data on each of 2000 servers. Because it's a cache. What is the point of a central cache if it's as slow to access as the original data?
reply
upvoteby inigyou2 hours ago|
parent|
[-]
TFA gives numbers closer to 5GB.
reply
upvoteby fc417fc8029 hours ago|
parent|
prev|
next
[-]
I'm no expert but presumably all of throughout, latency, and churn. DNS is approximately a giant KV store where the typical record has a TTL of ~5 minutes.
reply
upvoteby robotresearcher8 hours ago|
parent|
prev|
[-]
This is smart, task-specific caching in front of database.
reply
upvoteby seiferteric10 hours ago|
prev|
next
[-]
You have to cache, cloudflare doesn't know all the records ahead of time, they have to do recursive lookups to the authoritative servers that own the records and that is only good for the period of the TTL of the record. There is no "global" DNS record database or something like that.
reply
upvoteby pbhjpbhj9 hours ago|
parent|
[-]
>that is only good for the period of the TTL of the record.

Not really, TTLs are often short, but IPs might not change for years.

You can probably generate your own TTL, at scale, and avoid many DNS requests.

reply
upvoteby fc417fc8029 hours ago|
parent|
next
[-]
Why would anyone want to use a DNS resolver that tampered with records on a large scale? The TTL is intentionally set by the originator of the record.

Or alternatively, if you don't tamper why would I want to use a service that serves stale data?

reply
upvoteby robotresearcher8 hours ago|
parent|
[-]
Every distributed system serves stale data.

You can define away ‘stale’ by picking a consistency model, but look inside the consistency machinery and you will see fresher data you aren’t allowed to have yet.

reply
upvoteby fc417fc8023 minutes ago|
parent|
next
[-]
That's like refusing to use a fridge, insisting on letting your produce gradually spoil on the counter because "all harvested produce is in a state of decay anyway".
reply
upvoteby 5 hours ago|
parent|
prev|
[-]
deleted
reply
upvoteby otterley9 hours ago|
parent|
prev|
next
[-]
In DNS, the owner of each record has full control over its TTL. Intermediary DNS servers are required to honor them and are not permitted to replace TTLs with their own.
reply
upvoteby dizhn9 hours ago|
parent|
next
[-]
DNS servers do in fact do that but it would not be a good look for the world's largest DNS provider.
reply
upvoteby inigyou2 hours ago|
parent|
[-]
I don't think cloudflare cares about how it looks, also I think Google is bigger.
reply
upvoteby ButlerianJihad9 hours ago|
parent|
prev|
next
[-]
Actually that is not true. The IETF has expanded the definition of “TTL” and explicitly permits resolvers to serve “stale” RRs beyond their expiration time.

https://www.rfc-editor.org/info/rfc8767/

As a corollary, there is obviously no floor on refetching unexpired RRs, of course, except for efficiency concerns.

reply
upvoteby seiferteric9 hours ago|
parent|
[-]
That's only when the authoritative server cant be reached though
reply
upvoteby pbhjpbhj9 hours ago|
parent|
prev|
[-]
You are obliged to pass on the TTL, you're not obliged to cache according to it.

At least in my country (UK) I know of no law relating to DNS caching.

Why throwaway perfectly good data every few minutes that is only modified every couple of years, just so someone can move their domain quickly when they eventually wish to? It is my contention that a [caching] DNS service can do far better. Trusting user (domain owner) input blindly is not for me.

reply
upvoteby otterley9 hours ago|
parent|
[-]
It's not some sort of public law with public enforcement, but it is in the RFCs that govern the protocol.

I should be a bit clearer here; the TTL is an upper bound on how long it can be cached. Caches are free to consult more frequently but not less frequently. That said, out of respect for upstream cache operators and authoritative servers, most DNS caches honor TTLs as best they can.

reply
upvoteby inigyou2 hours ago|
parent|
[-]
The IETF isn't the internet police. You don't have to follow its advice.
reply
upvoteby otterley54 minutes ago|
parent|
[-]
No, but if you didn’t, the internet wouldn’t really work all that well. It was the fact that participants, despite being independent, all agreed (either explicitly or implicitly) to adhere to the standards that it became a global network. If they hadn’t, the result would have been more of the same: independent networks that only had narrow interoperability at best. It’s a textbook example of how global cooperation can yield incredible results.

Advocating to do things against agreed-upon standards without a compelling reason and without giving due consideration to the adverse consequences is one of the hallmarks of a bad engineer. Even Microsoft played nice with Internet standards for the most part (although with some notable exceptions at the application layer that got them well-deserved criticism).

reply
upvoteby inigyou51 minutes ago|
parent|
[-]
It was the fact that adhering to the standard was in the best interest of each participant. When it isn't, they don't.
reply
upvoteby otterley42 minutes ago|
parent|
[-]
It's impossible to know how often participants chose to stick to the standards anyway despite it not being in their best interest. Do you have any specific examples?
reply
upvoteby seiferteric9 hours ago|
parent|
prev|
next
[-]
then they would be breaking DNS at scale.
reply
upvoteby 8 hours ago|
parent|
prev|
[-]
deleted
reply
upvoteby BowBun32 minutes ago|
prev|
next
[-]
> If your cache is that big it isn't a cache.

This is an incorrect statement. Caches do not have a requirement of being smaller than their source data set. CDN is an example of a cache that generally matches the size of the source data.

reply
upvoteby toast010 hours ago|
prev|
next
[-]
It's a recursive resolver. The global DNS dataset is not something you could collect to serve directly vs caching from observations.

The data source is authoritative name servers operated by third parties, some of which are slow on their own, some of which are behind slow or lossy networks. Origin response times vary between probably 1 ms and 2 seconds +/- origins that never respond.

reply
upvoteby eggnet10 hours ago|
prev|
next
[-]
They’re adding the cache consumed across all of their servers. It’s not one giant deep cache.
reply
upvoteby otterley9 hours ago|
prev|
[-]
The simple answer is that if you didn't cache, DNS traffic would skyrocket, and the load would pile up on the authoritative servers, which were intended to be small, and during the early days of the Internet, were frequently on bandwidth-constrained links.

DNS is designed to distribute query load to the edge as much as possible, and that's enabled by caching. It just so happens that "the edge" is now becoming concentrated among a small set of providers because they wanted to make a business out of it.[1] They knew that this would be expensive going in, though.

[1] Nobody has to use 8.8.8.8 or 1.1.1.1. Most people can use their ISP's cache or a local cache instead without any noticeable difference in behavior.

reply
upvoteby fragmede9 hours ago|
parent|
[-]
The problem is there is a noticable difference in behavior because the ISP cache is overloaded so queries take longer. Sure, that's not everyone's experience, but there's a reason people chose to use alternate servers.
reply