We have thumb drives that can store petabytes of data?
replyOr did you mean the "big data" crowd which thought 500GB was noteworthy? I don't think anyone took those serious, neither in 2010s nor now. That was always "small" data
My rule of thumb was "can it fit in RAM on a server?" If it can, then it's not big data.
reply500GB is in the "fits" category.
You can quadruple that and could still fit in server RAM
reply> We have thumb drives that can store petabytes of data
replyWe do?
It was a question that you've edited out the punctuation. You're asking the exact same thing as the person you've replied
replyPlease provide a link.
replyYou would need 4 and change of these 245tb Kioxias to hold 1 petabyte, and an entire server grade computer to run them.
replyhttps://www.tomshardware.com/pc-components/ssds/kioxia-unvei...
Or 250 of these ~$400 4tb flash drives and an insane number of dongles to connect them all:
https://www.slashgear.com/1847725/largest-usb-thumb-drive-hi...
Plus one more for your parity drive.
replyMost companies using term "big data" had datasets in TB region. One company I had a gig at had full Hadoop cluster setup and their whole dataset was 40GB. Their marketing had all the big data adjacent keywords over the brochures for clients.
replyThat's a decent quality 3 hours movie :D
replyTo some degree IMO big data is still a mindset when it might take a day to process your data in a normal SQL query. Some tech doesn't scale to the data size for all use cases, and you need different solutions.
reply