My rule of thumb was "can it fit in RAM on a server?" If it can, then it's not big data.
reply500GB is in the "fits" category.
You can quadruple that and could still fit in server RAM
replyMost companies using term "big data" had datasets in TB region. One company I had a gig at had full Hadoop cluster setup and their whole dataset was 40GB. Their marketing had all the big data adjacent keywords over the brochures for clients.
replyThat's a decent quality 3 hours movie :D
reply> We have thumb drives that can store petabytes of data
replyWe do?
It was a question that you've edited out the punctuation. You're asking the exact same thing as the person you've replied
replyPlease provide a link.
replyYou would need 4 and change of these 245tb Kioxias to hold 1 petabyte, and an entire server grade computer to run them.
replyhttps://www.tomshardware.com/pc-components/ssds/kioxia-unvei...
Or 250 of these ~$400 4tb flash drives and an insane number of dongles to connect them all:
https://www.slashgear.com/1847725/largest-usb-thumb-drive-hi...
Plus one more for your parity drive.
reply