I've had a program where some R code stuffs some input in an URL, fires it at a server, then reads the returned CSV. No error handling or validation is attempted.
replyAs it happens, basically anything is a valid CSV.
If the input is wrong, the server responds with an XML error. This got interpreted as a CSV. Every line became a record, but no column headers were found so every expected field is null/NA. All these nulls get nicely sorted and processed, which causes more string errors, etc ... Mayhem ensues when thst stuff ends up in a database.
CSV is not a format, its a vague idea, https://www.youtube.com/watch?v=YrqSp8m7fmk - this is a great video on how to make it fast, and how its pretty bad
replyCan't agree more. I've dealt with a CSV where multiple CSVs were cat'ted together, but the originals were not in the same dialect and differed even in charset encoding. Wrote a statistics based parser that decided which part of the file had to be decoded which way. Interesting technical challenge, but my god the reasons for its existence were dumb.
replyFalsehoods programmers believe about CSV: "Yeah, sure it's a mess and there are like, a gazillion different dialects of it, but at least once you know the dialect of a file, you can decode it"...
reply