EFSA full review: animal studies show nervous-system effects from aluminum exposure. https://efsa.onlinelibrary.wiley.com/doi/pdf/10.2903/j.efsa....
CDC/ATSDR: oral risk levels are based on neurological effects seen in exposed animals. https://wwwn.cdc.gov/TSP/ToxProfiles/ToxProfiles.aspx?id=191...
However, it is worth noting that humans were not exposed to aluminum in the environment until relatively recently, when we started extracting it from bauxite and melting it.