Outside of science it seems to be that people measure intelligence as "what I feel I have" and nothing else.

Memorization and recall is a huge building block of intelligence itself. Almost everything we call intelligent has some kind of memory to store and work on problems. The more storage you have the more potential intelligence.