It's not a question of being smart or stupid. It's whether the tool fits the task it's applied to and the affordances it gives the user.
Scheme is intended more as a teaching tool than an actual language. Its simplicity is perfect for reasoning about programs. It's less well suited to practical tasks.
About the only really difficult lesson of Scheme is if you use it as a purely declarative language. Imperative features are a natural affordance of the human brain. Working with them is beautiful and alien.
I think it's refreshing change of perspective, and certainly worth pursuing if you're interested at all in in building programming structures rather than just using them. but if its not at all to your taste I wouldn't beat yourself up about it.