Some potential ideas to explore. Take what you want, leave what you don't.
a) if you're training for attention span, make sure the target is appropriate and also within reach of your child.
b) have a plan for the visit: when I helped at a school library, classes for kids in your kid's age group would come in, the librarian would read them a story, then the kids would look for a book, check out at the desk and read (or look at the book anyway) quietly until the end of the visit. I think we'd get about 40 minutes for a visit. Most days, at least some of the kids would be getting ansy before it was time to go.
c) Plan around your kid's activity needs. Some kids will do long still antention tasks better after doing some amount of physical activity. Some kids will do these kinds of things better after a meal. Some will do it better in the morning or the afternoon. Many kids will have a harder time if the library visit was a surprise. You know your kid, try to have your library visits when they're likely to work well. If he likes story time, try to visit when there's a story time available.
d) don't expect that you can both go to the library and work independently. You're going to the library with him, and he's going to need you to help him out for much of the time. But you might be able to find him a book together, then find you a book together, then sit down and read for a bit together.
e) if all you can get done is finding a book, no big deal. You can read at home too.
If a lion can figure out how to behave in the library, so can your kid ;) https://www.michelleknudsen.com/library_lion_77788.htm