I found this little gem thanks to @optimom. After tweeting out I was in desperate need for something to do with the boys she recommended Dundurn Castle. Located in Hamilton, the castle is the restored home of Sir Allan Napier MacNab, who was apparently really rich, because the house is huge. Built in 1835 the house has something like 73 rooms.
The tour of the house is about an hour and is done by a tour guide wearing a costume from that era. Not only did the kids love it, they actually learned stuff. Plus we got to eat freshly baked bread. With butter. I would have gone just for that.
After we toured the castle, we walked across the lawn to the Hamilton Military Museum where the boys dressed up in military uniforms, played accordians and learned how to tie knots. Which is way better than the fighting they were doing at home.
All in all it was a great day. Thanks Optimom!