I had a 2005 KLR (briefly). I didn't think it was too heavy for a dual sport at all. In fact, if anything, I thought it was too light. It's not fair to compare a dual sport to a dirt bike with lights hung off it. Obviously the dirt bike will be far moire nimble off road. But it will also suck big time on-road.
The idea of a dual sport (to me) is that it should be decent at both on and off road. Obviously this is a compromise and each rider needs to find the balance point they desire.
I did not like the KLR on road at all. When I rode my KLR anything faster than about 50 it scared the crap out of me. I was not be comfortable taking that bike on the interstate at all, where you get blown all around by the dirty air-wake from semi trailers.
So I sold it to guy living on Martha's Vinyard (no highways there) and bought my used Pegaso. The Pegaso is the same thing as a BMW F650. It weighs considerably more than a KLR at ~440 lbs wet, and the road manners reflect the added weight. It's great on the road all the way up to max speed. It's a little bit tougher to hoss around in the dirt, but for running fire roads and jeep trails it's great.