That's interesting. I'm third owner of a 2004, have owned it now for 5 years, and never have a problem with the battery / charging system (yet).
Just started her up today to run some fuel stabilizer through her gullet and tuck her in for the winter. Hadn't been run for just over a month, and even at 3 degrees Celcius (about 36 F), she spun over and started right up.
Yes battery age / condition may be a factor, but I am also wondering about your charging system. Have you checked the idle voltage, and voltage at about 1500 RPM? Mine shows about 14.1 volts.
Years ago on a Yamaha RD400, I experienced charging circuit frustration, eventually realizing my rectifier was leaking charge to ground when running. Dead battery about drove me nuts - always seemed to happen at the worst possible times.
I'm sure I've seen other, related threads to charging system troubleshooting on this forum. Good luck in your quest!