Since I am one of the very few folks who went from a 2004-ABS back to a 2003 standard, I'm in a position to lend some insight into this issue.
It is not so much an issue that the 2003 brakes "suck", or the like, rather, they are simply an inappropriate size for a particular usage. To wit: when riding 2 up, heavily loaded, and riding 8/10's or above.
I personally know of two instances where people (couples) have actually been hurt bad from crashing on their 2003 FJR when riding in a way that basically meets the above criteria. One was in Europe (a famous thread on the English forum back in 2003), and the other happened at WFO-3 back in 2004. In the 2004 incident, they where not too heavily loaded, but it was two-up, and certainly was aggressive riding, IIRC.
Suspect there have been a number of additional un-reported cases, too. Speculation here, mind you.
The 2003 calipers were sourced directly off the 2002 YZF-R1, and used 298mm rotors. They were among the very best mono-block calipers at the time, and their performance simply outstanding on that 397-lb sportbike.
It's was quite a different tune on a 600-lb (wet) sport-touring machine. On top of that, add another 300+ lbs of 2 people with riding gear, and another 80-100 pounds of cargo between hard bags, tank bag and topcase... there is no way the 298mm configuration is going to perform the same when trying to bring 1000+ lbs of machinery down from high speed. No way, no how.
Now, all that having been said... I, too, have had ZERO problems bringing my 2003 FJR down from some pretty dag-gone HIGH SPEED situations. But I rarely ride 2-up, and the bike is no longer my LD platform, so I don't carry a ton of weight on it. And like BikeEffects, I have Galfer lines and rotors, which have made a substantial improvement in feel and feedback.
To answer the original question.... no, you can not upgrade your 2003 FJR's brakes to the current 320mm rotors without installing 2004 or later forks, as you need to offset the caliper mount an additional 11mm to account for the larger rotor diameter.