You would probably be best to provide at least a basic schematic (using Fritzing would be good as this is a Fritzing forum 
) of the circuits (or at least what I assume are serial ports, the serial ports being multiplexed somehow by fets). That said, first recommendation would be dump the 328s. A PI Zero would be a good bet (may be still hard to obtain though as I see they are still out of stock on Adafruit) or another 32 bit large ram ARM CPU. That should allow you to do the graphics stuff on the micro and eliminate the radio link. Your pwm woes are easy to solve (even on a 328), if not the full board then at least the chip driving it.
I believe that I2C should be able to run on pretty much any pin (spi not so much) as it is slow. I'm not familiar with XBee transceivers bur radio in general (unless it has internal error correction) is subject to drop out and you will thus lose characters due to transmission errors. Your code needs to deal with that by for instance retransmitting lost packets if the radio doesn't do so internally. The real issue is likely to be 2 uarts both trying to send down the same path (which sounds like what should be happening here, modulo what the fets do, thus the diagram). Uarts expect to be the only thing sending usually and will tend to write over each other and cause transmission errors even without the radio.
Peter Van Epp