What you are probably talking about is in regards to the American marines based in Okinawa, right? It's not exactly true that they can't be prosecuted, its just that most of the time the cases are dealt with by military courts. I'm not sure wheter or not they can be tried in Japanese ones?
There has been increasing pressure on the US military to leave Okinawa, since they cause a LOT of trouble, not just rape, mugging, etc... but all the noise pollution etc... from the base. Theres talk of them being moved to Guam, but I'm not sure that'll happen soon or if ever. Although, with N. Korea being so close and so dangerous they could be useful, since the Japanese military are so bloody hopeless and have absolutely ZERO front-line experience.