Yeah, it's pretty much context. I'm not sure I've ever heard anyone say that they went to the hospital or were at the hospital without providing context. ("I went to the hospital for blood work." "I went to the hospital to visit my aunt.")

Since, if I understand correctly, "in hospital" only means as a patient, it would appear that you can differentiate without having to provide additional information. That seems like a better linguistic solution.