The way I interpreted it - and that was a while ago, to be fair - is less that Amy and Rory don't care about the baby, and more like, if you want to interfere in River's past, you really should ask River first - which don't get me wrong, they should have done - and River seemed fine with who she ended up being. It could have been a very interesting question, but instead it never came up.