Is there a way for us to rename conquered capitals once we have taken them? If not, how difficult would that be?

Yes, you can name/rename any sector you own. Just click the sector and then click the rename icon ("N").

I did that. I renamed a city "Cave of the Jackbear" after my most excellent son, but when I "Explore Map" it still shows as the former name.

Actually, the "Explore Map" shows the name of the country. Eventually, it'll have more details available that will show sector names.

So we can't actually rename the country?

Well, once you completely conqueror a country by taking every zone it'll be absorbed to your nation.