Advice on horse that dislikes being ridden outside?

Basically, I got a horse today and he was a total gem the two times I went to go try him out. He's a 16.3hh Irish Draught 5yr old gelding. I went to go and try and ride him in the outdoor today and he was horribly spooky. He seemed really tense and his ears were back the whole time, but when I moved him into the indoor he relaxed completely, and was a total plod (which is what he was when I went to try him). I asked the person I got him from if it was normal, and she said that it would be normal until I get to know him, and it would take a few weeks until he would be comfortable being ridden outside. Problem is, I'm moving stables in a few weeks coz the one I'm in at the moment is waaaay too expensive, so is there another reason/solution to this? Sorry if it doesn't make sense...