When i want my horse to go somewhere he doesnt want to he starts backing up and keeps backing up?

When I am riding and want my gelding to go some where he doesn't want to; he starts throwing his head; and backing up jumping and backing to the point he'll back us down over a hill I get off work with him re-mount and he's fine for a few minutes then its the same thing. How do I make him yeild and listen when he doesn't want to?