The former Chancellor claimed he was ‘not keen’ on the decision by David Cameron to stage a referendum on Britain’s membership of the EU.
Osborne, who campaigned strongly for Remain, said he had always been ‘gloomy’ about the likely result of last year’s vote.
However he said he had not wanted to create difficulties for Mr Cameron who remained a close friend and ally.
‘I was not keen on having a referendum in the first place but I chose to be part of the collective decision-making of the government,’ he told the BBC Radio 4 Today programme.
‘I certainly did not walk out of the government at the time.
‘That is not the way I tried to behave as chancellor.
‘I was not trying to be the difficult next door neighbour.
‘I was trying to be the person who, with David Cameron, my friend, made things work for the country.’
Mr Osborne said, as the vote approached, he had become increasingly pessimistic about the chances of a Remain victory.
‘I was always quite gloomy about the referendum and its outcome,’ he said.
‘I went round the country and I kept hearing from people I would assume wanted to stay in the EU that they wanted to leave so I was pretty pessimistic.’
Mr Osborne said, while he could not see the decision to leave the EU being stopped, there was still scope to debate the terms on which Britain withdraws.
‘I don’t accept that just because we are leaving the EU we have to, for example, leave the customs union or radically clamp down on immigration,’ he said.