The Duco promoter, who represents Parker, it at odds with the Fury’s and Mick Hennessy of the appointment of an English official in Terry O’Connor.
In dramatic events on Monday, Higgins turned up late and gave Hughie’s father and trainer Peter both barrels over a breach of contract.
“It looks dramatic, but the reality is – I was angry because Peter Fury agreed to a neutral referee, the contract states that,” Higgins exclusively told Sky Sports.
“The WBO guidelines said that. That’s why Peter got angry. If you watch the press conference again, you will focus on my behaviour, but I tell you what I was calmly pushed out. Peter Fury was very angry and he knows, because he’s an honourable man, that I’m right.
“I don’t blame the Furys, I blame the British Boxing Board of Control, but the reason Peter was angry, because he knows I’m right. Peter Fury went mental, he’s used to being respected, and he deserves some respect. I respect him, because I believe him that he does want a neutral referee.”
Higgins even went as far as stating he could withdraw Parker from this Saturday’s fight, if he should wish to do so.
“Of course the fight is going ahead and I have got nothing against the Furys (but) I just want a fair fight,” he said.
“But if I want to call the fight off, I might be able to, because the contract does say a neutral referee. I feel sorry for Peter and Hughie Fury.
“It’s a bit embarrassing. They probably want a neutral ref,” added Higgins.
The UK TV rights being agreed for broadcast on YouTube have also been a sore point between camps, although at this stage the Manchester Arena fight is still on.