"Delta's such a sweet person and an incredible artist," Joe tells TV Week. "So it makes sense that the public would be rooting for her."
But Joe admits it can be frustrating when Delta uses her popularity to get in front.
"I came in knowing this," he says. "So it's about not letting her use the whole 'my tribe' and 'Aussie pride' thing against us."
Joe has previously confessed to being "competitive" with other coaches.
"We're very competitive and spend a lot of time together," Joe tells TV Week of his on-screen rivalry with Delta Goodrem, Kelly Rowland and Boy George.
"Sometimes you feel like your artist is getting bullied. So you jump in protectively and say, 'OK, stop talking – it's enough now'."