The 30 year-old, without a fight since 2016, believes Brook could potentially harm his well being after being stopped in his last two bouts with eye problems.
“I’m very surprised he is carrying on because he can harm himself and harm his career. If I was him, or his advisors, I would tell him to retire and enjoy life,” Khan told Sky Sports.
“After two big losses, two bad losses as well, if he’s thinking about fighting me that could be a third big loss and bad for his health.
“In his last fight against Errol Spence, he took a knee and gave up. In my opinion, he was doing really well in that fight, and he could have won it, there was only two rounds left.
“But he just went on his knee and didn’t want any more of it. Sometimes, that tells you about someone’s heart.
“Us fighters, we are into the ring until we die or get knocked out – that’s the only way you can get rid of us.
“With Kell, I think when he showed that he gave up, and didn’t want any of the punches, it just shows to me his heart is not in it.”
Brook was halted by Gennady Golovkin in September before the Spence loss eight months later but has signalled his intention to carry on.
A move up to 154 pounds could be on the cards for Brook, who underwent a second surgery for a broken eye socket inflicted by the American, who also took his world title.
Khan was high on Brook’s hitlist prior to facing mandatory Spence, although an encounter between the two seems as far away as ever.