The problem with Stannis porn is always that our brains say, "wait, stop, Stannis wouldn't do that, he's far too repressed and he'd stop after one kiss." So every Stannis porn that isn't with his wife just demands some suspension of disbelief.

But your explanation that the universe would just combust is also very believable.