When God does good to his people it is not so much like a reluctant judge showing kindness to a criminal whom he finds despicable (though that analogy has truth in it); it is like a bridegroom showing affection to his bride.
Sometimes we joke and say about a marriage, “The honeymoon is over.” But that’s because we are finite. We can’t sustain a honeymoon level of intensity and affection. But God says his joy over his people is like a bridegroom over a bride.
He is talking about honeymoon intensity and honeymoon pleasures and honeymoon energy and excitement and enthusiasm and enjoyment. He is trying to get into our hearts what he means when he says he rejoices over us with all his heart.
And add to this, that with God the honeymoon never ends. He is infinite in power and wisdom and creativity and love and will see to it that we get more and more beautiful forever; and he is infinitely creative to think of new things to do together so that there will be no boredom for the next trillion ages of millenniums.