What is wrong with him?

Maybe he is the one that is falling for you. He just doesn't want to admit it, and he was upset about you going to date a guy. Since it didn't work out and you came back to him he feels like a rebound. Or maybe he thinks you will fall for him and he might hurt you, but that doesn't seem right. Hopefully you can work things out :)