Im scared hes going to leave me?

how do i tell my man im ready to move into our new place cause i feel if i tell him hes going to leave me. cause i told him that once before in he said if u wanna leave leave your just thinking about yourself. thats what he said in thats not true im thinking about our family but he choose to live with his friend in our new place is only 3 minutes away in a car in im just just fed up cause his friends are money hungry in over in beyond nosey so please help cause i dont want to loose him.