Adam and Eve Question?

The bible says that adam and eve were the first people on earth. It also says they had two sons one of which killed the other and was banished. Here is my question....where did all the other people of the earth come from. We are all supposed to be direct decendants of adam and eve. If this is true then how did the other people come to be? If Cain was banished for killing Abel then who did Cain have children with? If he did not have kids then did Adam and Eve have more children that just had with each other and that is how the other people came to be? If so, does the bible not say that imbreading is a sin? Help me understand all of this please.