If he was being pulled from home to home, with most of those homes belonging to drug addicts, then the chances of him being sexually abused are very high. Sexual abuse doesn't have to be done through direct touching. It can include: adults talking to him about sex, and showing him sex (pictures or movies, internet, or directly in front of him). I'm sure he may have seen or heard alot of things he shouldn't have.
If you have taken on responsibility as guardian, and if you care deeply about this issue, I might suggest probing a little bit further to find out just what he's been put through. Is there an adult male figure around that he trusts, that can have a healthy "birds and the bees" talk with him?
You have a good heart for caring so much, and taking them in. Good luck and please keep us updated!
