Yes, but there are other particles that may slow down debris, although I'm not sure how much impact those have on bigger chuncks. Nevertheless gravity is way too low from an asteroid to pull any significantly sized rocks back.