Hmmm.......I dont actually think the odd drums at the start are bad at all. I think they sit well, almost making the listener want to dance, but not knowing how to lol! Then when it drops (which IMO is a little abrupt,) it all falls together and we are able to dance again...
Oh, and your basses are sick, I've been trying to re create that sound for a while now, I can get near it but not quite dirty enough.......fancy sharing your secrets?