Any frequency sound will damage your hearing. You just notice bass less, as our ears are less sensitive to it.
This is also why hearing damage from bass goes relatively unnoticed - if hi/mid sounds are too loud, we turn down the volume, but our ears are less sensitive to low frequencies, so we may be doing damage, and not realise it.
When you get hearing damage, loss of bass perception is usually not noticed due to its subtlety, and our ear's insensitivity to it. In any case, most adults can't hear low bass frequencies well anyway, but they do physically feel it, which "makes up" for it.