On iconoclasm and slavery

There is a lot of controversy surrounding the antiracist protests and the -sometimes destructive- removal of controversial statues from squares and parks in several countries. So when does iconoclasm turn from a defensible act of protest into senseless vandalism? And: does erasing the symbols erase the reason why these symbols were there in the first place? A personal view.