I found that they are (now) appearing in the threads section of my watchlist, where I could remove one.