If you have javascript, it should change to observe.
Oh, so it is different without scripts.
I found that they are appearing in the threads section of my watchlist, where I could remove one.