If you think it should be replaced, then replace it... they are not that expensive.
If you have a plug gasket, it should be replaced if you notice a little leak, or if there is a rip/tear on it. If you have a cloth feeling one, probably a good idea to replace it when you do an oil change.