We had a test we just re-ran until it went green. Everyone knew the one; it even had a nickname. Then the same race condition showed up in a payment flow in production, and we realised the test had been telling us about it for months.
Now a flake that only passes on retry opens a ticket instead of a green build. You probably have a nicknamed test of your own.