If you find yourself daydreaming about another person, especially a co-worker, then your crush is no longer harmless.
If you suddenly have an interest in getting into shape, then your crush is no longer harmless. While working out and getting physically fit is always a good thing, your motives behind doing so may not be. If you are trying to impress someone other than yourself or your spouse, then you need to think about what you are really doing
If you are forgetting (i.e choosing not) to tell your spouse about phone calls and lunches, then your “crush” and relationship with this person is no longer harmless