<Assuming this is all the case...why not avoid the EV adj all together and just use the ISO or shutter speed manually. That way I would be more aware of what I am changing and avoid the possibility of having the EV adjusted and forgetting that I did it in later shots.>
Probably a better solution is to get in the habit of looking at the settings in your viewfinder. The automation and metering in the camera is really quite good. It's just a matter of learning how to use it.
Most photographers use Aperture priority a very high percentage of the time. Most use a fixed ISO unless there is a good reason. That means Exp Comp essentially adjusts shutter speed.