I agree leave it. On my SE2 I make sure the pulley and the crank are as close to the engine frame as
Possible to reduce movement of the crankshaft from side to side keeping rattling noise to a minimum.
Your right,probably being a bit fussy,thanks guys.