According to the current help documentation:

When a call is matched to an Auto Close Rule, it remains associated with that rule until:

• the call's Last Updated Date is changed
• the call is closed
• the Auto Close Rule is deactivated by deselecting the Active check box
• the rule is deleted from the Call Auto Close Rules window
• the criteria in the Auto Close Rule is changed

I would like to suggest that when the rules are re-ordered - it should clear out all rule associations and redo them.  Or at least give the administrator the option to do so.