If you cast spells in front of a Sim without a wand (Townie, Somebody, etc.) the Mystery Man who left the box at your door will fine you for showing your magic, and will erase the memory of the Sims who saw you (casting magic that requires a non-magic Sim will not be counted unless you cast those spells rapidly)