Do not get caught using your Magic at home around more than one other Sim visitor or the Mystery Man will appear and fine you for exposing your magic. However, in Magic Town you do not have to worry about this, because almost everyone uses it.