Cover raisins with brandy or water. Set aside.
Combine brown sugar, cinnamon, allspice. Reserve 3 tablespoons of this mixture for topping. Toss bread with remaining sugar mixture. Pour melted margarine on before you toss the bread crumbs. Your sugar mixture will stick better that way.
Place 1/2 of bread mixture in a lightly greased 1 1/2 quart baking dish. Drain raisins, place 1/2 raisins over bread crumbs. Repeat layers.
Combine egg, salt, and vanilla in a bowl. Beat for 1 minute. Add milk slowly. Pour over mix in dish and let stand for 5 minutes. Sprinkle reserved sugar mix on top.
Bake at 325F for 45 to 50 minutes or until a toothpick inserted into the center comes out clean