1) After you've dispatched the lesser riders, keep parrying. The Captain can't touch you this way. He might knock you down, but this doesn't hurt.

2) After a few advances, he reigns up on his hind legs. Time to strike. Slice em up real good, but he comes back up, and with him another Rider. Kill that one and repeat.