Your characters may be able to perform special attacks by pressing special button sequences. The only reliable one is rapidly pushing X either:

Before a character is about to attack (resulting in a double attack)
Before a monster is about to attack (resulting in a guard or counterattack)
Before a character recovers from grogginess (resulting in gaining more VP).
Another suggestion is to hit X repeatedly while either holding Up or Down. Holding Up when a blue diamond appears result sin the double attack and Down when the diamond is gold results in the super attack.