In this paper, a method of operator based nonlinear vibration control for flexible arm using a Shape Memory Alloy(SMA) actuator with hysteresis is considered. SMA actuators have several advantages such as large force to mass ratio, light weight, and clean and silent actuation. However, due to the hysteretic characteristic of SMA actuators, it is difficult to design a controller which is satisfied with desired performance. In order to eliminate the effect of the hysteresis, a nonlinear compensator is designed by using hysteresis model, which is described by play operator-based Prandtl- Ishlinskii(PI) hysteresis model. Based on the concept of Lipschiz operator and robust right coprime factorization condition, nonlinear vibration controllers are designed, robust stability of the closed-loop system with unconsidered vibration modes of the flexible arm as uncertainties is guaranteed. Further, a tracking operator is designed to ensure the output tracking performance of the considered system. Finally, numerical simulation results are presented to support the effectiveness of the proposed design method.