Chatter vibration in cutting processes usually leads to surface finish degradation, tool damage, cutting noise, energy loss, etc. Self-excited vibration particularly seems to be a problem that is easily increased to large vibration. The regenerative effect is considered as one of the causes of chatter vibration. Although the chatter vibration occurs in various types of processing, the end-milling is a typical process that seems to cause the chatter vibration due to a lack of rigidity of one or more parts of the machine tools, cutting tool, and work-piece. The aim of our research is to propose a simple method to control chatter vibration of the end-milling process on the basis of a coupling model integrating the related various elements. In this study, hammering tests were carried out to measure the transfer function of a machine tool and cutting tool system, which seems to cause vibration. By comparing these results, finite elemental method (FEM) analysis models were constructed. Additionally, cutting experiments were carried out to confirm the chatter vibration frequencies in end-milling with a machining center. In the hammering tests, impulse hammer and multiple acceleration pick-ups are connected to a multi-channel FFT analyzer and estimate the natural frequencies and natural vibration modes. A simplified FEM model is proposed by circular section stepped beam elements on the basis of the hammering test results, considering a coupling effect. In comparisons of the calculated results and hammering test results, the vibration modes are in good agreement. As a result, the proposed model accurately predicts the chatter vibration considering several effects among the relating elements in end-milling. Moreover, it can be seen that the chatter vibration is investigated from a viewpoint of the integrating model of the end-milling process.