The rubber hand illusion (RHI) is a bodily illusion that a fake rubber hand is felt as if it was one's own hand when both the hands are synchronously stimulated. Also, it is well-known that people can experience the RHI when the movements of the visible fake and invisible real hands are synchronized. However, underlying mechanism of the RHI and relationship between action and body ownership are still open questions. Based on the RHI paradigm, the present paper discusses how the agency of a body part affects the sense of body ownership at the other body part. In this study, we examined if a simple up-and-down action of right index finger induces the RHI to left hand which was kept still by using virtual reality and robotics technologies. Our experimental results indicated that the action of the right index finger allowed the embodiment of the virtual hands at both the right and left hands. This implies that only a simple action of a body part increases the sense of body ownership at the other body part.