In this paper, we propose a system to collect human behavior in detail with higher-level tags such as attitude, accompanying-person, expenditure, tasks and so on. The system makes it possible to construct database to model higher-level human behaviors that are utilized in context-aware services. The system has two input methods; i.e., on-the-fly by a smartphone and post-processing by a PC browser. On the PC, users can interactively know where and when they were, which results in easily and accurately constructing human behavior database. To decrease numbers of operations, users' repetitions and similarity of among users are used. According to experiment results, the system successfully decreases operation time, higher-level tags are properly collected and behavior tendencies of the users are clearly captured.