Presence is a key variable in virtual reality (VR), however, it is a complex variable consisting of different aspects. The Multimodal Presence Scale (MPS) by Makransky, Lilleholt, and Aaby was developed to measure physical, social and self-presence. But how well can it actually detect changes in one aspect of presence, and differentiate between different aspects of presence? To answer these questions, we use a German translation of the MPS in an experiment with a VR game with 45 participants. We examine social presence -- the sense of being with another -- specifically, and compare three conditions (abstraction levels) that should differ in the degree of social presence, but not in other aspects of presence. Results indicate that the MPS is reliable and useful for measuring social presence, while still correlating strongly with presence overall.