An avatar in a video game is controlled based on input from multiple users. An input composition system receives a first input from a first user. The first user uses a first input source for controlling the avatar. The input composition system receives a second input from a second user. The second user users a second input source for controlling the avatar. The input composition system alters the first input based on a first user performance modifier. The input composition system alters the second input based on a second user performance modifier. The input composition system creates an avatar input value based on the first altered input and the second altered input. The input composition directs the avatar in the video game based on the avatar input value.