A method and a system for making contextual recommendations to users on a network-based system. For example, activity associated with a user interacting with a network-based system is tracked. Based, at least in part, on the tracked user activity on the network-based system, a recommendation relationship type is selected. The recommendation relationship type can be either a substitute relationship type or a complement relationship type. A recommended object can be selected based at least in part on the recommendation relationship type and a first object accessed by the user interacting with the network-based system. A recommendation can be generated for the recommended object for presentation to the user interacting with the network-based system.