Disclosed are various embodiments for detecting malicious activities by imported software packages. A monitoring service determines that untrusted code executing in at least one computing device has invoked a privileged operation. A context in which the privileged operation is invoked is identified. The monitoring service determines whether the context and the privileged operation corresponds to an expected behavior of the untrusted code based at least in part on a past behavior profile of the untrusted code. An action is performed in response to determining that the context and the privileged operation do not correspond to the expected behavior.