Opioids are the cornerstone medication for the management of moderate to severe pain. Unfortunately, vast inter-individual differences in dose requirements complicate their effective and safe clinical use. Mechanisms underlying such differences are incompletely understood, are likely multifactorial, and include genetic and environmental contributions. While accumulating evidence suggests that variants of several genes account for some of the observed response variance, the relative contribution of these factors remains unknown. This study used a twin paradigm to provide a global estimate of the genetic and environmental contributions to inter-individual differences in pain sensitivity and analgesic opioid effects. Eighty one monozygotic and 31 dizygotic twin pairs successfully underwent a computer-controlled infusion with the μ-opioid agonist alfentanil in a single occasion, randomized, double-blind and placebo-controlled study design. Pain sensitivity and analgesic effects were assessed with experimental heat and cold pressor pain models along with important covariates including demographic factors, depression, anxiety, and sleep quality. Significant heritability was detected for cold pressor pain tolerance and opioid-mediated elevations in heat and cold pressor pain thresholds. Genetic effects accounted for 12–60% of the observed response variance. Significant familial effects accounting for 24–32% of observed variance were detected for heat and cold pressor pain thresholds and opioid-mediated elevation in cold pressor pain tolerance. Significant covariates included age, gender, race, education, and anxiety. Results provide a strong rationale for more detailed molecular genetic studies to elucidate mechanisms underlying inter-individual differences in pain sensitivity and analgesic opioid responses. Such studies will require careful consideration of the studied pain phenotype.