Human milk contains essential micronutrients for growth and development during early life. Environmental pollutants, such as potentially toxic metals, can also be transferred to the infant through human milk. These elements have been well-studied, but changing diets and environments and advances in laboratory technology require re-examining these elements in a variety of settings. The aim of this study was to characterize the concentrations of essential and toxic metals in human milk from four diverse populations. Human milk samples (n = 70) were collected in Argentina (n = 21), Namibia (n = 6), Poland (n = 23), and the United States (n = 20) using a standardized mid-feed collection procedure. Milk concentrations of calcium, zinc, iron, copper, manganese, lead, arsenic, and cadmium were determined using inductively coupled plasma mass spectrometry (ICP-MS). We used standard multiple linear regression models to evaluate differences among populations, while including infant age, infant sex, and maternal parity status (multiparous or primiparous) as covariates. Concentrations of all elements, except zinc, varied across populations after controlling for infant age, infant sex, and maternal parity. Calcium and magnesium showed more differences across populations than iron or copper. There were no significant differences among population in zinc concentrations. Mean concentrations of lead, but not arsenic, were low compared to recently published values from other populations. The concentrations of trace elements in human milk are variable among populations. Limitations due to small sample sizes and environmental contamination of some samples prevent us from drawing robust conclusions about the causes of these differences.