We used body mass index (BMI) and waist circumference (WC) as fat indicators to assess whether perinatal and early adulthood factors are associated with adiposity in early adulthood. We hypothesized that risk factors differ between men and women and are also different when WC is used for measuring adiposity as opposed to BMI. We conducted a longitudinal study based on a sample of 2,063 adults from the 1978/1979 Ribeirão Preto birth cohort. Adjustment was performed using four sequential multiple linear regression models stratified by sex. Both perinatal and early adulthood variables influenced adulthood BMI and WC. The associations differed between men and women and depending on the measure of abdominal adiposity (BMI or WC). Living with a partner, for both men and women, and high fat and alcohol intake in men were factors that were consistently associated with higher adulthood BMI and WC levels. The differences observed between sexes may point to different lifestyles of men and women, suggesting that prevention policies should consider gender specific strategies.