Experiencing adversity in childhood could influence cognitive performance in older age. Adults who had fewer years of education, economic hardships, and poor health during childhood were more likely to experience cognitive decline and memory problems later in life.