Millennials aren't as debt-ridden as you may think, but many struggle to wipe out their balances

It turns out millennials are perhaps more responsible about taking on debt than given credit for, with the majority carrying less than $5,000.

About 43%, of millennials (defined here as those ages 23 to 38) say that since turning 18, they haven't accumulated more than $5,000 in personal debt at any given time, excluding mortgages. In fact, only 4% of millennials say they have over $100,000 in debt, according to a new survey commissioned by New York Life of over 2,200 U.S. adults, including over 600 millennials.

"When we look at total debt or average debt, we see the kind of big numbers that align with the notion that millennials are the most debt-ridden generation," Stefanie O'Connell Rodriguez, a millennial money author, tells CNBC Make It. "But what I think this new data points out is that millennials are not a monolith."

"I think 99% of the talk about millennials and debt revolves around student loans," certified financial planner Bryan Kuderna with New Jersey-based Kuderna Financial Team tells CNBC Make It. He notes that the $1.6 trillion of outstanding student loan debt is typically associated with young professionals, with the average balance of about $30,000 per graduate, according to the The Institute for College Access & Success.

But millennials aren't the only ones with student loan debt. Americans over the age of 50 have student loan debt that accounts for $290 billion of the national equation, according to a report from AARP. And parents using PLUS loans to help their children pay for college account for another $89 billion.

Yet it's true that as a generation, millennials do face higher levels of student loan debt. The number of households with student loans doubled from 1998 to 2016, Pew Research Center found. The median amount of loan debt millennials carried was $19,000, significantly higher than Gen Xers' balance of $12,800 at the same age.

That doesn't mean every millennial who takes out student loans has a high balance. In fact, very few students have the six-figure balances you hear so much about. Only about 6% of borrowers take out over $100,000 in student loans.

It's also worth noting many millennials also graduated debt-free, or perhaps did not attend college at all. Nearly two thirds of millennials do not have a bachelor's degree.

The point is that millennials are a massive generation with diverse experiences — averages and other big picture numbers can only help so much, O'Connell Rodriguez says.

Although many millennials may not have massive amounts of debt, becoming debt-free is still a struggle, according to New York Life's survey. Over half, 57%, say they still have half or more of their debt to pay off, while only about 15% say they've wiped out their balance.

That makes sense, O'Connell Rodriguez says. "Even lower levels of debt can be a struggle to pay off for millennials struggling to find room in a budget dominated by record levels of health care, housing and childcare costs," she says.

Today, the average American spends almost $5,000 a year on out-of-pocket health care expenses and insurance premiums, up 101% over the past 34 years. About 38 million households are spending more than 30% of their income on housing, with about one in six spending half of their paychecks on rent or mortgage payments. When it comes to childcare, Americans spent a record $8,700 last year.

This hits millennials, who have lower incomes than prior generations at the same age, particularly hard. In fact, a recent report from the nonprofit, nonpartisan think tank New America predicts that millennials will not replicate the financial success of their parents or grandparents. The average millennial today has 41% less wealth than those who were at a similar age in 1989, according to "The Emerging Millennial Wealth Gap" report.

"With necessities now taking up a larger share of household income, it becomes increasingly difficult for individuals who are trying to put in place a basic financial foundation — debt pay off, emergency savings, etc. — to find the resources to do so," O'Connell Rodriguez says.

In addition to rising household expenses, many millennials also have to confront a lack of high-quality jobs and stagnating wages. Many graduated at the height of the Great Recession, which negatively impacted their ability to land jobs that could help them pay off their debt sooner.

"They've faced a very difficult labor market, and while we've seen that tide turn, starting a career off under those circumstances impacts earnings and job prospects long after the economy has improved," says Brian Madgett, head of consumer education at New York Life.

In general, money and debt can cause a lot of anxiety, not only for millennials, but for all generations, O'Connell Rodriguez says. That often manifests in avoidance: forgetting to regularly check your account balances, letting credit card and bank statements go unopened, and refusing to set up a budget.

The first step toward making headway on your debt: Take inventory. A full 34% of Americans don't know how much of their monthly income is going toward debt, according to Northwestern Mutual's 2019 Planning & Progress Study.

"Tracking and engaging with your money, no matter what age you are, is one of the best things you can do to set yourself up for financial success," O'Connell Rodriguez says. "It can be intimidating or overwhelming when managing a heavy debt load or trying to figure out how to pay for a medical emergency."

From there, start with simple steps and work your way up to completely paying off your debt. Come up with a plan to pay it down and set up a budget that works with your strategy. You can use either the so-called "avalanche method," where you start by paying off the debt with the highest interest rate, or the "snowball method," where you start with the smallest balance, pay it off, and then work your way toward tackling the biggest debts.

Don't forget that even though you're paying down debt, you should also put some money aside in savings each month, if possible. Having a savings cushion will help you out in an emergency, such as a car or home repair, so you don't end up adding more money to your credit card balance.

You don't have to do it alone, either, although about one in four millennials say that's exactly what they're doing, according to New York Life's survey. "It's a savvy move to ask for help when it comes to money, whether someone is looking to get out of a rut or start accumulating wealth," Madgett says.

Don't miss: If you can answer these 5 questions the right way, you can still spend money—even if you're in debt

Like this story? Subscribe to CNBC Make It on YouTube!