The Fed delivered a dovish enough message to drive stocks to record highs and bond rates lower, but that tone could change if inflation really does become a problem.
Some traders said they were surprised the Fed did not say more about inflation, after Wednesday's consumer price index showed inflation running at 2.1 percent over last year. Bond yields rose on the inflation data but were lower Wednesday.
"The Fed is deathly afraid of a rise in rates stomping out the recovery. It's clear they will allow inflation to run higher than it should be in order to achieve that," said Peter Boockvar, chief market analyst with Lindsey Group. "What's going to be the end result is the bond market is going to start dictating policy...next week PCE is going to point to that."
The Fed nudged up projections for short term rates in 2015 and 2016, and slightly cut their outlook for interest rates in the longer term. The Fed also trimmed another $10 billion in monthly bond purchases, taking it to $35 billion a month.
It also pared back its GDP forecast for 2014 to 2.1 to 2.3 percent, from 2.8 to 3 percent, but did not trim 2015 and 2016, indicating it believes winter weather temporarily hit the economy.
Although the Fed did not make any fresh comments on inflation, Fed Chair Janet Yellen, during a press briefing, said she thinks the CPI data is "noisy," in response to a question asked by CNBC senior economic correspondent Steve Liesman.
Yellen also said the FOMC has expected a gradual return to inflation toward its 2 percent objective for the PCE, its preferred inflation gauge. The price index of the PCE, or personal consumption expenditures, was up 1.6 percent in April from a year earlier. The Fed's own targets on inflation have 2 percent as the top of the inflation range for both 2015 and 2016.
"She did walk the tight rope handily. She kind of shrugged off inflation and when they asked about bubble territory, she said these seem to be historic norms, so she likes the stock market too," said Art Cashin, director of floor operations at UBS.
BlackRock co-head of Americas fixed income Rick Rieder said the Fed's dovish message would keep Treasury yields low for now and be positive for stocks since low rates are good for the stock market.
"It's hard to state this was anything but dovish...We think for the foreseeable future the Fed has clearly laid out that they're going to wait for a longer period of time before they start to move rates significantly, so we're going to be in a period of easy money...certainly until the next meeting," he said.
Stocks rallied after the Fed meeting, with the S&P 500 hitting a new closing high of 1,956, up 14 points. The Dow was up 98 points at 16,906, and the Nasdaq rose 25 points, to 4,362. The yield on the 10-year Treasury was at 2.58 percent in late afternoon, off its pre-Fed level of 2.62 percent. The 5-year yield fell to 1.67 in late trading, from 1.70 percent before the Fed statement.
Read MoreFed tapers another $10 billion
Cashin said stocks will return focus to the economic news, now that the Fed is out of the way. "Now we go back to looking at data and setting up for the new earnings season. There's going to be a huge (options/futures) expiration Friday, so tomorrow we could see volatility ahead of that," Cashin said.
Traders will also be watching weekly jobless claims at 8:30 a.m. Thursday; the Philadelphia Fed survey at 10 a.m. and leading indicators, also at 10 a.m.
Read MoreWhy stocks and bonds could duke it out all summer
"(Markets) continue to ignore that the tapering is only three meetings away from being done. It's obviously more focused now on when the Fed is going to raise rates. To me the end of the taper is a lot more dangerous than the first rate hike," Boockvar said.
He also noted the Fed's projections have the unemployment rate at 5.4 to 5.7 by the end of 2015, but the Fed funds is still going to be well below normal.
"Maybe the bond market is reading this wrong. Bond yields should be going up," said John Canally, investment strategist and economist at LPL Financial. "If the Fed is going to tolerate more inflation in return for a lower unemployment rate and better economic growth, that should be bad for the bond market, but the bond market is reading it like it's great news for us."
Read More
Canally said the bond market is still priced for perfection for slow growth and disinflation. "That's not what the Fed is forecasting. The bond market is saying we're going to get 1 to 2 percent growth in the second half of the year, and the Fed is saying we're going to get three percent," he said.
—By CNBC's Patti Domm