He's the top Republican elected official in the state, but House Speaker Kurt Daudt is facing a challenge Tuesday from within his own party.

Daudt, of Zimmerman, is the most notable incumbent facing a primary challenge in the 28 contested races for the state Legislature. The primary will set the final slate of candidates for voters to choose from in November.

Outside groups are getting involved in Daudt's race, on both sides.

A recent mailing from a group called New Leadership PAC declared, "Kurt Daudt can't pay his bills, but he'll raise your taxes." It's a veiled reference to Daudt's recent problems with personal credit-card debt. An earlier mailer from the same group referred to a 2013 run-in Daudt had with police in Montana.

And it's hard for voters in District 31A to miss the trailer parked near Hwy. 65. It holds a billboard that reads, "Stop Wasteful Spending and Kurt Daudt."

A trailer parked in District 31A near Hwy. 65 holds an anti-Daudt billboard sponsored by the Tea Party, here on June 25, 2016. Sara Meyer | MPR News

The Minnesota Tea Party Alliance is responsible for the trailer. Jack Rogers, president of the group, said Daudt has broken promises to cut taxes and spending.

"I'm not pulling any punches and I'm not cursing at anybody," Rogers said. "But enough taxes. Enough of all of the garbage. Stop and clean up the messes we have. Give the people back their money." Rogers lives in the House district adjacent to 31A.

Daudt, a third-term incumbent, brushes off the criticism as the work of "fringe groups."

"I'm the best friend that they have in St. Paul right now," he said. "I'm the firewall. Our caucus is the firewall that's preventing all of the things that they want to prevent: higher taxes, more spending. It seems to me it would be more productive and useful of their time if they would join forces with me and help me fight against those things, rather than fight against me."

Daudt was first elected to the Legislature in 2010. Two years later, he was picked by House Republicans to be their minority leader. He was elected speaker after he led the campaign effort in 2014 that flipped control of the House. He's often mentioned as a potential candidate for governor in 2018.

Alan Duff of Isanti, here on July 6, 2016, is challenging Kurt Daudt from the right. Tim Pugmire | MPR News

His primary opponent, Alan Duff of Isanti, said he's challenging Daudt because he's frustrated.

"I believe our politics in this state are just completely shattered," said Duff, a veteran, small business owner and former member of the City Council and Isanti County Board. "I'm tired of the backroom deals. I'm tired of the people not being served by the political elite or the political system. I believe we also need a conservative voice in St. Paul that's going look out for the wallets of the citizens of 31A."

Duff's chief complaint was Daudt's brief support last spring for an increase in license tab fees.

"I believe that's something that the average person would rather see going in the other direction," he said. "I would support reducing that, not increasing it."

Daudt and Duff have a history. They served together for two years on the Isanti County Board. Daudt said he's proud of his record in county government and in the state Legislature. He also contends that he's more conservative and more effective than Duff.

Senate Majority Leader Tom Bakk, a Democrat, center, walks with Republicans, House Majority Leader Joyce Peppin and House Speaker Kurt Daudt as they arrive at the governor's residence Wednesday, May 18, 2016. Jim Mone | AP

"At times, you have to work with other people," he said. "I know from my time on the county board that my opponent had trouble working with other people and that's probably why he didn't get re-elected," Daudt said.

A pro-Daudt group called MN Conservative Grassroots Fund recently got involved in the primary contest with a cable TV ad critical of Duff.

Previous vote tallies show strong support for Daudt in the reliably Republican district. He was unopposed for re-election in 2014, and scored comfortable general-election wins in 2010 and 2012.

The recent criticism of Daudt has been unfair, said Nadette Waligora, a Daudt supporter who lives in Spencer Brook Township in Isanti County. She said she's admired his approach to politics for many years.

"I've talked to Kurt about all these situations that have come up as criticism about him, and asked him questions," Waligora said. "I believe things were taken quite out of context, and unfortunately they're using that against him. So, I think that he is above board in character."