Mr Clarke insists he is the right person to sort out the problem.
They are among hundreds of foreign prisoners freed at the end of their sentences without being deported.
Police and immigration officers are undertaking a series of raids to detain some of the higher risk offenders.
Opposition MPs are calling for Mr Clarke to quit. Tory David Davis said there had been a "massive failure".
With probably victims out there who would not have been victims had it not been for this massive failure - I'm afraid, you know, he has to go
Shadow home secretary David Davis
High risk prisoner 'in Scotland'
Liberal Democrat president Simon Hughes told BBC News 24: "Somebody has got to put things right...but it does not have to be him (Charles Clarke).
"In politics, if things go wrong then somebody has to carry the can and that is the person at the top."
The Home Office says more than 200 police and immigration officers are taking part in the raids across Britain, to track down 63 high-risk foreign offenders, following initial raids overnight.
They are among the 1,023 prisoners released into the community without being considered for deportation over the past seven years - 79 of whom had been in jail for serious offences.
Whoever is responsible for this should be sacked right now
Piers Catton, Gosport
Send us your comments
It comes after the home secretary said on Friday that five of those released had reoffended and had been convicted for offences relating to drugs, violent disorder and inflicting bodily harm.
Two have also faced rape claims, with one case dropped because of lack of evidence.
The other case, which is still under investigation, is alleged to have taken place in August 2005.
This was after a report had been published calling for more speed in dealing with overseas criminals nearing the end of their sentences.
Action 'too late'
Mr Clarke has so far dismissed fresh calls for his resignation.
But shadow home secretary David Davis, speaking on BBC Radio 4's Today programme, said it was time for Mr Clarke to quit.
He said the home secretary was told "effectively for the fourth time, but told in terms 10 months ago" about the problem and should then have taken action.
Labour voices have been among those calling for Mr Clarke to resign.
FOREIGN PRISONERS 1,023 freed without being considered for deportation 79 of them originally jailed for more serious crimes, including 13 of them for murder, manslaughter, rape or child sex offences Five of them known to have committed offences since release - including violent disorder, drug crimes and grievous and actual bodily harm Deportation action now started for 63 of the 79 more serious criminals Officials have decided nine of the offenders should not be deported
Full text: Clarke statement
Labour MP Lindsay Hoyle told BBC News: "The public out there expect to be protected and what they're saying is, 'Hang on a minute, if these people have had seven months, 10 months to put this right, why didn't they? Why did releases continue?'"
Meanwhile another Labour MP, Graham Stringer, echoed Mr Hoyle's sentiment.
"As the facts change, then it's likely that Charles's position will become untenable," he told BBC Radio Four's PM programme.
He added: "I think the prime minister, in not accepting his resignation, is being cruel and unfair to the home secretary. Charles is a man of integrity: he did the right thing in the first place, and he should have been allowed to go."
However fellow Labour MP Ian Gibson told BBC Two's Newsnight the home secretary had the "next week" to sort out the issues.
Labour MP Stephen Pound told News 24 that Mr Clarke should be "part of the solution" to the problems.
George Howarth was another party colleague of Mr Clarke's who publicly backed the embattled Home Secretary, saying that he was the right man to fix the problems.
I think it all boils down to Charles Clarke and what has been going on in the media lately and they are just looking for a scapegoat
Andre Bester, father of re-arrested immigrant
Mr Clarke said the 79 serious offenders had all been identified on the police national computer. All but seven had now been considered for deportation, with action started in 63 cases.
Six have already been detained pending deportation.
Among them was Kyle Bester from Zimbabwe, who was convicted of rape but not deported upon release. He is being held at Belmarsh top security prison.
His father, Andre, said Mr Bester was appealing against his rape conviction and had been told by the Home Office that he could remain in the UK until it was heard. He thinks his son's arrest has been prompted by political events.
"I think it all boils down to Charles Clarke and what has been going on in the media lately and they are just looking for a scapegoat," he said.
Prime Minister Tony Blair refused Mr Clarke's offer to resign earlier this week.
British Home Secretary, Charles Clarke, is facing increasing pressure to resign from office over a scandal involving the release of foreign prisoners without subsequent deportation hearings.

It was recently revealed that, since 1999, 1,023 prisoners were released after serving their sentences, but were not subjected to deportation hearings to determine whether they should be allowed to stay in the United Kingdom. Five of the criminals have subsequently re-offended, with crimes involving drugs and violence. "It's clear that the increasing numbers of cases being referred for consideration led to the process falling down," said Clarke, "I do apologise. I have apologised, I continue to do so." 

Despite the admission of error on behalf of his department, the Home Secretary faces the very real threat of further crimes committed by the released offenders and still more pressure to resign. An intense effort is under way by the police and immigration services to track down 63 of the higher-risk released offenders, and more arrests are expected today.