"There is a lot of interest in seeing it disappear," said Judge Markus Sonnleitner, who is handling Fritzl's insolvency.
"This seems to be the sensible course as there is little chance of ever selling the property for a profit. It should be done as quickly as possible."
His announcement puts an end to official uncertainty about what to do with the property ever since its terrible secret became known.
The house at number 4 Ybbstrasse in the small provincial Austrian town of Amstetten achieved worldwide infamy in April 2008 when it was discovered he had imprisoned his own daughter Elisabeth in it when she was 18 as his own sex slave.
The house was filled with tenants who never knew of the agonies being endured in the soundproofed cellar below them.
There Fritzl raped his daughter an estimated 3,000 times over 24 years, fathering seven incest children with her, one of whom died in the rat infested prison guarded by eight secret doors.
Fritzl, 75, was sentenced to life last year, for enslavement, imprisonment, rape, and for the murder of one of the children he fathered.
Judge Sonnleitner said he was looking for a way to pull down the building without attracting "too much attention".
Fritzl had built a dungeon under the house for his daughter and the children, while he lived a seemingly normal life with his wife upstairs. Three of the incest children were brought up by them after he convinced authorities that Elisabeth, now 43, had run away to join a cult and had left the babies on his doorstep.
The decision to destroy one property that belonged to Fritzl continues as controversy over a decision to allow him planning permission for others still rages in Austria.
On Thursday last week he received in his jail cell the green light for his insolvency lawyer to pursue a building project for houses, an office block and an underground car park. He lodged the blueprints for the scheme two years before his secret life was exposed.
The mayor of Amstetten said he was embarrassed by the plan but had no choice but to allow it to proceed as all planning criteria were met.
As neighbours of this new development announced an appeal against the decision, the old neighbours at Ybbstrasse 1,500 feet away pleaded for a swift end to the "house of horrors". Their fear was always that an entrepreneur might buy it and turn it into a grisly museum.
People still visit Ybbstrasse to photograph the house, the doors of which have been welded shut to prevent souvenir hunters trying to break into the cellar.
Josef Fritzl's mugshot 
The house where  used to live and where he imprisoned his daughter is set to be demolished. 

Fritzl locked his daughter Elisabeth in the make-shift dungeon for 24 years. During that time he raped her repeatedly — and fathered seven children with her, one of whom died in the basement of the house.

Judge  stated, "There is a lot of interest in seeing it disappear. This seems to be the sensible course as there is little chance of ever selling the property for a profit. It should be done as quickly as possible." Ever since the case became known in 2008, there has been a question over what would happen with the property.

The property, in , Austria, has become an unlikely tourist attraction with several people showing up to take pictures of the house. The door is sealed shut to stop anyone from gaining entry to the building.

On Thursday, Fritzl received the news that he has been granted planning permission to build a project consisting of houses, an office block and an underground carpark. Fritzl produced the blueprints for the project two years before police were aware of his crimes.

Judge Sonnleitner announced the plans as part of Fritzl's ongoing  case.