<p>Yeah the server is probably misconfigured. Craft relies on a 404 happening to do some of what it does, but the status code should be changed before the response is returned.</p>

<p>I don't have enough information to tell you how it is misconfigured. Would depend on the webserver (Nginx/Apache/whatever) and we'd need to see the specific config.</p>

<p>There also is no <code>/sitemap.xml</code> on the site.</p>

CC BY-SA 4.0
2019-12-09T20:31:06.853Z
2019-12-09T20:31:06.853Z
andrew.welch
<p>I'm trying to crawl a site, I've tried using Screaming Frog and <a href="https://www.internetmarketingninjas.com/seo-tools/google-sitemap-generator/" rel="nofollow noreferrer">https://www.internetmarketingninjas.com/seo-tools/google-sitemap-generator/</a> and both return the same results when I try a crawl.  It finds the home page, but it then will give 404's for all the links it finds on the page.  If I got to the site manually I don't get 404's.</p>

<p>I'm new to Craft CMS so I don't know much (yet).  Inherited this site not too long ago.</p>

<p>Site is: <a href="https://highparknaturecentre.com/" rel="nofollow noreferrer">https://highparknaturecentre.com/</a></p>

<p>In addition, I also noticed that If I go to <a href="https://highparknaturecentre.com/about" rel="nofollow noreferrer">https://highparknaturecentre.com/about</a> the server returns http status 404 but returns the body content.  Could this be a server issue?</p>

<seo>
Trying to crawl a site
# Trying to crawl a site

(Asked by: username_0 on 2019-12-09)

I'm trying to crawl a site, I've tried using Screaming Frog and <https://www.internetmarketingninjas.com/seo-tools/google-sitemap-generator/> and both return the same results when I try a crawl. It finds the home page, but it then will give 404's for all the links it finds on the page. If I got to the site manually I don't get 404's.

I'm new to Craft CMS so I don't know much (yet). Inherited this site not too long ago.

Site is: <https://highparknaturecentre.com/>

In addition, I also noticed that If I go to <https://highparknaturecentre.com/about> the server returns http status 404 but returns the body content. Could this be a server issue?

**Tags:** seo


---

## 1 answers

### Answer by username_1 on 2019-12-09. Score: 2

Yeah the server is probably misconfigured. Craft relies on a 404 happening to do some of what it does, but the status code should be changed before the response is returned.

I don't have enough information to tell you how it is misconfigured. Would depend on the webserver (Nginx/Apache/whatever) and we'd need to see the specific config.

There also is no `/sitemap.xml` on the site.


---

