An Enterprise Technical SEO Case Study
Challenge
- 18 Million pages site with too many page templates
- Over 50k links pointing to 404 pages
- Multiple teams working on website
- Inventory changing frequently
- Time consuming update every 6 months
- Needed to use historical solution and include manual overrides
Results
- Over 50k links pointing to 404s removed
- 25% of useless links removed
- Designed a new architecture with approx. 99% cleanliness
- Daily recommendations and automated updates sent
Our Goals
- Fix the internal link structure for a very large site (25 Million pages indexed) with over 50k links pointing to 404 pages
- Ensure all internal links point to canonical pages and reduce 301 redirects
- Design a dynamic, automated solution to provide ongoing updates to the internal architecture
What We Did
Expedia already had an existing internal link structure widget which was providing some level of internal linking, however the links were being rapidly outdated by changes in the site caused by ongoing business activity.
Built a Customised Web Crawler
The currently available crawler solution would not make the cut for this site.
We ended up upgrading our own customised web crawler to rapidly and efficiently crawl through the website to identify broken or underperforming links.
Working closely in regards to the business concerns of the client, we also built an automated logic engine which identified the best action to take with a broken link – to change its target, remove it, or to flag it for human attention.
Developed a Recommendation Engine
Using some of our existing Machine Learning solutions, we developed an algorithm capable of suggesting the most accurate replacement for an error page and turned a painstaking manual process into an automated one.
The internal link structure is now progressively kept up to date, with fix recommendations being issued every 24 hours. This has brought the number of bad internal links down from tens of thousands to less than a thousand at any given time, and has considerably improved the valid-to-invalid Googlebot crawl ratio, meaning that the site is updated faster and is seeing improved organic results on search engines.
Connected Crawl Rate with Sales Numbers
The next step on this project was to start analysing the landing page conversion outcomes of pages, and influence the internal link structure to prioritise pages with better conversion outcomes, while maintaining the breadth of internal links necessary to ensure that crawlers can still find everything on the site, and that internal links keep high levels of relevancy.
Results
- Over 50k links pointing to 404s removed
- 25% of useless links removed
- Designed a new architecture with approx. 99% cleanliness
- Daily recommendations and automated updates sent
- Integrated with historical system including manual overrides
- Tied up to business metrics, increasing sales
Get A Free Consultation
Speak to us today to find out how we can help you get similar results and receive a response from our Commercial Director within 24 hours.
