Setting up the LiteSpeed Cache crawler
Updated 4 months ago
by
Tristan Payne
Sometimes when you visit your site, you may notice that the initial load takes significantly longer than normal, even though you're already using a caching plugin.
The cache is designed in a way so that the first user that hits an un-cached page will generate a fresh copy that will then be served to all subsequent visitors. During quiet periods however, there may be situations where a page is only visited frequently enough that each user encounters an expired cache and has to generate a fresh copy when they load the page, increasing the load times.
To get around this, we can utilise the LiteSpeed crawler to hit each page and generate a cached copy and then re-crawl the site every few hours replacing the cache.
Setting up the LiteSpeed Crawler
- Log in to your WordPress Dashboard
- Navigate to the LiteSpeed Cache plugin
- Select the "Crawler" option
- Navigate to the "Sitemap Settings" tab and enter your sitemap URL.
https://yourdomainname.tld/wp-sitemap.xml", however, this can change depending on the plugins you're using:All in one SEO :
https://yourdomainname.tld/sitemap.xml
https://yourdomainname.tld/sitemap_index.xml- Once this has been added, you can navigate to "General settings" and set the crawler to "ON".
- Then navigate to the "Map" tab and click the "Refresh Crawler Map" button.
- Lastly, head to the "Summary" tab and select "Manually Run" to start the first crawl straight away.
- Done!
The LiteSpeed crawler will initially do a first pass of every page on the site and ensure that this is cached. The first pass will normally take upwards of a few hours so you won't see the benefit immediately.
Once the first pass is complete, any further passes will only crawl pages that are un-cached or due to expire to ensure that every single page has a pre warmed cache to be served to human visitors.
