The server snapshot command allows you to see exactly what our servers see on the webpage. You can use this tool for debugging your project.
You can add this command at any point of your project. This tool can help you determine if the website is blocking ParseHub (in that case you can use our IP rotation feature), or if the format of the website is different on our servers compare to what you are seeing on the ParseHub app.
You can check the server snapshots from the Run page while your project is running:
An indication of the error is provided and you can click on the link to view the page. The section further down called Troubleshooting Server Snapshot Elements has more details on what each error means.
You can also see the captured snapshot on your template when the run is completed:
Troubleshooting Server Snapshot Elements
Each error message will give you an indication of the error found for that snapshot. These include:
- element wait exceeded for [element_name] in template [template_name] - this indicates that a longer time than the "wait" time set for element_name passed and ParseHub was unable to find that element. If the element does appear in the page, you can resolve this issue by increasing the "wait" time on that element. To do this click on the element on your template, tick the "Wait up to X second for elements to appear" option and increase the number of seconds.
- page load failed in template [template_name] - this indicates that the page did not load as fast as we expected. If you click on the snapshot and see an error page or Captcha page, read more below on how to resolve these issues.
- [element_name] selected no elements in template [template_name] - this indicates that ParseHub was unable to find the selected element with name element_name on the page. You can visit click on the snapshot to visit the page and re-train ParseHub to find the element on that page if it is there.
If you click on the snapshot you will be able to see exactly what our servers see on the webpage:
- If this page is sometime like a "Access Denied", "Blocked IP Address", "Proxy Server Refused Connection" or "Failed to Connect", this typically means that the ParseHub scraper is being blocked by this website. You can usually resolve this by enabling the IP Rotation feature on our paid plans.
- If this page is a Captcha, you can create a template for this page to resolve it by following the instructions on our Solving Captchas tutorial.