Now that you've built your first scraping project you can test run it to see the project in action and ensure that it's working as expected before running the project.
Test runs run locally on your device and allow you to preview how your project will behave and the data that will be extracted.
Test running your project
You can access the test run tool in two ways:
1/ Accessing the test run tool from "Get Data" 1. Click on the "Get Data" arrow at the bottom your project: 2. Click on "Test Run" |
2/ Accessing the test run tool from the project options menu 1. Click on the options icon
2. Choose "Get Data" to go to the screen above from where you will be able to choose "Test Run" |
Once you've clicked into your test run you will see the following page:
On the top left-hand corner there are four icons:
Step-In: each click will move to the next command on your project so that you can test your project in action step-by-step. This is very useful for troubleshooting as you can slowly move through each command and understand the project's behaviour. | |
Play: each click will run the project up to 5 pages. This is useful to see your project in action and see if your project's logic is working. | |
Fast Forward: like the Play button, each click will run the project up to 5 pages. This mode allows you to quickly scrape your data, helping identify issues such as elements taking too long to load. | |
Pause: if the test run is playing, this will pause the test run. | |
Stop: this will stop test running and exit the test running tool. |
You can play your project first to ensure everything is running as expected and that elements have had time to load. Sometimes, if ParseHub runs through the commands too fast to find your element, you may need to click on the Select command and, in the command options, enable the "Wait up to X seconds for element to load" checkbox.
As you step-in or play through your project, the data will appear in JSON format on the preview pane at the bottom of the ParseHub screen after each command has been executed:
Troubleshooting your test run
When you open a template by double-clicking on its name, two circles appear to the left of each command on your test run, one in red and one in green:
The red circle pauses your project at that command. For example, if you wanted to play the project up until the "Select nextButton" command in order to be able to use the step-in option and observe it's behaviour, you could select the red circle next to that command.
The green circle skips the command next to it. For example, if you just wanted to test run the main_template without clicking into the details template, you could select the green circle next to the "Click each Product item" command.
While you can modify the project in test run mode, you can always click on the "Stop" button to return to your project.