This tutorial will show you:
1. How to scrape Etsy.com categories. To follow along you can Start a Project in ParseHub on https://www.etsy.com/ca/c/jewelry/necklaces?ref=catcard-1217-216044426.
2. How to add pagination/paging to a project where you're already extracting multiple elements and their details from another page.
3. How to get a full set of data off of an eCommerce website.
Note: You might not be able to get all the data you need, as ParseHub's Free tier only offers a maximum of 200 pages per run. If you need to scrape more, consider upgrading to one of our premium or enterprise plans!
Building a paginating web scraper
1. Click on the "Select page" command + button that is located on the right of the command. From the tool box that appears, choose the "Select" tool.
2. Click on the "Next" button on the page to select it. It will highlight in green when selected.
3. Rename the "Select & Extract selection1" command by clicking on the text and typing in "button"
4. Click on the + button on the "Select & Extract button" command. Choose the "Click" tool from the tool box.
5. It will ask you whether it's a "next page" button, on which you should click Yes, after which it will default to repeating the current template. Click on "Repeat Current Template".
If you would like to stop the pagination at a specific page, you can change the number of repeats value which is one of the click command's options to something other than 0 (unlimited pages). For example, if you would like to click on the next button twice to scrape 3 pages of results in total (including the first page of the results), you can change the max depth value to 2:
You can update the number of repeats on the click command after adding it by clicking on the command line as well:
Download this Project
You can download the project that we just created here: Etsy Project