Navigating and scraping pages with no "Next" button (pagination)

ParseHub can easily navigate through a series of pages linked by a 'Next' button to scrape them. However, this isn't always possible, if there is no next button on the page.

 

With a few easy steps, pages like this can be scraped through just the same.

How to scrape multiple pages without a "Next" button

1. Click on the plus button on the right of the Select page command. Create a new Select command.

2. Click on the current page number, in the row of links, to select the element.

3. Change to Browse mode.

4. Click on the next page, and check to ensure that the current page number is still selected. If not, click on the current page number to train ParseHub on the right element.

5. Click on the plus button to the right of the current Select command. Create a new Relative Select command.

6. Click on the current page number to define it as the parent of the relative select. An arrow will appear. Next, click on the next page number. This should train ParseHub to select the next page relative to the current one.

7. Change to Browse mode again and go to the next page. Click on the relative select and make sure that Relative Select is selecting the correct next page number on this page as well. If it is not going to the next page number. Switch to Select mode, click on the Relative Select command that you created in the previous step and go from the current page to the next page number to train the relative selection on more examples.

8. Finally, click on the plus button to the right of the new relative selection, and add a Click command.

9. A popup will appear. Choose Go to Existing Template. Use the dropdown to select the template that you're currently working in. This will let ParseHub loop through the same template and move to every page

With all of this, ParseHub should be able to navigate through every page, to scrape all of the data you need on each.

You can read our other tutorials on scraping eCommerce products from each page, or when you should use the Relative Select command

Have more questions? Submit request!

0 Comments

Article is closed for comments.