Scrape two eCommerce websites in one project

On ParseHub we normally recommend to build separate projects for each website but in some cases you might need to combine two different websites in one project.

In this tutorial, you will learn how to scrape data from a website and use it as an input for another website to scrape data.

For this example, we will scrape the Amazon & ebay websites. We will use Amazon data as an input to search for the same products on ebay.

Amazon:

1. Click on the + button next to the select page, then choose the Select tool. Next, select the first product title:

2017-03-28_14-48-49.png

 

2. Select the second product's title (highlighted in yellow) in order to select all the listed products:

2017-03-28_14-51-43.png

3. By selecting all the products, Parsehub creates a Begin New Entry (list) node and extracts the name and the URL of each product. If you are not interested in the URL you can hover on the extract command and remove it.

4. Select the relevant price for each product on the same page. Click on the + button next to the Begin New Entry and choose the Relative Select tool:

2017-03-28_15-05-26.png

 

ebay:

5. Click on the + button next to the Select page and click on advanced then choose the Go to template  tool:

2017-03-28_17-12-17.png     

2017-03-28_17-11-19.png

 

6. Now Parsehub will show you the configuration in a pop up for Go to template command. Check the box "Go to URL" and enter 'www.ebay.com' with quotations. Now choose the create new template box and create a new template for ebay. Click on the "Create new Template" button. This will open an ebay page in a newly created template.

2017-03-28_17-14-45.png

 

7. Now we want Parsehub to search the name of the products scraped from amazon on ebay. Click on the + button next to the Select page in your ebay template and click on advanced then choose the Loop tool to create a loop and call the items of "AmazonProducts" list from the main template. In the List field (the second box of the Loop configuration) enter the name of the Amazon products' list (the Begin New Entry command's name) which was "AmazonProducts".

2017-03-28_17-36-47.png     2017-03-28_17-37-15.png

 

8. Click on the + button next to the loop (For each item in AmazonProducts) in your ebay template and click on advanced then add the Begin new entry command to create a list for the ebay products.

2017-04-13_14-49-22.png

 

9. Click on the + button next to the Begin new entry and choose the Select  tool and click on the ebay search field.

2017-04-13_14-51-57.png

   2017-04-13_14-53-15.png

 

10. In this step we want to enter one of the products' names from the "AmazonProducts" list. As an expression you can enter item.name to call each item's "name" (the title of the extract command on the main_template) from the "AmazonProducts" list.

 2017-04-05_16-19-29.png    

11. Click on the + button next to the Begin new entry and choose the Select tool and click on the ebay search button.

 

2017-04-13_14-55-35.png

 

12. Change to browse mode and enter one of Amazon's product names into ebay search field. We will be able to see the function of item.name during the test-run only and it will not call the item from the list while building the project.

Click on the + button next to the searchButton and add a click command to search for the names scraped from amazon on ebay. Parsehub will show you the configuration in a pop up for click command, choose the option to create a new template. Now this will open a new template and will load the search results of ebay.

2017-04-13_14-57-08.png
 

2017-04-13_14-58-44.png

 

12. We can now extract the data from ebay search results such as name and price for the same products listed on amazon. In the final results you will have both Amazon and ebay products along with their prices. This project will allow you to use the final dataset for the price comparison purposes.

2017-04-13_15-04-18.png