How fast will my projects run?
Each plan on ParseHub grants you access to a certain number of workers on our servers, each of which scrapes about 5 pages/minute. The number of workers and the approximate overall speed for each plan are as follows:
- Free plan: 1 worker (approx. 5 pages/minute)
- Standard plan: 4 workers (approx. 20 pages/minute)
- Professional plan: 24 workers (approx. 120 pages/minute)
Please note that these speeds are the overall speed on your account. For example, if you have the Standard plan and run one project, it may use up to all 4 workers and run at a speed of up to 20 pages/minute. However, if you are running two projects, the workers will be distributed among these so each of the projects may be running at a speed of up to 10 pages/minute.
Can I run multiple projects at the same time (parallelize)?
Yes. If you are on a paid plan you can run more than one project at the same time.
Your workers will automatically be distributed across running projects, so on the Standard plan you can run up to 4 projects at a time (at a speed of 5 pages/minute each) and on the Professional plan you can run up to 24 projects at the same time (at a speed of 5 pages/minute each).
Note that if you run a project, it may take up all of your workers and any projects you try to run after that will be queued until more workers are available. You can limit the maximum number of workers a project can use by following the instructions below.
Can I limit the number of workers running on a project?
Yes. If you want to limit the number of workers on a project to ensure that more projects can run at the same time or to prioritize some projects over others, you can follow these instructions:
1. Go to your project and click on the "Settings" tab.
2. In the "Max Workers" section, choose the maximum number of workers you would like to allocate to this project. Note that "0" indicates that there are no limits (within the confines of your plan).
Can I see how many workers are being allocated to my project?
Yes. You can see how many workers are being used on each project you are running. To do this:
1. From the home page, click on Runs in the sidebar.
2. You will be able to see the number of workers in use and how many workers are being allocated to each run:
Why aren't my projects using all of my workers?
Typically, ParseHub will automatically allocate a number of workers to each project depending on the number of pages you are going to visit - so one may have 7 workers and the other 2, for example. You can always limit the number of workers on specific projects to prioritize others by following the instructions above.