How to use pagination() generator with 20 pages per request
complete
S
Samuel Haysom
I am trying to use the pagination feature based on the code at (https://github.com/serpapi/google-search-results-python#pagination-using-iterator). I want to request 20 results per API call but pagination by default iterates by 10 results only instead of 20, meaning I my requests end up overlapping. Is there a way to change the iteration interval?
Justin O'Hara
complete
Justin O'Hara
Hi Samuel Haysom we have pushed a new python package: https://pypi.org/project/google-search-results/2.4.0/ This will address the issues that you are currently having with that pagingation() generator
S
Samuel Haysom
Hi Justin O'Hara, I am using the google scholar api which can return 20 results per API call, hence why I want to iterate by this number in the pagination. The code to allow iteration by 20 results seems to be present but incomplete in the google-search-results-python package. I have opened an issue (https://github.com/serpapi/google-search-results-python/issues/24) in that Github repository to report the changes that need to be made in the code for it to work correctly
Justin O'Hara
Hi Samuel Haysom I understand you may want to request 20 results per API call but the maximum amount of results depends on what engine you are using for your querying. You can set up the
num
parameter to fit the 20 results on the Google Organic Search, but with an engine that provides less than 20 you would not be able to retrieve that many results. I have started a set of blogs to figure the maximum results per page. You can check out the blogs here: https://medium.com/serpapi/maximum-amount-of-results-per-page-across-serpapis-supported-google-engine-part-2-488669942a3e and https://medium.com/serpapi/maximum-amount-of-results-per-page-across-serpapis-supported-google-engines-7dfc5a4d52ccJustin O'Hara
under review
S
Samuel Haysom
I think I have found a solution to this. Looking in the package, the pagination.py file has a Pagination class which takes a page_size variable that changes the size of the offset between returned results.
The Pagination class is imported in the serp_api_client.py file within the pagination method starting on line 170 but here the page_size variable wasn't included. I just added page_size = 10 on lines 170 and 174 and now I can use the page_size variable if I call search.pagination(page_size = 20).
S
Samuel Haysom
There seems to be a pagination class in the python package that has a page_size variable but when I try and set this variable in my code the variable isn't recognised