{"id":130695,"date":"2022-12-20T11:15:35","date_gmt":"2022-12-20T11:15:35","guid":{"rendered":"https:\/\/blog.finxter.com\/?p=991056"},"modified":"2022-12-20T11:15:35","modified_gmt":"2022-12-20T11:15:35","slug":"the-hidden-gems-4-best-google-search-libraries-for-python-you-cant-miss","status":"publish","type":"post","link":"https:\/\/sickgaming.net\/blog\/2022\/12\/20\/the-hidden-gems-4-best-google-search-libraries-for-python-you-cant-miss\/","title":{"rendered":"The Hidden Gems: 4 Best Google Search Libraries for Python You Can\u2019t Miss!"},"content":{"rendered":"\n
\n
\n
\n
\n
<\/div>\n<\/p><\/div>\n
\n
<\/div>\n<\/p><\/div>\n
\n
<\/div>\n<\/p><\/div>\n
\n
<\/div>\n<\/p><\/div>\n
\n
<\/div>\n<\/p><\/div>\n<\/p><\/div>\n
\n
\n
<\/div>\n<\/p><\/div>\n
\n
<\/div>\n<\/p><\/div>\n
\n
<\/div>\n<\/p><\/div>\n
\n
<\/div>\n<\/p><\/div>\n
\n
<\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/div>\n
Rate this post<\/span> <\/div>\n<\/p><\/div>\n

A few days ago, a coding friend asked me: \u201cWhat is the best Google search library for Python?\u201d<\/strong><\/em><\/p>\n

Well, to be honest, I had no clue, so I did my investigation and did some quick testing. And I thought that it might be useful to share it with some Pythonistas newbies out of there.<\/p>\n

So let\u2019s review some great libraries to access the powerful search of a Google query within your Python code.<\/p>\n

Module 1: PyWhatKit<\/h2>\n

First, let\u2019s start simply by using the PyWhatKit Module:<\/p>\n

PyWhatKit is a Python Library that allows you to schedule and send WhatsApp messages and perform other functions such as playing a video on YouTube, converting an image to ASCII art, and converting a string to an image with handwritten characters. Other features include sending emails, taking screenshots, and shutting down or canceling a shutdown on a Linux or Mac OS machine.<\/p>\n

GitHub <\/strong>:\u00a0 https:\/\/github.com\/Ankit404butfound\/PyWhatKit<\/a><\/p>\n

But you can simply use it as well to run your favorite Google Queries.<\/p>\n

So let\u2019s start with the basics!<\/p>\n

How to install it:<\/p>\n

pip install pywhatkit\u00a0<\/pre>\n
First to test your code:<\/p>\n
import pywhatkit as pwk # To perform a Google search and to open your default browser automatically\nprint(\"Let's ask Google!\")\nsearch1 = \"FIFA world Cup\"\npwk.search(search1)<\/pre>\n
Now a better code that asks for the user\u2019s input<\/a>:<\/p>\n
# Importing the search function from the pywhatkit library\nfrom pywhatkit import search # Prompting for the user input query = input(\"Ask Google about? : \")\nprint(\"Searching for ...\")\n# Running the search query\nsearch(query)<\/pre>\n
You can run multiple queries, but this will open as many tabs in your browser. Maybe you have a use case for that?<\/p>\n
# To import the search function from the pywhatkit library\nfrom pywhatkit import search # Hardcoding your queries\nquery1 = \"Fifa\"\nquery2 = \"world cup\"\nquery3 = \"Mundial\"\nquery4 = \"world soccer\" # Searching at once\nsearch(query1)\nsearch(query2)\nsearch(query3)\nsearch(query4)<\/pre>\n
Ok, hold on! I can hear you saying but what about all Google search options. Well let\u2019s investigate another Python library: google<\/code>!<\/p>\n
Module 2: ‘Google’<\/h2>\n
To install it:<\/p>\n
pip install google<\/pre>\n
Now let’s ask the user what is looking for and return the result of all queries with a list of URLs. More useful when doing an investigation.<\/p>\n
# Importing the search function from the google library\nfrom googlesearch import search # Asking the user how many queries he wants to run num_searches = int(input(\"How many queries do you to do, (ie: 3): \"))\nsearchQueries = []\nwhile num_searches>0: # Then asking the user the subject of the search query query = input(\"Ask Google about? : \") # This will display a max of 5 results per query for i in search(query, tld=\"com\", num=5, stop=5, pause=3): print(i) num_searches=num_searches-1<\/pre>\n
Let\u2019s have a look at those search function options:<\/p>\n
query # a string which contain what you are looking for; ie: \"FIFA World cup\"\ntld = 'com', # The top level domain of google; ie: 'co.uk, fr' , ... lang = 'en', # Language of search result, 'ie: fr=French; gr=german',... num = 10, # Number of results per page\u00a0 start = 0, # First result to retrieve\u00a0 stop = None, # Last result to retrieve\u00a0 pause = 2.0, # Lapse between HTTP requests, this is important because if this value is too low Google can block your IP, so I recommend 2 or 3 seconds<\/pre>\n
You will get some results in the following format (for example):<\/p>\n
How many queries do you to do, (ie: 3): 1\nAsk Google about? : fifa<\/pre>\n
https:\/\/www.beinsports.com\/france\/fifa-coupe-du-monde-2022\/video\/coupe-du-monde-2022-lenorme-occasion-pour-le-\/2000861\nhttps:\/\/www.tf1.fr\/tf1\/fifa-coupe-du-monde-de-football\/videos\/giroud-le-patient-anglais-91919296.html\nhttps:\/\/www.sports.fr\/football\/equipe-de-france\/classement-fifa-injustice-vue-bleus-672799.html\nhttps:\/\/www.fifa.com\/\nhttps:\/\/twitter.com\/FIFAcom\/status\/1600418112830115841?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet<\/code>\n<\/pre>\n
Module 3: Google-API-Python-Client<\/h2>\n
Now, if you are interested in getting your results back in a JSON format, then the solution is to use the official Google API.<\/p>\n
Before using the Google Python library, you need to create a Google account and get your API key here<\/a>. You will need to create as well a service account in the Google Search Console<\/a>.<\/p>\n
I do recommend reading this blog<\/a> if you are struggling to create your account and keys.<\/p>\n
For the installat<\/a>i<\/a>on<\/a> of the library:<\/p>\n
pip install google-api-python-client<\/pre>\n
Now we are ready to create a script:<\/p>\n
from googleapiclient.discovery import build #Import the Google API library\nimport json #we will need it for the output of course<\/pre>\n
Then, you will need to create two variables that contain your token and key to be authenticated by Google.<\/p>\n
For example, they may look as follows (do not copy them, they are fake \"\ud83d\ude0a\"):<\/p>\n
my_api_key = \"AIzaSyAezbZKKKKKKr56r8kZk\"\nmy_cse_id = \"46c457999997\"\n\ufeff<\/pre>\n
Now let\u2019s create a function that we can call to do our search:<\/p>\n
def search(search_term, api_key, cse_id, **kwargs): service = build(\"customsearch\", \"v1\", developerKey=api_key) result = service.cse().list(q=search_term, cx=cse_id, **kwargs).execute() return result<\/pre>\n
In the second line, we are using the build()<\/code> command, which has many options because this Google API can be used for many things, but the one we are interested here is the: customsearch<\/code>.<\/p>\n
\"\ud83c\udf0d\" Resources<\/strong>: More information on all Google services is available here<\/a>. More information on the build<\/code> command here<\/a>.<\/p>\n
Let\u2019s try our function to see if everything is working fine:<\/p>\n
result = search(\"FIFA\", my_api_key, my_cse_id)\nprint(json.dumps(result, sort_keys=True, indent= 4))<\/pre>\n
Hold on, the output is not in a JSON file. No worries, let’s modify the last bit so you can save it<\/a> directly to a JSON file:<\/p>\n
result = search(\"FIFA\", my_api_key, my_cse_id)\njson_file = open(\"searchResult.json\", \"w\")\njson.dump(result, json_file)<\/pre>\n
Et voil\u00e0!<\/p>\n
Module 4: SerpAPI<\/h2>\n
There is another option if you do not want to use the official Google API. <\/p>\n
The google-search-results<\/code> library is not free for unlimited searches. But it can be free for 100 searches per day if you create an account at SerpApi.com<\/code>, and retrieve your API Key can add it to your code as you can guess!<\/p>\n
from serpapi import GoogleSearch params = { \"device\": \"desktop\", \"engine\": \"google\", \"q\": \"caf\u00e9\", \"location\": \"France\", \"google_domain\": \"google.fr\", \"gl\": \"fr\", \"hl\": \"fr\", \"api_key\": \"cfc232bade8efdb3956XXXXXXxx02251b63f7c751b001a800b7c\"\n} search = GoogleSearch(params)\nresults = search.get_dict()<\/pre>\n
You will get a nice JSON file<\/a> as a result. This library might be interesting for you if you want to do an internet search on other engines. Serapi.com<\/code> supports Bing, DuckDuckGo, Yahoo, Yandex, eBay, Youtube, and many more.<\/p>\n
I wanted to share this option as well, even though it is commercial. It can be useful for your use case.<\/p>\n
Finishing Up<\/h2>\n
Some fun to finish for all the fans of Google out there? \"\ud83d\ude0a\"<\/p>\n
By using the first library that we play with, you can transform any images into an ASCII<\/a> art file, special memories for the 80s.<\/p>\n
Please download any image by searching: \u201cI love Google\u201d<\/strong><\/em> and save it as a PNG.<\/p>\n
# To import the search function from the pywhatkit library\nimport pywhatkit as pwk pwk.image_to_ascii_art('loveGoogle.png', 'loveGascii.txt')<\/pre>\n
\n
\"\"<\/figure>\n<\/div>\n
Thanks for reading! \"\u2665\" <\/p>\n","protected":false},"excerpt":{"rendered":"
Rate this post A few days ago, a coding friend asked me: \u201cWhat is the best Google search library for Python?\u201d Well, to be honest, I had no clue, so I did my investigation and did some quick testing. And I thought that it might be useful to share it with some Pythonistas newbies out […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[857],"tags":[73,468,528],"class_list":["post-130695","post","type-post","status-publish","format-standard","hentry","category-python-tut","tag-programming","tag-python","tag-tutorial"],"_links":{"self":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/130695","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/comments?post=130695"}],"version-history":[{"count":0,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/130695\/revisions"}],"wp:attachment":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/media?parent=130695"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/categories?post=130695"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/tags?post=130695"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}