{"id":117751,"date":"2020-09-08T16:52:16","date_gmt":"2020-09-08T16:52:16","guid":{"rendered":"https:\/\/blog.finxter.com\/?p=12832"},"modified":"2020-09-08T16:52:16","modified_gmt":"2020-09-08T16:52:16","slug":"python-one-line-http-get","status":"publish","type":"post","link":"https:\/\/sickgaming.net\/blog\/2020\/09\/08\/python-one-line-http-get\/","title":{"rendered":"Python One Line HTTP Get"},"content":{"rendered":"<p><em>You may already know about Python&#8217;s capability to <a href=\"https:\/\/blog.finxter.com\/python-one-liner-webserver\/\" title=\"Python One-Liner Webserver HTTP\" target=\"_blank\" rel=\"noreferrer noopener\">create a simple web server in a single line<\/a> of Python code. Old news. Besides, what&#8217;s the point of creating a webserver that only runs on your machine? It would be far more interesting to learn how to access existing websites in a single line of code. Surprisingly, nobody talks about this in the <a href=\"https:\/\/pythononeliners.com\/\" target=\"_blank\" rel=\"noreferrer noopener\" title=\"https:\/\/pythononeliners.com\/\">Python One-Liners<\/a> community. Time to change it!<\/em><\/p>\n<p>This tutorial shows you how to perform simple <strong>HTTP get and post requests<\/strong> to an existing webserver!<\/p>\n<p><strong>Problem<\/strong>: Given the URL location of a webserver serving websites via HTTP. How to access the webserver&#8217;s response in a single line of Python code?<\/p>\n<p><strong>Example<\/strong>: Say, you want to accomplish the following:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">url = 'https:\/\/google.com'\n# ... Magic One-Liner Here...\nprint(result)\n# ... Google HTML file: '''\n&lt;!doctype html>&lt;html itemscope=\"\" itemtype=\"http:\/\/schema.org\/WebPage\" lang=\"de\">&lt;head>&lt;meta content=\"text\/html; charset=UTF-8\" http-equiv=\"Content-Type\">&lt;meta content=\"\/images\/branding\/googleg\/1x\/googleg_standard_color_128dp.png\" itemprop=\"image\">&lt;title>Google&lt;\/title>... '''<\/pre>\n<p>You can try it yourself in our interactive Python shell:<\/p>\n<p> <iframe loading=\"lazy\" height=\"400px\" width=\"100%\" src=\"https:\/\/repl.it\/@finxter\/KookyDeepskyblueHardware?lite=true\" scrolling=\"no\" frameborder=\"no\" allowtransparency=\"true\" allowfullscreen=\"true\" sandbox=\"allow-forms allow-pointer-lock allow-popups allow-same-origin allow-scripts allow-modals\"><\/iframe> <\/p>\n<p><em><strong>Exercise<\/strong>: Does this script download the complete source code of the Google.com website?<\/em><\/p>\n<p>Let&#8217;s learn about the three most important methods to access a website in a single line of Python code&#8212;and how they work!<\/p>\n<h2>Method 1: requests.get(url)<\/h2>\n<p>The simplest one-liner solution is the following:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">import requests; print(requests.get(url = 'https:\/\/google.com').text)<\/pre>\n<p>Here&#8217;s how this one-liner works:<\/p>\n<ul>\n<li>Import the Python library <a href=\"https:\/\/requests.readthedocs.io\/en\/master\/\" target=\"_blank\" rel=\"noreferrer noopener\" title=\"https:\/\/requests.readthedocs.io\/en\/master\/\"><code>requests<\/code> <\/a>that handles the details of requesting the websites from the server in an easy-to-process format.<\/li>\n<li>Use the <code>requests.get(...)<\/code> method to access the website and pass the URL <code>'https:\/\/google.com'<\/code> as an argument so that the function knows which location to access.<\/li>\n<li>Access the actual body of the get <code>request<\/code> (the return value is a request object that also contains some useful meta information like the file type, etc.).<\/li>\n<li><a href=\"https:\/\/blog.finxter.com\/the-separator-and-end-arguments-of-the-python-print-function\/\" target=\"_blank\" rel=\"noreferrer noopener\" title=\"Python Print Function [And Its SECRET Separator &amp; End Arguments]\">Print <\/a>the result to the shell.<\/li>\n<\/ul>\n<p>Note that the semicolon is used to <a href=\"https:\/\/blog.finxter.com\/how-to-write-multiple-statements-on-a-single-line-in-python\/\" target=\"_blank\" rel=\"noreferrer noopener\" title=\"How to Write Multiple Statements on a Single Line in Python?\">one-linerize this method<\/a>. This is useful if you want to run this command from your operating system with the following command:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">python -r \"import requests; print(requests.get(url = 'https:\/\/google.com').text)\"<\/pre>\n<p>The output is the desired Google website:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">'''\n&lt;!doctype html>&lt;html itemscope=\"\" itemtype=\"http:\/\/schema.org\/WebPage\" lang=\"de\">&lt;head>&lt;meta content=\"text\/html; charset=UTF-8\" http-equiv=\"Content-Type\">&lt;meta content=\"\/images\/branding\/googleg\/1x\/googleg_standard_color_128dp.png\" itemprop=\"image\">&lt;title>Google&lt;\/title>... '''<\/pre>\n<p>Note that you may have to install the requests library with the following command in your operating system terminal:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">pip install requests<\/pre>\n<p>A similar approach can be taken if you want to issue a post request:<\/p>\n<h2>Method 2: requests.post(url)<\/h2>\n<p>What if you want to post some data to a web resource? Use the post method of the <code>requests<\/code> module! Here&#8217;s a minimal one-liner example of the <code>request.post()<\/code> method:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">import requests as r; print(r.post('https:\/\/example.com', {'key': 'val'}).text)<\/pre>\n<p>The approach is similar to the first one:<\/p>\n<ul>\n<li>Import the <code>requests<\/code> module.<\/li>\n<li>Call the <code>r.post(...)<\/code> method.<\/li>\n<li>Pass the URL <code>'https:\/\/example.com'<\/code> as the first parameter into the function.<\/li>\n<li>Pass the value to be posted to the URL&#8212;in our case a simple key-value pair in a <a href=\"https:\/\/blog.finxter.com\/python-dictionary\/\" title=\"Python Dictionary \u2013 The Ultimate Guide\" target=\"_blank\" rel=\"noreferrer noopener\">dictionary <\/a>data structure. <\/li>\n<li>Access the body via the <code>text<\/code> attribute of the <code>request<\/code> object. <\/li>\n<li>Print it to the shell.<\/li>\n<\/ul>\n<h2>Method 3: urllib.request<\/h2>\n<p>A recommended way to <a href=\"https:\/\/docs.python.org\/3\/howto\/urllib2.html\" target=\"_blank\" rel=\"noreferrer noopener\" title=\"https:\/\/docs.python.org\/3\/howto\/urllib2.html\">fetch web resources<\/a> from a website is the <code>urllib.request()<\/code> function. This also works to create a simple one-liner to access the Google website in Python 3 as before:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">import urllib.request as r; print(r.urlopen('https:\/\/google.com').read())<\/pre>\n<p>It works similarly than before by returning a Request object that can be accessed to read the server&#8217;s response. We&#8217;re cramming everything into a single line so that you can run it from your OS&#8217;s terminal:<\/p>\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">python -r \"import urllib.request as r; print(r.urlopen('https:\/\/google.com').read())\"<\/pre>\n<p>Congrats! You now have mastered the art of accessing websites in a<a href=\"https:\/\/blog.finxter.com\/python-one-line-x\/\" target=\"_blank\" rel=\"noreferrer noopener\" title=\"Python One Line X\"> single line of Python code<\/a>. If you&#8217;re interested in boosting your one-liner power, have a look at my new book:<\/p>\n<h2>Python One-Liners Book<\/h2>\n<p><strong>Python programmers will improve their computer science skills with these useful one-liners.<\/strong><\/p>\n<figure class=\"wp-block-image size-medium is-resized\"><a href=\"https:\/\/www.amazon.com\/gp\/product\/B07ZY7XMX8\" target=\"_blank\" rel=\"noopener noreferrer\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/blog.finxter.com\/wp-content\/uploads\/2020\/06\/3D_cover-1024x944.jpg\" alt=\"Python One-Liners\" class=\"wp-image-10007\" width=\"512\" height=\"472\" srcset=\"https:\/\/blog.finxter.com\/wp-content\/uploads\/2020\/06\/3D_cover-scaled.jpg 1024w, https:\/\/blog.finxter.com\/wp-content\/uploads\/2020\/06\/3D_cover-300x277.jpg 300w, https:\/\/blog.finxter.com\/wp-content\/uploads\/2020\/06\/3D_cover-768x708.jpg 768w\" sizes=\"auto, (max-width: 512px) 100vw, 512px\" \/><\/a><\/figure>\n<p><a href=\"https:\/\/amzn.to\/2WAYeJE\" target=\"_blank\" rel=\"noreferrer noopener\" title=\"https:\/\/amzn.to\/2WAYeJE\"><em>Python One-Liners<\/em> <\/a>will teach you how to read and write &#8220;one-liners&#8221;: concise statements of useful functionality packed into a single line of code. You&#8217;ll learn how to systematically unpack and understand any line of Python code, and write eloquent, powerfully compressed Python like an expert.<\/p>\n<p>The book&#8217;s five chapters cover tips and tricks, regular expressions, machine learning, core data science topics, and useful algorithms. Detailed explanations of one-liners introduce key computer science concepts and boost your coding and analytical skills. You&#8217;ll learn about advanced Python features such as list comprehension, slicing, lambda functions, regular expressions, map and reduce functions, and slice assignments. You&#8217;ll also learn how to:<\/p>\n<p><strong>\u2022<\/strong>&nbsp;&nbsp;Leverage data structures to solve real-world problems, like using Boolean indexing to find cities with above-average pollution<br \/><strong>\u2022<\/strong>&nbsp;&nbsp;Use NumPy basics such as array, shape, axis, type, broadcasting, advanced indexing, slicing, sorting, searching, aggregating, and statistics<br \/><strong>\u2022<\/strong>&nbsp;&nbsp;Calculate basic statistics of multidimensional data arrays and the K-Means algorithms for unsupervised learning<br \/><strong>\u2022<\/strong>&nbsp;&nbsp;Create more advanced regular expressions using grouping and named groups, negative lookaheads, escaped characters, whitespaces, character sets (and negative characters sets), and greedy\/nongreedy operators<br \/><strong>\u2022<\/strong>&nbsp;&nbsp;Understand a wide range of computer science topics, including anagrams, palindromes, supersets, permutations, factorials, prime numbers, Fibonacci numbers, obfuscation, searching, and algorithmic sorting<\/p>\n<p>By the end of the book, you&#8217;ll know how to write Python at its most refined, and create concise, beautiful pieces of &#8220;Python art&#8221; in merely a single line.<\/p>\n<p><strong><a href=\"https:\/\/amzn.to\/2WAYeJE\" target=\"_blank\" rel=\"noreferrer noopener\" title=\"https:\/\/amzn.to\/2WAYeJE\"><em>Get your Python One-Liners Now!!<\/em><\/a><\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"<p>You may already know about Python&#8217;s capability to create a simple web server in a single line of Python code. Old news. Besides, what&#8217;s the point of creating a webserver that only runs on your machine? It would be far more interesting to learn how to access existing websites in a single line of code. [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[857],"tags":[73,468,528],"class_list":["post-117751","post","type-post","status-publish","format-standard","hentry","category-python-tut","tag-programming","tag-python","tag-tutorial"],"_links":{"self":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/117751","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/comments?post=117751"}],"version-history":[{"count":0,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/117751\/revisions"}],"wp:attachment":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/media?parent=117751"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/categories?post=117751"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/tags?post=117751"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}