Posted on Leave a comment

Add Google reCaptcha V3 Example with PHP

by Vincy. Last modified on August 11th, 2022.

Google reCaptcha V3 is the latest version provided with the highest security in comparison. Google contains different captcha services with reCaptcha V2. There are the (I am not a Robot) checkbox, invisible captcha and etc.

With the V3, Google guarantees zero friction while predicting the score for the website interactions. The score and the response report returned by the reCaptcha V3 is a very good security measure. It helps to take action accordingly to safeguard the website.

Like other Google reCaptcha concepts, the V3 also has more than one method to integrate the captcha challenge. Those are two as listed below.

  1. Programmatic invocation of the challenge.
  2. Automatic binding of the challenge with the form button.

This article explains to implement both methods by creating examples.

The below diagram will help to have a quick look at the Google reCaptcha V3 process flow.  Continue reading to learn how to get the API keys and integrate reCaptcha for your website.

Read also,

  1. PHP Contact Form with Google reCAPTCHA.
  2. Customize Google Invisible reCAPTCHA on a Webpage.

If you prefer custom captcha code using PHP, you may check the linked article for example code.

google recaptcha validation process

How to get Google reCaptcha keys

This is a two-step simple process to get the API keys to add Google reCaptcha to a website.

These steps are listed below with screenshots.

  1. Register your site domain.
  2. Copy the reCaptcha site key and the secret key.

Step1: Register your site domain

Go to the Google recaptcha admin console to register a domain to integrate reCaptcha V3.

The below screenshot masked the data related to the domain and site owner details. Enter your site details in the place of the masked data.

register domain create keys

Step2: Copy the reCaptcha site key and the secret key.

Once registration is done, the reCaptcha V3 keys are displayed below. Copy these details and configure them into the website code.

The site key is used to render the Google reCaptcha on the client side. And, the secret key is used for server-side verification.

The following example contains an application configuration file for this. Continue reading to know how to configure.

google recaptcha keys

About this example

This example renders the reCaptcha script and element in the landing UI. It has the configuration to make the website reCaptcha ready with the API keys.

It gets the token from the Google reCaptcha API and appends it to the form. On submit, the server-side PHP script receives the token to send the siteverify request to the API.

The reCaptcha API returns a JSON response with a score, success boolean, and more details. Based on the score (between 0 to 1), it helps to gauge the interaction’s standard. We have enabled a custom captcha solution based on the login validity standard.

It integrates the Google reCaptcha V3 programmatic challenge in the root. If you want to implement the “automatic binding” method, then it is in a separate folder.

The below structure shows the reCaptcha example files order and location. It will be helpful to set up this code correctly in a development environment.

google recaptcha php files

Application configuration file

It configures the Google reCaptcha V3 site and the secret keys used in the examples below.

The site key is used to render the Google recaptcha element on the client side. The secret key is used in the PHP files to build the site verification request parameters.

Config.php

<?php class Config
{ const GOOGLE_RECAPTCHA_SITE_KEY = ''; const GOOGLE_RECAPTCHA_SECRET_KEY = '';
} ?>

Method 1: Programatically invoking Google reCaptcha token via script

This method is used when the developer wants to have more programming control over the reCaptcha token.

During the explicit execution, it sets parameters to the request. These parameters can be returned with the Google reCaptcha V3 response. This will be helpful for additional verification.

HTML page renders form with reCaptcha JS

This landing page loads the JavaScript API while rendering the form UI. It uses the Google reCaptcha site key while loading the JavaScript.

index.php

<?php
require_once __DIR__ . '/Config.php';
?>
<html>
<head>
<link rel="stylesheet" type="text/css" href="css/form.css" />
<link rel="stylesheet" type="text/css" href="css/style.css" />
<title>Form with Google reCaptcha V3</title> <script src="https://www.google.com/recaptcha/api.js?render=<?php echo Config::GOOGLE_RECAPTCHA_SITE_KEY; ?>"></script>
</head>
<body> <div class="phppot-container tile-container"> <h3 class="text-center">Google reCaptcha V3</h3> <form id="frm" name="frm" method="post" onSubmit="getToken(event)"> <div> <div class="row"> <label>Feedback</label> <input type="text" name="txt_report" class="full-width" required> </div> <div class="row"> <input type="submit" value="Send" class="full-width"> </div> <div id="ack-message"></div> </div> </form> </div>
</body>
</html>

Execute reCaptcha JavaScript API for token

It executes the Google reCaptcha request explicitly. The callback function of this action will return the reCaptcha token.

Then, the callback dynamically creates an input element using JavaScript. It loads the token to this element and appends it to the form.

After getting the token field, the form is submitted via JavaScript. The submitForm function posts the form data to the PHP via AJAX. In a previous article, we have seen how to enable PHP custom captcha using AJAX.

index.php (Javascript Google reCaptcha execute)

// Execute Google reCaptcha v3 to get token function getToken(event) { event.preventDefault(); grecaptcha.ready(function() { grecaptcha.execute('<?php echo Config::GOOGLE_RECAPTCHA_SITE_KEY; ?>', { action: 'submit' }).then(function(token) { var button = document.createElement('input'); button.type = 'hidden'; button.name = 'recaptcha_token'; button.id = 'recaptcha_token'; button.value = token; var form = document.getElementById("frm"); form.appendChild(button); submitForm(); });; });
} // Submit reCaptcha token to the PHP
function submitForm() { const form = document.getElementById('frm'); const formData = new FormData(form); var xhttp = new XMLHttpRequest(); xhttp.open('POST', 'form-action.php', true); xhttp.send(formData); xhttp.onreadystatechange = function() { if (xhttp.readyState == 4 && xhttp.status == 200) { document.getElementById("ack-message").innerHTML = xhttp.responseText; document.getElementById('recaptcha_token').remove(); } }
}

Verify website interaction using Google reCaptcha V3 API

The form submits action calls this PHP script by sending the reCaptcha token. It builds the post parameters with the Google reCaptcha v3 secret key and token.

This PHP script uses cURL to post the request to the reCaptcha API. The cUrl response returns the JSON  data as given by the Google API.

It returns the score about the interaction made on the website. This score will be between 0.0 (lower) and 1.1(higher) ranking. It helps to predict the necessary steps to make to protect the site.

form-action.php

//PHP reCaptcha validation
<?php
require_once __DIR__ . '/Config.php'; $reCaptchaToken = $_POST['recaptcha_token'];
$postArray = array( 'secret' => Config::GOOGLE_RECAPTCHA_SECRET_KEY, 'response' => $reCaptchaToken
); $postJSON = http_build_query($postArray); $curl = curl_init();
curl_setopt($curl, CURLOPT_URL, "https://www.google.com/recaptcha/api/siteverify");
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_POSTFIELDS, $postJSON);
$response = curl_exec($curl);
curl_close($curl);
$curlResponseArray = json_decode($response, true); if ($curlResponseArray["success"] == true && ! empty($curlResponseArray["action"]) && $curlResponseArray["score"] >= 0.5) { mail("admin@site.com", "New report", $_POST["txt_report"]); $output = "<div id='phppot-message' class='success'>Feedback received.</div>";
} else { $output = "<div id='phppot-message' class='error'>Invalid request.</div>";
}
print $output;
exit();

Method 2: Automatic binding callback with the submit button

This is a basic and simple method of integrating Google reCaptcha V3 for a site. In this automatic binding of the reCaptcha challenge, it gets the token in the callback.

Site HTML to bind the reCaptcha challenge automatically

It loads the Google reCaptcha JavaScript API as we did in method 1.

The g-recaptcha field binds the callback, action, and reCaptcha site key with the HTML5 data attributes.

 automatic-binding/index.php

<?php
session_start();
require_once __DIR__ . '/../Config.php';
?>
<html>
<head>
<link rel="stylesheet" type="text/css" href="./../css/form.css" />
<link rel="stylesheet" type="text/css" href="./../css/style.css" />
<title>Form with Google reCaptcha V3</title> <script src="https://www.google.com/recaptcha/api.js"></script>
</head>
<body> <div class="phppot-container tile-container"> <h3 class="text-center">Google reCaptcha Automatic Binding</h3> <form id="frm" name="frm" method="post" action="action.php"> <div> <div class="row"> <label>Feedback</label> <input type="text" name="txt_report" class="full-width" required> </div> <div class="row"> <input type="button" class="g-recaptcha full-width" data-sitekey="<?php echo Config::GOOGLE_RECAPTCHA_SITE_KEY; ?>" data-callback='onSubmit' data-action='submit' value="Send" /> </div> <?php if (! empty($_SESSION["ack_message"])) { ?> <div id="ack-message"><?php echo $_SESSION["ack_message"]; ?></div> <?php } $_SESSION["ack_message"] = ""; ?> </div> </form> </div>
</body>
</html>

JavaScript callback to append token field to the form

This callback function is linked with the Google reCaptcha element, which is the send button of the form. So, on clicking the send button, it calls the onSubmit JavaScript function.

This callback has the reCaptcha token to be appended to the form data.

automatic-binding/index.php (JavaScript callback)

// JavaScript
function onSubmit(token) { var button = document.createElement('input'); button.type = 'hidden'; button.name = 'recaptcha_token'; button.value = token; var form = document.getElementById("frm"); form.appendChild(button); form.submit();
}

PHP action to predict Google reCaptcha score

In PHP, it verifies the site and checks the interaction score. It contains the same code as the form-action.php we used in method 1.

The difference is that the response is sent via session instead of printing it to AJAX callback.

automatic-binding/action.php

//Google reCaptcha V3 server-side verification
<?php
require_once __DIR__ . '/Config.php'; $reCaptchaToken = $_POST['recaptcha_token']; $postArray = array( 'secret' => Config::GOOGLE_RECAPTCHA_SECRET_KEY, 'response' => $reCaptchaToken
); $postJSON = http_build_query($postArray); $curl = curl_init();
curl_setopt($curl, CURLOPT_URL, "https://www.google.com/recaptcha/api/siteverify");
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_POSTFIELDS, $postJSON);
$response = curl_exec($curl);
curl_close($curl); $curlResponseArray = json_decode($response, true); if ($curlResponseArray["success"] == true && $curlResponseArray["score"] >= 0.5) { mail("admin@site.com", "New report", $_POST["txt_report"]); $output = "<div id='phppot-message' class='success'>Feedback received.</div>";
} else { $output = "<div id='phppot-message' class='error'>Invalid request.</div>";
} $_SESSION["ack_message"] = $output;
header("Location: automatic-binding.php");
?>

Download

↑ Back to Top

Posted on Leave a comment

A Simple Guide for Using Command Line Arguments in Python

5/5 – (2 votes)
YouTube Video

In this lesson, we will learn several methods for using arguments in the command line and how we can manipulate them to run in our pre-written Python scripts.

The three methods we will explore and compare are:

  • sys.argv
  • argparse
  • getopt

These are ordered for ease of use and simplicity. 

I’ve added the getopt() method for demonstration purposes and have included it at the end because I find it the least useful of the three – you may have a different opinion, so check it out and make your own conclusions.

Method 1- sys.argv

First, we will need to import the sys module – “System- specific parameters and functions.”

argv stands for “argument vector”, and is basically a variable that contains arguments passed through the command line.

I’m using the VScode text editor for my scripts, as you can see in the file path, and then follow that by calling “Python” before the actual file name and arguments.  This will be the same with each method. 

You can abbreviate Python as py after vscode if you wish to save on typing and that will work fine.

import sys print('What is the name of the script?', sys.argv[0])
print('How many arguments?', len(sys.argv))
print('What are the arguments?', str(sys.argv))
# Adding arguments on command line
(base) PS C:\Users\tberr\.vscode> python test_command.py 3 4 5 6

Output:

What is the name of the Script? test_command.py
How many arguments? 5
What are the arguments? ['test_command.py', '3', '4', '5', '6']

We can see that the file name is the first argument located at the [0] index position and the four integers are located at index[1:] (1, 2, 3, 4) in our list of strings.

Now let’s do some code that is a little more involved.

Simple script for adding numbers with the numbers entered on the command line.

import sys # total arguments
n = len(sys.argv)
print("Total arguments passed:", n) # Arguments passed
print("\nName of Python script:", sys.argv[0]) print("\nArguments passed:", end = " ")
for i in range(1, n): print(sys.argv[i], end = " ") # Addition of numbers
Sum = 0
# Using argparse module (we will talk about argparse next)
for i in range(1, n): Sum += int(sys.argv[i]) print("\n\nResult:", Sum)

Output with arguments entered on command line:

(base) PS C:\Users\tberr\.vscode> python test_command.py 4 5 7 8
Total arguments passed: 5 Name of Python script: test_command.py Arguments passed: 4 5 7 8 Result: 24

This gives the user the total arguments passed, the name of the script, arguments passed (not including script name), and the “Sum” of the integers.  Now let’s get to the argparse method.

💡 Note: If you are getting an ‘error’ or different results than expected when you pass arguments on the command line, make sure that the file you’re calling has been saved. If your Python file has been changed or is new it will not work until you do so.

Method 2 – argparse

Parser for command-line options, arguments, and sub-commands.

argparse is recommended over getopt because it is simpler and uses fewer lines of code.

Code:

import argparse # Initialize the parser
parser = argparse.ArgumentParser(description = 'process some integers.') # Adding Arguments
parser.add_arguments('integers', metavar = 'N', type = int, nargs = '+', help = 'an integer for the accumulator') parser.add_arguments(dest = 'accumulate', action = 'store_const", const = sum, help = 'sum the integers') args = parser.parse_args()
print(args.accumulate(args.integers))

We see that first, we initialize the parser and then add arguments with the ‘parser.add_arguments’ section of the code. 

We also add some help messages to guide the user on what is going on with the script.  This will be very clear when we enter arguments on the command line and see our output.

Output:

# Add arguments on the command line. -h (for help) and four integers (base) PS C:\Users\tberr\.vscode> python argparse.py -h 5 3 6 7
usage: argparse.py [-h] N [N ...] Process some integers. positional arguments: N an integer for the accumulator accumulate sum the integers optional arguments: -h, – help show this help message and exit # Run code again without the help argument, just sum the integers.
(base) PS C:\Users\tberr\.vscode> python argParse.py 5 3 6 7
21

This is an excellent, clean way to pass arguments on the command line, and the addition of the ‘help’ argument can make this very clear for the user.

For more details on arguments like [‘metavar’], [‘const’], [‘action’], and [‘dest’], check out this LINK

Method 3 – getopt

A method for parsing command line options and parameters, very similar to the getopt() function in the C language.  This is some basic code to get the name of the user on the command line.

Code:

import sys
import getopt def full_name(): first_name = None last_name = None argv = sys.argv[1:] try: opts, args = getopt.getopt(argv, "f:l:") except: print("Error") for opt, arg in opts: if opt in ['-f']: first_name = arg elif opt in ['-l']: last_name = arg print( first_name +" " + last_name) full_name() 

We have set arguments ‘f’ and ‘l’ for first and last name, and will pass them in the command line arguments.

Output in command line:

(base) PS C:\Users\tberr\.vscode> py getOpt.py -f Tony -l Berry Tony Berry

This is certainly a lot of code to get such a simple result as ‘Full Name’, and is the reason I prefer both the sys.argv and argparse modules over getopt.  That doesn’t mean you won’t find some value in the getopt module, this is simply my preference.

Summary

These are all powerful Python tools that can be helpful when users want to interact with your code and can make the process simple and clear. 

We have covered the basics here to get you started and give you an idea of a few built-in modules of Python. 

Good luck with your Python coding career!


Posted on Leave a comment

How to Convert Tab-Delimited File to CSV in Python?

5/5 – (1 vote)

The easiest way to convert a tab-delimited values (TSV) file to a comma-separated values (CSV) file is to use the following three lines of code:

  1. import pandas as pd
  2. df = pd.read_csv('my_file.txt', sep='\t', header=None)
  3. df.to_csv('my_file.csv', header=None)

We’ll explain this and other approaches in more detail next—scroll down to Method 3 for this exact method.

Problem Formulation

Given a tab-delimited file with one tab character '\t' between two values in a given column.

Input: 'my_file.tsv'

Figure: File 'my_file.tsv' with tab '\t' separated values.
Alice	DataScience	$100000
Bob	Programmer	$90000
Carl	Manager	$122000
Dave	Freelancer	$144000

How to convert the tab-delimited values (TSV) to a comma-separated values (CSV) file?

Output: 'my_file.csv'

0,Alice,DataScience,$100000
1,Bob,Programmer,$90000
2,Carl,Manager,$122000
3,Dave,Freelancer,$144000

We’ll also look at slight variations of this problem. Let’s go!

Method 1: String Replace Single Tab

The most straightforward way to convert a tab-delimited (TSV) to a comma-separated (CSV) file in Python is to replace each tabular character '\t' with a comma ',' character using the string.replace() method. This works if two values are separated by exactly one tabular character.

Here’s an example input file 'my_file.tsv':

Here’s an example of some code to convert the tab-delimited file to the CSV file:

with open('my_file.tsv') as f: # Read space-delimited file and replace all empty spaces by commas data = f.read().replace('\t', ',') # Write the CSV data in the output file print(data, file=open('my_file.csv', 'w'))

Output file 'my_file.csv':

If you have any doubts, feel free to dive into our related tutorials:

Method 2: Regex Replace Arbitrary Tabs

To replace one '\t' or more tabs '\t\t\t' between two column values with a comma ',' and obtain a CSV, use the regular expressions operation re.sub('[\t]+', ',', data) on the space-separated data.

If you have any doubts, feel free to dive into our related tutorials:

Here’s an example input file 'my_file.tsv', notice the additional tabular characters that may separate two column values:

Here’s an example of some code to convert the TSV to the CSV file:

import re with open('my_file.txt') as infile: # Read space-delimited file and replace all empty spaces by commas data = re.sub('[ ]+', ',', infile.read()) # Write the CSV data in the output file print(data, file=open('my_file.csv', 'w'))

Output file 'my_file.csv':

Method 3: Pandas read_csv() and to_csv()

To convert a tab-delimited file to a CSV, first read the file into a Pandas DataFrame using pd.read_csv(filename, sep='\t+', header=None) and then write the DataFrame to a file using df.to_csv(outfilename, header=None).

Here’s an example input file 'my_file.tsv':

Here’s an example of some code to convert the tab-delimited file to the CSV file:

import pandas as pd # Read space-delimited file
df = pd.read_csv('my_file.tsv', sep='\t+', header=None) # Write DataFrame to file
df.to_csv('my_file.csv', header=None)

Output file 'my_file.csv':

You can also use the simpler sep='\t' if you are sure that only a single tabular character separates two column values.

If you have any doubts, feel free to dive into our related tutorials:

Summary

We examined three great ways to convert a space-delimited to a comma-separated CSV file:

Thanks for taking the time to read this article, my friend! 🐍💛


Regex Humor

Wait, forgot to escape a space. Wheeeeee[taptaptap]eeeeee. (source)
Posted on Leave a comment

JavaScript Autocomplete TextBox (autosuggest) from Database

by Vincy. Last modified on August 9th, 2022.

AutoComplete is a feature to suggest relevant results on typing into a textbox. For example, Google search textbox autosuggest search phrases on keypress.

It can be enabled using client-side tools and attributes. The data for the autosuggest textbox can be static or dynamic.

For loading remote data dynamically, the source possibility is either files or databases. This article uses the database as a source to have dynamic results at the backend.

The below example has an idea for a quick script for enabling the autocomplete feature. It uses JavaScript jQuery and jQuery UI libraries to implement this easily.

The jQuery autocomplete() uses the PHP endpoint autocomplete.php script. Then, load the remote data into the textbox on the UI.

View Demo

Example 1: Simple autocomplete

Quick example

<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.13.2/themes/base/jquery-ui.min.css" />
<script src="https://code.jquery.com/jquery-3.6.0.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.13.2/jquery-ui.min.js"></script>
<script>
$(document).ready(function(){ $( "#textbox" ).autocomplete({ source: "autocomplete.php", minLength: 2 });
});
</script>
<input id="textbox" class="full-width" />

This PHP endpoint script reads the database results and forms the output JSON for the autocomplete textbox.

It receives the searched term from the UI and looks into the database for relevant suggestions.

autocomplete.php

<?php
$name = $_GET['term'];
$name = "%$name%";
$conn = mysqli_connect('localhost', 'root', '', 'phppot_autocomplete');
$sql = "SELECT * FROM tbl_post WHERE title LIKE ?";
$statement = $conn->prepare($sql);
$statement->bind_param('s', $name);
$statement->execute();
$result = $statement->get_result();
$autocompleteResult = array();
if (! empty($result)) { while ($row = $result->fetch_assoc()) { $autocompleteResult[] = $row["title"]; }
}
print json_encode($autocompleteResult);
?>

This database is for setting up the database created for this quick example. The next example also needs this database for displaying the autosuggest values.

Run the below database queries for getting a good experience with the above code execution.

CREATE TABLE `tbl_post` ( `id` int(11) UNSIGNED NOT NULL, `title` text DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; --
-- Dumping data for table `tbl_post`
-- INSERT INTO `tbl_post` (`id`, `title`) VALUES
(1, 'Button on click event capture.'),
(2, 'On key press action.'),
(3, 'Overlay dialog window.);

javascript autocomplete
Example 2: Load autocomplete with ID

The AutoComplete function sends an additional parameter with the default term argument. That is to limit the number of results shown in the autocomplete textbox.

It returns the database results based on the searched term as a key-value pair. A JavaScript callback iterates the result and maps the key-value as label-value pair.

It is helpful when the result id is required while selecting a particular item from the autosuggest list.

The below screenshot shows the item value and id is populated. This data is put into the textbox on selecting the autocomplete list item.

autocomplete result with id

The below JavaScript code has two textboxes. One textbox is enabled with the autocomplete feature.

On typing into that textbox, the JavaScript autocomplete calls the server-side PHP script. The callback gets the JSON output returned by the PHP script.

This JSON data contains an association of dynamic results with their corresponding id. On selecting the autocomplete result item, the select callback function access the UI.item object.

Using this object, it gets the id and post title from the JSON data bundle. Then this JavaScript callback function targets the UI textboxes to populate the title and id of the selected item.

<script>
$(document).ready(function() { $("#textbox").autocomplete({ minlength: 3, source: function(request, response) { $.ajax({ url: "get-result-by-additional-param.php", type: "POST", dataType: "json", data: { q: request.term, limit: 10 }, success: function(data) { response($.map(data, function(item) { return { label: item.title, value: item.postId }; })); } }); }, select: function(event, ui) { event.preventDefault(); $('#textbox').val(ui.item.label); $('#itemId').val(ui.item.value); } });
});
</script>
<div class="row"> <label>Type for suggestion</label> <input id="textbox" class="full-width" />
</div>
<div class="row"> <label>Item id</label> <input id="itemId" class="full-width" />
</div>

This PHP script receives the post parameters sent via the autocomplete function.

The search keyword and the result limit are sent from the source callback of the autocomplete initiation.

This PHP script substitutes those parameters into the database query execution process.

Once found the results, it bundles the array into a JSON format to print as an auto-suggestion list.

get-result-by-additional-param.php

<?php
$name = $_POST['q'];
$limit = $_POST['limit'];
$name = "%$name%";
$conn = mysqli_connect('localhost', 'root', '', 'phppot_autocomplete');
$sql = "SELECT * FROM tbl_post WHERE title LIKE ? LIMIT $limit";
$statement = $conn->prepare($sql);
$statement->bind_param('s', $name);
$statement->execute();
$result = $statement->get_result();
$autocompleteResult = array();
if (! empty($result)) { $i = 0; while ($row = $result->fetch_assoc()) { $autocompleteResult[$i]["postId"] = $row["id"]; $autocompleteResult[$i]["title"] = $row["title"]; $i ++; }
}
print json_encode($autocompleteResult);
?>

Example 3: AutoComplete with recent search

This example shows the autocomplete box with text and image data. The database for this example contains additional details like description and featured_image for the posts.

If you want a sleek and straightforward autocomplete solution with text, then use the above two examples.

This example uses BootStrap and plain JavaScript without jQuery. It displays recent searches on focusing the autocomplete textbox.

Create AutoComplete UI with Bootstrap and JavaScript Includes

See this HTML loads the autocomplete textbox and required JavaScript and CSS assets for the UI. The autocomplete.js handles the autosuggest request raised from the UI.

The autocomplete textbox has the onKeyPress and onFocus attributes. The onKeyPress attribute calls JavaScript to show an autosuggest list. The other attribute is for displaying recent searches on the focus event of the textbox.

autocomplete-with-search-history/index.php

<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.2.0/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-gH2yIJqKdNHPEq0n4Mqa/HGKIhSkIHeL5AyhkYV8i59U5AR6csBvApHHNl/vI1Bx" crossorigin="anonymous">
<script src="./assets/autocomplete.js"></script>
<style>
.post-icon { width: 50px; height: 50px; border-radius: 50%; margin-right: 15px;
} .remove-link { font-size: 0.75em; font-style: italic; color: #0000FF; cursor: pointer;
}
</style>
<input id="textbox" class="form-control" onkeyup="showSuggestionList(this.value)" onfocus="showRecentSearch()" autocomplete="off" />
<span id="auto-suggestion-box"></span>

Get the autosuggest list from the tbl_post database table

The below JavaScript function is called on the keypress event of the autocomplete field. In the previous examples, it receives a JSON response to load the dynamic suggestion.

In this script, it receives the HTML response from the endpoint. This HTML is with an unordered list of autosuggest items.

function showSuggestionList(searchInput) { if (searchInput.length &gt; 1) { var xhttp = new XMLHttpRequest(); xhttp.open('POST', 'ajax-endpoint/get-auto-suggestion.php', true); xhttp.setRequestHeader(&quot;Content-type&quot;, &quot;application/x-www-form-urlencoded&quot;); xhttp.send(&quot;formData=&quot; + searchInput); xhttp.onreadystatechange = function() { if (xhttp.readyState == 4 &amp;&amp; xhttp.status == 200) { document.getElementById('auto-suggestion-box').innerHTML = xhttp.responseText; } } } else { document.getElementById('auto-suggestion-box').innerHTML = ''; }
}

ajax-endpoint/get-auto-suggestion.php

<?php
require_once __DIR__ . '/../lib/DataSource.php';
$dataSource = new DataSource(); if (isset($_POST["formData"])) { $searchInput = filter_var($_POST["formData"], FILTER_SANITIZE_STRING); $highlight = '<b>' . $searchInput . '</b>'; $query = "SELECT * FROM tbl_post WHERE title LIKE ? OR description LIKE ? ORDER BY id DESC LIMIT 15"; $result = $dataSource->select($query, 'ss', array( "%" . $searchInput . "%", "%" . $searchInput . "%" )); if (! empty($result)) { ?>
<ul class="list-group">
<?php foreach ($result as $row) { ?> <li class="list-group-item text-muted" data-post-id="<?php echo $row["id"]; ?>" onClick="addToHistory(this)" role="button"><img class="post-icon" src="<?php echo $row["featured_image"]; ?>" /><span> <?php echo str_ireplace($searchInput, $highlight, $row["title"]); ?> </span></li>
<?php } ?>
</ul>
<?php }
}
?>

javascript autocomplete without jquery

Add to search history

When selecting the suggested list item, it triggers this JavaScript function on click.

This function reads the post id and title added to the HTML5 data attribute. Then passes these details to the server-side PHP script.

function addToHistory(obj) { var selectedResult = obj.dataset.postId; fetch("ajax-endpoint/add-to-history.php", { method: "POST", body: JSON.stringify({ selectedResult: selectedResult }) }).then(function() { document.getElementById('textbox').value = obj.innerText; });
}

This PHP endpoint checks if the selected item is already added to the database history table.

In the database, the tbl_search_history stores the search history.

If data is not found in the database, then the search instance will be added to this table.

ajax-endpoint/add-to-history.php

<?php
require_once __DIR__ . '/../lib/DataSource.php';
$dataSource = new DataSource(); $post_data = json_decode(file_get_contents('php://input'), true);
$selectedResult = filter_var($post_data['selectedResult'], FILTER_SANITIZE_STRING);
if (isset($selectedResult)) { $query = "SELECT * FROM tbl_search_history, tbl_post WHERE tbl_search_history.post_id = tbl_post.id AND tbl_post.id = ?"; $result = $dataSource->select($query, 'i', array( $selectedResult )); if (empty($result)) { $query = " INSERT INTO tbl_search_history (post_id) VALUES (?)"; $result = $dataSource->insert($query, 'i', array( $selectedResult )); }
}
?>

Show search history by focusing on the autocomplete textbox

This function calls the PHP endpoint to fetch the stored search history. It also receives the HTML response from the server side.

The response HMTL is loaded into the autosuggest textbox in the UI.

function showRecentSearch() { if (!(document.getElementById('textbox').value)) { fetch("ajax-endpoint/show-search-history.php", { method: "POST" }).then(function(response) { return response.text(); }).then(function(responseData) { if (responseData != "") { document.getElementById('auto-suggestion-box').innerHTML = responseData; } }); }
}

In this PHP file, it joins the tbl_post and the tbl_search_history database tables. It is to filter the already searched keyword list.

ajax-endpoint/show-search-history.php

<?php
require_once __DIR__ . '/../lib/DataSource.php';
$dataSource = new DataSource(); $post_data = json_decode(file_get_contents('php://input'), true);
$query = "SELECT tbl_post.* FROM tbl_search_history, tbl_post WHERE tbl_search_history.post_id = tbl_post.id ORDER BY id DESC LIMIT 10"; $result = $dataSource->select($query); ?>
<ul class="list-group">
<?php foreach ($result as $row) { ?> <li class="list-group-item text-muted" role="button"><img class="post-icon" src="<?php echo $row["featured_image"]; ?>" /><span><?php echo $row["title"]; ?></span> <span title="Remove from history" class="remove-link" onClick="removeFromHistory(this, <?php echo $row["id"]; ?>)">[remove]</span></li>
<?php } ?>
</ul>

Remove history from the autosuggest textbox

When focusing on the autocomplete textbox, the UI will display the recently searched post titles.

If the user wants to remove the recent searches, it is possible by this code.

The autosuggest entries have the remove link in the UI. On clicking the link, the corresponding record will be deleted.

function removeFromHistory(obj, postId) { fetch("ajax-endpoint/remove-history.php", { method: "POST", body: JSON.stringify({ postId: postId }) }).then(function() { obj.parentNode.remove(); });
}

This PHP code removes the search instances stored in the tbl_search_history database. The delete request posts the record id to fire the delete action.

ajax-endpoint/remove-history.php

<?php
require_once __DIR__ . '/../lib/DataSource.php';
$dataSource = new DataSource(); $post_data = json_decode(file_get_contents('php://input'), true);
$postId = filter_var($post_data['postId'], FILTER_SANITIZE_STRING);
$query = " DELETE FROM tbl_search_history WHERE post_id = ?"; $result = $dataSource->insert($query, 'i', array( $postId
));
?>

autocomplete with search history
View DemoDownload

↑ Back to Top

Posted on Leave a comment

How to Convert a List of Dicts to a CSV File in Python [4 Ways]

5/5 – (1 vote)

Problem: How to convert a list of dictionaries to a csv file?

Example: Given is a list of dicts—for example salary data of employees in a given company:

salary = [{'Name':'Alice', 'Job':'Data Scientist', 'Salary':122000}, {'Name':'Bob', 'Job':'Engineer', 'Salary':77000}, {'Name':'Carl', 'Job':'Manager', 'Salary':119000}]

Your goal is to write the content of the list of dicts into a comma-separated-values (CSV) file format. Your out file should look like this:

my_file.csv:

Name,Job,Salary
Alice,Data Scientist,122000
Bob,Engineer,77000
Ann,Manager,119000

Solution: There are four simple ways to convert a list of dicts to a CSV file in Python.

  1. Pandas: Import the pandas library, create a Pandas DataFrame, and write the DataFrame to a file using the DataFrame method DataFrame.to_csv('my_file.csv').
  2. CSV: Import the csv module in Python, create a CSV DictWriter object, and write the list of dicts to the file in using the writerows() method on the writer object.
  3. Python: Use a pure Python implementation that doesn’t require any library by using the Python file I/O functionality.
  4. Reduce Problem: You can first convert the list of dicts to a list of lists and then use our related tutorial’s methods to write the list of lists to the CSV.

My preference is Method 1 (Pandas) because it’s simplest to use, concise, and most robust for different input types (numerical or textual).

Method 1: Pandas DataFrame to_csv()

You can convert a list of lists to a Pandas DataFrame that provides you with powerful capabilities such as the to_csv() method. This is the easiest method and it allows you to avoid importing yet another library (I use Pandas in many Python projects anyways).

salary = [{'Name':'Alice', 'Job':'Data Scientist', 'Salary':122000}, {'Name':'Bob', 'Job':'Engineer', 'Salary':77000}, {'Name':'Carl', 'Job':'Manager', 'Salary':119000}] # Method 1
import pandas as pd
df = pd.DataFrame(salary)
df.to_csv('my_file.csv', index=False, header=True)

Output:

Name,Job,Salary
Alice,Data Scientist,122000
Bob,Engineer,77000
Carl,Manager,119000

You create a Pandas DataFrame—which is Python’s default representation of tabular data. Think of it as an Excel spreadsheet within your code (with rows and columns).

The DataFrame is a very powerful data structure that allows you to perform various methods. One of those is the to_csv() method that allows you to write its contents into a CSV file.

  • You set the index argument of the to_csv() method to False because Pandas, per default, adds integer row and column indices 0, 1, 2, …. Again, think of them as the row and column indices in your Excel spreadsheet. You don’t want them to appear in the CSV file so you set the arguments to False.
  • You set the and header argument to True because you want the dict keys to be used as headers of the CSV.

If you want to customize the CSV output, you’ve got a lot of special arguments to play with. Check out this article for a comprehensive list of all arguments.

🌍 Related article: Pandas Cheat Sheets to Pin to Your Wall

Method 2: Python CSV Module DictWriter

You can convert a list of dicts to a CSV file in Python easily—by using the csv library. This is the most customizable of all four methods.

Here are the six easy steps to convert a list of dicts to a CSV with header row:

  1. Import the CSV library with import csv.
  2. Open the CSV file using the expression open('my_file.csv', 'w', newline=''). You need the newline argument because otherwise, you may see blank lines between the rows in Windows.
  3. Create a csv.DictWriter() object passing the file and the fieldnames argument.
  4. Set the fieldnames argument to the first dictionary’s keys using the expression salary[0].keys().
  5. Write the header using writer.writeheader()
  6. Write the list of dicts using writer.writerows()

Here’s the full code for copy&paste:

salary = [{'Name':'Alice', 'Job':'Data Scientist', 'Salary':122000}, {'Name':'Bob', 'Job':'Engineer', 'Salary':77000}, {'Name':'Carl', 'Job':'Manager', 'Salary':119000}] # Method 2
import csv
with open('my_file.csv', 'w', newline='') as f: writer = csv.DictWriter(f, fieldnames=salary[0].keys()) writer.writeheader() writer.writerows(salary)

Output file named 'my_file.csv' and located in the same folder:

Name,Job,Salary
Alice,Data Scientist,122000
Bob,Engineer,77000
Carl,Manager,119000 

You can customize the CSV writer in its constructor (e.g., by modifying the delimiter from a comma ',' to a whitespace ' ' character). Have a look at the specification to learn about advanced modifications.

Method 3: Pure Python Without External Dependencies

If you don’t want to import any library and still convert a list of dicts into a CSV file, you can use standard Python implementation as well: it’s not complicated and very efficient.

This method is best if you won’t or cannot use external dependencies.

  • Open the file f in writing mode using the standard open() function.
  • Write the first dictionary’s keys in the file using the one-liner expression f.write(','.join(salary[0].keys())).
  • Iterate over the list of dicts and write the values in the CSV using the expression f.write(','.join(str(x) for x in row.values())).

Here’s the concrete code example:

salary = [{'Name':'Alice', 'Job':'Data Scientist', 'Salary':122000}, {'Name':'Bob', 'Job':'Engineer', 'Salary':77000}, {'Name':'Carl', 'Job':'Manager', 'Salary':119000}] # Method 3
with open('my_file.csv','w') as f: f.write(','.join(salary[0].keys())) f.write('\n') for row in salary: f.write(','.join(str(x) for x in row.values())) f.write('\n')

Output:

Name,Job,Salary
Alice,Data Scientist,122000
Bob,Engineer,77000
Carl,Manager,119000

In the code, you first open the file object f. Then you iterate over each row and each element in the row and write the element to the file—one by one. After each element, you place the comma to generate the CSV file format. After each row, you place the newline character '\n'.

Note: to get rid of the trailing comma, you can check if the element x is the last element in the row within the loop body and skip writing the comma if it is.

🌍 Finxter Recommended: Join the Finxter community and download your 8+ Python cheat sheets to refresh your code understanding.

Method 4: Converting to List of Lists First

A simple approach to convert a list of dicts to a CSV file is to first convert the list of dicts to a list of lists and then use the approaches discussed in the following article (code block given).

salary = [['Alice', 'Data Scientist', 122000], ['Bob', 'Engineer', 77000], ['Ann', 'Manager', 119000]] # Method 1
import csv
with open('file.csv', 'w', newline='') as f: writer = csv.writer(f) writer.writerows(salary) # Method 2
import pandas as pd
df = pd.DataFrame(salary)
df.to_csv('file2.csv', index=False, header=False) # Method 3
a = [[1, 2, 3], [4, 5, 6], [7, 8, 9]] import numpy as np
a = np.array(a)
np.savetxt('file3.csv', a, delimiter=',') # Method 4
with open('file4.csv','w') as f: for row in salary: for x in row: f.write(str(x) + ',') f.write('\n')

🌍 Related Tutorial: How to Convert a List to a CSV File in Python [5 Ways]

Where to Go From Here?

Enough theory. Let’s get some practice!

Coders get paid six figures and more because they can solve problems more effectively using machine intelligence and automation.

To become more successful in coding, solve more real problems for real people. That’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

You build high-value coding skills by working on practical coding projects!

Do you want to stop learning with toy projects and focus on practical code projects that earn you money and solve real problems for people?

🚀 If your answer is YES!, consider becoming a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

If you just want to learn about the freelancing opportunity, feel free to watch my free webinar “How to Build Your High-Income Skill Python” and learn how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

Posted on Leave a comment

3 Simple Steps to Convert calendar.ics to CSV/Excel in Python

Rate this post

Step 1: Install csv-ical Module with PIP

Run the following command in your command line or PowerShell (Windows) or shell or terminal (macOS, Linux, Ubuntu) to install the csv-ical library:

pip install csv-ical

In some instances, you need to modify this command a bit to make it work. If you need more assistance installing the library, check out my detailed guide.

🌍 Full Guide: How to install a library/module in Python?

Step 2: Prepare files

Create a new Python code file with the extension .py or a Jupyter Notebook with the file extension .ipynb. This creates a Python script or Jupyter Notebook that can run the code in Step 3 to conver the .ics.

Now, put the .ics file to be converted in the same folder as the newly-created Python script.

Use Jupyter Notebook to create a new .ipynb file

Step 3: Convert

This step consists of running the code doing these three things:

  • Create and initialize a Convert object
  • Read the .ics file
  • Create the CSV object and save it at the specified location

Here’s the full code:

from csv_ical import Convert # Create and initialize a Convert object
convert = Convert()
convert.CSV_FILE_LOCATION = 'my_file.csv'
convert.SAVE_LOCATION = 'my_file.ics' # Read the .ics file
convert.read_ical(convert.SAVE_LOCATION) # Create the CSV object and save it at the specified location
convert.make_csv()
convert.save_csv(convert.CSV_FILE_LOCATION)

Thanks for going through the whole tutorial! <3


Posted on Leave a comment

How to Sort Words Alphabetically in Python?

5/5 – (1 vote)

Problem Formulation and Solution Overview

In this article, you’ll learn how to sort words alphabetically in Python.

To make it more fun, we have the following running scenario:

Some English sayings, also known as Tongue-Twisters, are fun to try and say quickly. They are also used to improve non-English speaking individuals and small children’s fluency and pronunciation of the English language.

Imagine how challenging tongue twisters would be if sorted in alphabetical order!


💬 Question: How would we write code to sort a string in alphabetical order?

We can accomplish this task by one of the following options:


Method 1: Use split() and sort()

This method uses Python’s built-in string library to reference split() and sort() to display the words in ascending alphabetical order.

twister = 'how much wood would a woodchuck chuck if a woodchuck could chuck wood?'.lower().split()
twister.sort()
print(twister)

Above declares a tongue twister, converts the entire string to lowercase (lower()) and breaks it apart (split()) by default, on the space (' ') character. The results save to twister in a List format. If output to the terminal, the following displays.

['How', 'much', 'wood', 'would', 'a', 'woodchuck', 'chuck', 'if', 'a', 'woodchuck', 'could', 'chuck', 'wood?']

The following line sorts twister (sort()) in ascending alphabetical order. If output to the terminal, the following would display.

['a', 'a', 'chuck', 'chuck', 'could', 'how', 'if', 'much', 'wood', 'wood?', 'woodchuck', 'woodchuck', 'would']
YouTube Video

Method 2: Use split(), sorted() and join()

This method uses Python’s built-in string library to reference split() and sorted() to display the words in descending alphabetical order.

twister = 'I scream, you scream, we all scream for ice cream!'.lower()
twister = ' '.join(sorted(twister.split(), reverse=True)) print(twister)

Above declares a tongue twister, then converts the string to lowercase (lower()). The results save to twister in a List format. If output to the terminal, the following would display.

['i', 'scream,', 'you', 'scream,', 'we', 'all', 'scream', 'for', 'ice', 'cream!']

The following line breaks it apart (split()) by default, on the space (' ') character. Then, twister is sorted (sorted()) in descending alphabetical order (reverse=True).

The words are combined using the join() function, saved to twister and output to the terminal.

['you', 'we', 'scream,', 'scream,', 'scream', 'ice', 'i', 'for', 'cream!', 'all']
YouTube Video

Method 3: Use Bubble Sort Algorithm

This method uses the famous Bubble Sort Algorithm. This function accepts a List and loops through each element, comparing two (2) values, the current element value and the next element value. The greater element floats to the top, and the loop continues until the List is sorted.

twister = 'Which wristwatches are Swiss wristwatches?'.lower().split() def bubblesort(lst): for passesLeft in range(len(lst)-1, 0, -1): for i in range(passesLeft): if lst[i] > lst[i + 1]: lst[i], lst[i + 1] = lst[i + 1], lst[i] return ' '.join(lst) print(bubblesort(twister)) 

Above declares a tongue twister, converts the entire string to lowercase (lower()) and breaks it apart (split()) by default, on the space (' ') character. The results save to twister in a List format. If output to the terminal, the following displays.

['which', 'wristwatches', 'are', 'swiss', 'wristwatches?']

Next, the bubblesort() function is declared and accepts one (1) argument, an iterable List. An explanation of this code is outlined above.

However, we modified the code slightly to return a new string containing the sorted List values.

The bubblesort() function is then called and passed twister as an argument. The results are output to the terminal.

are swiss which wristwatches wristwatches?
YouTube Video

Method 4: Use sort_values()

This function imports the Pandas Library to reference the sort_values() function. This function sorts column(s) in a DataFrame.

To run this code error-free, install the required library. Click here for installation instructions.

To follow along, click here to download the finxters.csv file. Move this file to the current working directory.

import pandas as pd df = pd.read_csv('finxters.csv', skip_blank_lines=True, usecols=['FID', 'Username', 'Rank'])
rank_sort = df.sort_values(by=["Rank"], ascending=True)
print(rank_sort)

Above, imports the Pandas library.

Then, the finxter.csv file is read in, omitting blank lines, selecting the three (3) stated columns and saving to df.

Next, sort is applied to the Rank column, which contains the words pertaining to a user’s achievement level. The DataFrame (df) is sorted based on this column and the results save to rank_sort and output to the terminal.

Below a snippet of the results displays.

FID Username Rank
0 30022145 wildone92 Authority
45 3002481 Moon_Star2 Authority
9 30022450 Gar_man Authority
4 30022359 AliceM Authority
24 3002328 Wall_2021 Authority
49 3002573 jJonesing Authority
47 3002521 KerrStreet Autodidact
YouTube Video

Summary

These four (4) methods of sorting words alphabetically should give you enough information to select the best one for your coding requirements.

Good Luck & Happy Coding!


Programmer Humor – Blockchain

“Blockchains are like grappling hooks, in that it’s extremely cool when you encounter a problem for which they’re the right solution, but it happens way too rarely in real life.” source xkcd

Posted on Leave a comment

Best Solidity Linter

Rate this post

💡 A code linter is a static code analysis tool to find programming errors, bugs, style mistakes, and suspicious constructs.

The best Solidity Linter is Ethlint with a close second Solhint. Most other linters are not well qualified to compete with those early tools!

Solidity Linter #1 – Ethlint

Ethlint comes with the popular slogan “yet another Solidity linting tool”.

I think the name is not well chosen because, due the fact that Solidity is super young, there is not a swamp of linting tools available, yet.

You can install it using the following expression:

npm install -g solhint

Here’s how you’d run this:

solhint [options] <file> […other_files]

💡 Learn More: Ethlint Linting Tool

Solidity Linter #2 – Solhint

Solhint is a linter for Solidity that provides security and a style guide validations.

You can install the Linter using this command:

npm install -g ethlintsolium -V

After initial configuration, the execution is as simple as running this command in your shell:

> npm run solhint

💡 Learn More: Solhint Linting Tool

I would recommend more but I think those are the two best tools at this point.

If you want to learn Soldity, I’d applause you because this means you rely less on Linters (a goal worth pursuing)! 🙂

You can check out our in-depth tutorial here:

Learn Solidity Course

Solidity is the programming language of the future.

It gives you the rare and sought-after superpower to program against the “Internet Computer”, i.e., against decentralized Blockchains such as Ethereum, Binance Smart Chain, Ethereum Classic, Tron, and Avalanche – to mention just a few Blockchain infrastructures that support Solidity.

In particular, Solidity allows you to create smart contracts, i.e., pieces of code that automatically execute on specific conditions in a completely decentralized environment. For example, smart contracts empower you to create your own decentralized autonomous organizations (DAOs) that run on Blockchains without being subject to centralized control.

NFTs, DeFi, DAOs, and Blockchain-based games are all based on smart contracts.

This course is a simple, low-friction introduction to creating your first smart contract using the Remix IDE on the Ethereum testnet – without fluff, significant upfront costs to purchase ETH, or unnecessary complexity.

Posted on Leave a comment

How to Read and Convert a Binary File to CSV in Python?

5/5 – (1 vote)

To read a binary file, use the open('rb') function within a context manager (with keyword) and read its content into a string variable using f.readlines(). You can then convert the string to a CSV using various approaches such as the csv module.

Here’s an example to read the binary file 'my_file.man' into your Python script:

with open('my_file.man', 'rb') as f: content = f.readlines() print(content)

Per default, Python’s built-in open() function opens a text file. If you want to open a binary file, you need to add the 'b' character to the optional mode string argument.

  • To open a file for reading in binary format, use mode='rb'.
  • To open a file for writing in binary format, use mode='rb'.

Now that the content is in your Python script, you can convert it to a CSV using the various methods outlined in this article:

🌍 Learn More: Convert a String to CSV in Python

After you’ve converted the data to the comma-separated values (CSV) format demanded by your application, you can write the string to a file using either the print() function with file argument or the standard file.write() approach.

Posted on Leave a comment

How to Upload Files to Google Drive with API using PHP

by Vincy. Last modified on August 5th, 2022.

Uploading files to Google Drive programmatically can be done by the Google API. It uses OAuth to authenticate requests and authorize access.

This tutorial describes uploading files to Google Drive using PHP. It gives a simple PHP script to easily understand the Google API and upload files.

It also uses a database to save the uploaded file details with the Google Drive reference.

It handles errors that can occur for the following reasons during the upload process.

  1. When the file binary is empty on the PHP script.
  2. When the user fails to submit the form and proceeds to upload without form data.
  3. When the Google OAuth request is failed to get the access token.
  4. When the cURL request to the Google API is failed to return the status code 200.

On successful upload without any of the above uncertainties, this code shows the Google Drive link to see the uploaded file preview.

The below figure shows the file upload form with the success and failure responses.

php google drive upload

How to create API credentials to access Google Drive

Login to your Google account and go to the developer console. Then follow the below steps to create API credentials to access Google Drive to upload a file.

  1. Create a new project or select an existing project from the Google console header.
  2. Click the Library menu and enable Google Drive API. Use the filter to shortlist this API.
  3. Choose the OAuth consent screen menu to create the app. Fill up the following to register the app.
    • App name
    • support email
    • authorized domain
    • developer contact detail (email).
  4. Select Credentials->Create Credentials, then select OAuth client ID. Then, enter the following details.
    • Choose Application type as Web Application.
    • Add authorized JavaScript origin.
    • Add authorized redirect URI.

After completing these steps, the console will display the Google web client id and the secret key. These credentials are used for the authentication process to get access to Google Drive.

oauth credential

Example application files structure

Let us see the PHP example code created for this article to upload a file to Google Drive. The following figure shows the file structure of this example.

google drive file upload example

Application config file

This PHP file contains the constants used in this example. The API credentials and the endpoints are stored as PHP constants with this file.

The endpoint URI configured in this file is to hit the Google Drive API for the following purpose.

  • To set scope during OAuth redirect.
  • To get the access token after authentication with the API credentials GOOGLE_WEB_CLIENT_ID and GOOGLE_WEB_CLIENT_SECRET.
  • To upload file to Drive
  • To add metadata to the uploaded file

The AUTHORIZED_REDIRECT_URI is to set the callback. The API will call this URI with the access code to proceed with file upload after authentication.

lib/Config.php

<?php class Config
{ const GOOGLE_WEB_CLIENT_ID = 'add client id'; const GOOGLE_WEB_CLIENT_SECRET = 'add client secret'; const GOOGLE_ACCESS_SCOPE = 'https://www.googleapis.com/auth/drive'; const AUTHORIZED_REDIRECT_URI = 'https://domain-name/php-google-drive-upload/callback.php'; const GOOGLE_OAUTH2_TOKEN_URI = 'https://oauth2.googleapis.com/token'; const GOOGLE_DRIVE_FILE_UPLOAD_URI = 'https://www.googleapis.com/upload/drive/v3/files'; const GOOGLE_DRIVE_FILE_META_URI = 'https://www.googleapis.com/drive/v3/files/';
} ?>

Landing form with file upload option

This is a simple HTML form that calls the PHP endpoint upload.php on submitting. The file data is posted to this PHP file to upload to a local directory and to Google Drive.

I have just managed field validation by using HTML5 required attribute. You can also add exclusive JavaScript validation for this file upload form.

We have already seen code for doing server-side file validation in PHP.

index.php

<?php
session_start(); ?>
<html>
<head>
<title>How to upload file to Google drive</title>
<link rel="stylesheet" type="text/css" href="css/style.css" />
<link rel="stylesheet" type="text/css" href="css/form.css" />
<style>
input.btn-submit { background: #ffc72c url("google-drive-icon.png") no-repeat center left 45px; text-align: right; padding-right: 45px;
}
</style>
</head>
<body> <div class="phppot-container tile-container"> <form method="post" action="upload.php" class="form" enctype="multipart/form-data">
<?php if(!empty($_SESSION['responseMessage'])){ ?> <div id="phppot-message" class="<?php echo $_SESSION['responseMessage']['messageType']; ?>"> <?php echo $_SESSION['responseMessage']['message']; ?> </div>
<?php $_SESSION['responseMessage'] = "";
}
?>
<h2 class="text-center">Upload file to drive</h2> <div> <div class="row"> <label class="inline-block">Select file to upload</label> <input type="file" name="file" class="full-width" required> </div> <div class="row"> <input type="submit" name="submit" value="Upload to Drive" class="btn-submit full-width"> </div> </div> </form> </div>
</body>
</html>

PHP code upload file to a directory, save to database and redirect to Google

This HTML form action endpoint performs file upload to a directory. It saves the file path to the database and redirects to the Google OAuth URI.

This URI sets the scope, app client id and redirect path (callback.php) to get the access code from the Google Drive API endpoint.

In case of error occurrence, it calls application utils to acknowledge and guide users properly.

upload.php

<?php
session_start();
require_once __DIR__ . '/lib/Util.php';
$util = new Util(); if (! empty($_POST['submit'])) { require_once __DIR__ . '/lib/Config.php'; require_once __DIR__ . '/lib/FileModel.php'; $fileModel = new FileModel(); if (! empty($_FILES["file"]["name"])) { $fileName = basename($_FILES["file"]["name"]); $targetFilePath = "data/" . $fileName; if (move_uploaded_file($_FILES["file"]["tmp_name"], $targetFilePath)) { $fileInsertId = $fileModel->insertFile($fileName); if ($fileInsertId) { $_SESSION['fileInsertId'] = $fileInsertId; $googleOAuthURI = 'https://accounts.google.com/o/oauth2/auth?scope=' . urlencode(Config::GOOGLE_ACCESS_SCOPE) . '&redirect_uri=' . Config::AUTHORIZED_REDIRECT_URI . '&response_type=code&client_id=' . Config::GOOGLE_WEB_CLIENT_ID . '&access_type=online'; header("Location: $googleOAuthURI"); exit(); } else { $util->redirect("error", 'Failed to insert into the database.'); } } else { $util->redirect("error", 'Failed to upload file.'); } } else { $util->redirect("error", 'Choose file to upload.'); }
} else { $util->redirect("error", 'Failed to find the form data.');
}
?>

Callback action to get access token and proceed file upload to Google Drive

This page is called by Google API after performing the OAuth request. The API sends a code parameter while calling this redirect URL.

It calls the getAccessToken() a function defined in the service class. It passes API credentials to get the access token.

When the token is received, this file builds the file content and file meta to be uploaded to Google Drive via cURL request.

The uploadFileToGoogleDrive() accepts access token and the file information to set the cURL options. It returns the file id of the uploaded file to Google Drive.

Then, the addFileMeta() PHP function accepts the array of file metadata. It returns the Google Drive file meta data received as a cURL response.

This metadata id is used in the success response to allow users to view the uploaded file in Google Drive.

callback.php

<?php
session_start();
require_once __DIR__ . '/lib/Util.php';
$util = new Util();
if (isset($_GET['code'])) { require_once __DIR__ . '/lib/Config.php'; require_once __DIR__ . '/lib/GoogleDriveUploadService.php'; $googleDriveUploadService = new GoogleDriveUploadService(); $googleResponse = $googleDriveUploadService->getAccessToken(Config::GOOGLE_WEB_CLIENT_ID, Config::AUTHORIZED_REDIRECT_URI, Config::GOOGLE_WEB_CLIENT_SECRET, $_GET['code']); $accessToken = $googleResponse['access_token']; if (! empty($accessToken)) { require_once __DIR__ . '/lib/FileModel.php'; $fileModel = new FileModel(); $fileId = $_SESSION['fileInsertId']; if (! empty($fileId)) { $fileResult = $fileModel->getFileRecordById($fileId); if (! empty($fileResult)) { $fileName = $fileResult[0]['file_base_name']; $filePath = 'data/' . $fileName; $fileContent = file_get_contents($filePath); $fileSize = filesize($filePath); $filetype = mime_content_type($filePath); try { // Move file to Google Drive via cURL $googleDriveFileId = $googleDriveUploadService->uploadFileToGoogleDrive($accessToken, $fileContent, $filetype, $fileSize); if ($googleDriveFileId) { $fileMeta = array( 'name' => basename($fileName) ); // Add file metadata via Google Drive API $googleDriveMeta = $googleDriveUploadService->addFileMeta($accessToken, $googleDriveFileId, $fileMeta); if ($googleDriveMeta) { $fileModel->updateFile($googleDriveFileId, $fileId); $_SESSION['fileInsertId'] = ''; $driveLink = '<a href="https://drive.google.com/open?id=' . $googleDriveMeta['id'] . '" target="_blank"><b>Open in Google Drive</b></a>.'; $util->redirect("success", 'File uploaded. ' . $driveLink); } } } catch (Exception $e) { $util->redirect("error", $e->getMessage()); } } else { $util->redirect("error", 'Failed to get the file content.'); } } else { $util->redirect("error", 'File id not found.'); } } else { $util->redirect("error", 'Something went wrong. Access forbidden.'); }
}
?>

PHP service class to prepare requests and hit Google Drive API via cURL

The service class contains functions that build the PHP cURL request to hit the Google Drive API.

All the cURL requests use POST methods to submit parameters to the API endpoints.

It gets the response code and the data in the specified format. In case of a cURL error or getting a response code other than 200, it throws exceptions.

On getting the status code 200, it receives the Google Drive file reference and metadata JSON response appropriately.

lib/GoogleDriveUploadService.php

<?php
require_once __DIR__ . '/Config.php'; class GoogleDriveUploadService
{ public function getAccessToken($clientId, $authorizedRedirectURI, $clientSecret, $code) { $curlPost = 'client_id=' . $clientId . '&redirect_uri=' . $authorizedRedirectURI . '&client_secret=' . $clientSecret . '&code=' . $code . '&grant_type=authorization_code'; $curl = curl_init(); curl_setopt($curl, CURLOPT_URL, Config::GOOGLE_OAUTH2_TOKEN_URI); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); curl_setopt($curl, CURLOPT_POST, 1); curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE); curl_setopt($curl, CURLOPT_POSTFIELDS, $curlPost); $curlResponse = json_decode(curl_exec($curl), true); $responseCode = curl_getinfo($curl, CURLINFO_HTTP_CODE); if ($responseCode != 200) { $errorMessage = 'Problem in getting access token'; if (curl_errno($curl)) { $errorMessage = curl_error($curl); } throw new Exception('Error: ' . $responseCode . ': ' . $errorMessage); } return $curlResponse; } public function uploadFileToGoogleDrive($accessToken, $fileContent, $filetype, $fileSize) { $curl = curl_init(); curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false); curl_setopt($curl, CURLOPT_URL, Config::GOOGLE_DRIVE_FILE_UPLOAD_URI . '?uploadType=media'); curl_setopt($curl, CURLOPT_BINARYTRANSFER, 1); curl_setopt($curl, CURLOPT_POST, 1); curl_setopt($curl, CURLOPT_POSTFIELDS, $fileContent); curl_setopt($curl, CURLOPT_RETURNTRANSFER, true); curl_setopt($curl, CURLOPT_HTTPHEADER, array( 'Content-Type: ' . $filetype, 'Content-Length: ' . $fileSize, 'Authorization: Bearer ' . $accessToken )); $curlResponse = json_decode(curl_exec($curl), true); $responseCode = curl_getinfo($curl, CURLINFO_HTTP_CODE); if ($responseCode != 200) { $errorMessage = 'Failed to upload file to drive'; if (curl_errno($curl)) { $errorMessage = curl_error($curl); } throw new Exception('Error ' . $responseCode . ': ' . $errorMessage); } curl_close($curl); return $curlResponse['id']; } public function addFileMeta($accessToken, $googleDriveFileId, $fileMeta) { $curl = curl_init(); curl_setopt($curl, CURLOPT_URL, Config::GOOGLE_DRIVE_FILE_META_URI . $googleDriveFileId); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); curl_setopt($curl, CURLOPT_POST, 1); curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false); curl_setopt($curl, CURLOPT_HTTPHEADER, array( 'Content-Type: application/json', 'Authorization: Bearer ' . $accessToken )); curl_setopt($curl, CURLOPT_CUSTOMREQUEST, 'PATCH'); curl_setopt($curl, CURLOPT_POSTFIELDS, json_encode($fileMeta)); $curlResponse = json_decode(curl_exec($curl), true); $responseCode = curl_getinfo($curl, CURLINFO_HTTP_CODE); if ($responseCode != 200) { $errorMessage = 'Failed to add file metadata'; if (curl_errno($curl)) { $errorMessage = curl_error($curl); } throw new Exception('Error ' . $responseCode . ': ' . $errorMessage); } curl_close($curl); return $curlResponse; }
}
?>

PHP model class to build queries and parameters to insert, read and update file data log

This PHP model class defines functions to keep track of the database log for the uploaded file.

In the callback, it writes the Google Drive file id with the reference of the last inserted id in the session.

lib/FileModel.php

<?php
require_once __DIR__ . '/DataSource.php'; class FileModel extends DataSource
{ function insertFile($fileBaseName) { $query = "INSERT INTO google_drive_upload_response_log (file_base_name, create_at) VALUES (?, NOW())"; $paramType = 's'; $paramValue = array( $fileBaseName ); $insertId = $this->insert($query, $paramType, $paramValue); return $insertId; } function getFileRecordById($fileId) { $query = "SELECT * FROM google_drive_upload_response_log WHERE id = ?"; $paramType = 'i'; $paramValue = array( $fileId ); $result = $this->select($query, $paramType, $paramValue); return $result; } function updateFile($googleFileId, $fileId) { $query = "UPDATE google_drive_upload_response_log SET google_file_id=? WHERE id=?"; $paramType = 'si'; $paramValue = array( $googleFileId, $fileId ); $this->update($query, $paramType, $paramValue); }
}
?>

This file is a simple PHP Util class having only a redirect function as of now.

We can enhance this function by adding more utils. For example, it can have JSON encode decode to convert the cURL response into an array.

lib/Util.php

<?php class Util
{ function redirect($type, $message) { $_SESSION['responseMessage'] = array( 'messageType' => $type, 'message' => $message ); header("Location: index.php"); exit(); }
}
?>

Installation steps

Before running this example to upload a file to Google Drive, do the following steps. It will let the development environment be ready with the required configurations and resources.

  1. Configure database details with lib/DataSource.php. The source code includes this file.
  2. Configure Google API keys with lib/Config.php. Also, provide the domain and subfolder for setting the callback with AUTHORIZED_REDIRECT_URI.
  3. Import the below SQL script into your target database.

sql/structure.sql

--
-- Table structure for table `google_drive_upload_response_log`
-- CREATE TABLE `google_drive_upload_response_log` ( `id` int NOT NULL, `google_file_id` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL, `file_base_name` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL, `create_at` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci; --
-- Indexes for dumped tables
-- --
-- Indexes for table `google_drive_upload_response_log`
--
ALTER TABLE `google_drive_upload_response_log` ADD PRIMARY KEY (`id`); --
-- AUTO_INCREMENT for dumped tables
-- --
-- AUTO_INCREMENT for table `google_drive_upload_response_log`
--
ALTER TABLE `google_drive_upload_response_log` MODIFY `id` int NOT NULL AUTO_INCREMENT;

Download

↑ Back to Top