Posted on Leave a comment

How to Sort Words Alphabetically in Python?

5/5 – (1 vote)

Problem Formulation and Solution Overview

In this article, you’ll learn how to sort words alphabetically in Python.

To make it more fun, we have the following running scenario:

Some English sayings, also known as Tongue-Twisters, are fun to try and say quickly. They are also used to improve non-English speaking individuals and small children’s fluency and pronunciation of the English language.

Imagine how challenging tongue twisters would be if sorted in alphabetical order!


đź’¬ Question: How would we write code to sort a string in alphabetical order?

We can accomplish this task by one of the following options:


Method 1: Use split() and sort()

This method uses Python’s built-in string library to reference split() and sort() to display the words in ascending alphabetical order.

twister = 'how much wood would a woodchuck chuck if a woodchuck could chuck wood?'.lower().split()
twister.sort()
print(twister)

Above declares a tongue twister, converts the entire string to lowercase (lower()) and breaks it apart (split()) by default, on the space (' ') character. The results save to twister in a List format. If output to the terminal, the following displays.

['How', 'much', 'wood', 'would', 'a', 'woodchuck', 'chuck', 'if', 'a', 'woodchuck', 'could', 'chuck', 'wood?']

The following line sorts twister (sort()) in ascending alphabetical order. If output to the terminal, the following would display.

['a', 'a', 'chuck', 'chuck', 'could', 'how', 'if', 'much', 'wood', 'wood?', 'woodchuck', 'woodchuck', 'would']
YouTube Video

Method 2: Use split(), sorted() and join()

This method uses Python’s built-in string library to reference split() and sorted() to display the words in descending alphabetical order.

twister = 'I scream, you scream, we all scream for ice cream!'.lower()
twister = ' '.join(sorted(twister.split(), reverse=True)) print(twister)

Above declares a tongue twister, then converts the string to lowercase (lower()). The results save to twister in a List format. If output to the terminal, the following would display.

['i', 'scream,', 'you', 'scream,', 'we', 'all', 'scream', 'for', 'ice', 'cream!']

The following line breaks it apart (split()) by default, on the space (' ') character. Then, twister is sorted (sorted()) in descending alphabetical order (reverse=True).

The words are combined using the join() function, saved to twister and output to the terminal.

['you', 'we', 'scream,', 'scream,', 'scream', 'ice', 'i', 'for', 'cream!', 'all']
YouTube Video

Method 3: Use Bubble Sort Algorithm

This method uses the famous Bubble Sort Algorithm. This function accepts a List and loops through each element, comparing two (2) values, the current element value and the next element value. The greater element floats to the top, and the loop continues until the List is sorted.

twister = 'Which wristwatches are Swiss wristwatches?'.lower().split() def bubblesort(lst): for passesLeft in range(len(lst)-1, 0, -1): for i in range(passesLeft): if lst[i] > lst[i + 1]: lst[i], lst[i + 1] = lst[i + 1], lst[i] return ' '.join(lst) print(bubblesort(twister)) 

Above declares a tongue twister, converts the entire string to lowercase (lower()) and breaks it apart (split()) by default, on the space (' ') character. The results save to twister in a List format. If output to the terminal, the following displays.

['which', 'wristwatches', 'are', 'swiss', 'wristwatches?']

Next, the bubblesort() function is declared and accepts one (1) argument, an iterable List. An explanation of this code is outlined above.

However, we modified the code slightly to return a new string containing the sorted List values.

The bubblesort() function is then called and passed twister as an argument. The results are output to the terminal.

are swiss which wristwatches wristwatches?
YouTube Video

Method 4: Use sort_values()

This function imports the Pandas Library to reference the sort_values() function. This function sorts column(s) in a DataFrame.

To run this code error-free, install the required library. Click here for installation instructions.

To follow along, click here to download the finxters.csv file. Move this file to the current working directory.

import pandas as pd df = pd.read_csv('finxters.csv', skip_blank_lines=True, usecols=['FID', 'Username', 'Rank'])
rank_sort = df.sort_values(by=["Rank"], ascending=True)
print(rank_sort)

Above, imports the Pandas library.

Then, the finxter.csv file is read in, omitting blank lines, selecting the three (3) stated columns and saving to df.

Next, sort is applied to the Rank column, which contains the words pertaining to a user’s achievement level. The DataFrame (df) is sorted based on this column and the results save to rank_sort and output to the terminal.

Below a snippet of the results displays.

FID Username Rank
0 30022145 wildone92 Authority
45 3002481 Moon_Star2 Authority
9 30022450 Gar_man Authority
4 30022359 AliceM Authority
24 3002328 Wall_2021 Authority
49 3002573 jJonesing Authority
47 3002521 KerrStreet Autodidact
YouTube Video

Summary

These four (4) methods of sorting words alphabetically should give you enough information to select the best one for your coding requirements.

Good Luck & Happy Coding!


Programmer Humor – Blockchain

“Blockchains are like grappling hooks, in that it’s extremely cool when you encounter a problem for which they’re the right solution, but it happens way too rarely in real life.” source xkcd

Posted on Leave a comment

Best Solidity Linter

Rate this post

đź’ˇ A code linter is a static code analysis tool to find programming errors, bugs, style mistakes, and suspicious constructs.

The best Solidity Linter is Ethlint with a close second Solhint. Most other linters are not well qualified to compete with those early tools!

Solidity Linter #1 – Ethlint

Ethlint comes with the popular slogan “yet another Solidity linting tool”.

I think the name is not well chosen because, due the fact that Solidity is super young, there is not a swamp of linting tools available, yet.

You can install it using the following expression:

npm install -g solhint

Here’s how you’d run this:

solhint [options] <file> […other_files]

đź’ˇ Learn More: Ethlint Linting Tool

Solidity Linter #2 – Solhint

Solhint is a linter for Solidity that provides security and a style guide validations.

You can install the Linter using this command:

npm install -g ethlintsolium -V

After initial configuration, the execution is as simple as running this command in your shell:

> npm run solhint

đź’ˇ Learn More: Solhint Linting Tool

I would recommend more but I think those are the two best tools at this point.

If you want to learn Soldity, I’d applause you because this means you rely less on Linters (a goal worth pursuing)! 🙂

You can check out our in-depth tutorial here:

Learn Solidity Course

Solidity is the programming language of the future.

It gives you the rare and sought-after superpower to program against the “Internet Computer”, i.e., against decentralized Blockchains such as Ethereum, Binance Smart Chain, Ethereum Classic, Tron, and Avalanche – to mention just a few Blockchain infrastructures that support Solidity.

In particular, Solidity allows you to create smart contracts, i.e., pieces of code that automatically execute on specific conditions in a completely decentralized environment. For example, smart contracts empower you to create your own decentralized autonomous organizations (DAOs) that run on Blockchains without being subject to centralized control.

NFTs, DeFi, DAOs, and Blockchain-based games are all based on smart contracts.

This course is a simple, low-friction introduction to creating your first smart contract using the Remix IDE on the Ethereum testnet – without fluff, significant upfront costs to purchase ETH, or unnecessary complexity.

Posted on Leave a comment

How to Read and Convert a Binary File to CSV in Python?

5/5 – (1 vote)

To read a binary file, use the open('rb') function within a context manager (with keyword) and read its content into a string variable using f.readlines(). You can then convert the string to a CSV using various approaches such as the csv module.

Here’s an example to read the binary file 'my_file.man' into your Python script:

with open('my_file.man', 'rb') as f: content = f.readlines() print(content)

Per default, Python’s built-in open() function opens a text file. If you want to open a binary file, you need to add the 'b' character to the optional mode string argument.

  • To open a file for reading in binary format, use mode='rb'.
  • To open a file for writing in binary format, use mode='rb'.

Now that the content is in your Python script, you can convert it to a CSV using the various methods outlined in this article:

🌍 Learn More: Convert a String to CSV in Python

After you’ve converted the data to the comma-separated values (CSV) format demanded by your application, you can write the string to a file using either the print() function with file argument or the standard file.write() approach.

Posted on Leave a comment

How to Upload Files to Google Drive with API using PHP

by Vincy. Last modified on August 5th, 2022.

Uploading files to Google Drive programmatically can be done by the Google API. It uses OAuth to authenticate requests and authorize access.

This tutorial describes uploading files to Google Drive using PHP. It gives a simple PHP script to easily understand the Google API and upload files.

It also uses a database to save the uploaded file details with the Google Drive reference.

It handles errors that can occur for the following reasons during the upload process.

  1. When the file binary is empty on the PHP script.
  2. When the user fails to submit the form and proceeds to upload without form data.
  3. When the Google OAuth request is failed to get the access token.
  4. When the cURL request to the Google API is failed to return the status code 200.

On successful upload without any of the above uncertainties, this code shows the Google Drive link to see the uploaded file preview.

The below figure shows the file upload form with the success and failure responses.

php google drive upload

How to create API credentials to access Google Drive

Login to your Google account and go to the developer console. Then follow the below steps to create API credentials to access Google Drive to upload a file.

  1. Create a new project or select an existing project from the Google console header.
  2. Click the Library menu and enable Google Drive API. Use the filter to shortlist this API.
  3. Choose the OAuth consent screen menu to create the app. Fill up the following to register the app.
    • App name
    • support email
    • authorized domain
    • developer contact detail (email).
  4. Select Credentials->Create Credentials, then select OAuth client ID. Then, enter the following details.
    • Choose Application type as Web Application.
    • Add authorized JavaScript origin.
    • Add authorized redirect URI.

After completing these steps, the console will display the Google web client id and the secret key. These credentials are used for the authentication process to get access to Google Drive.

oauth credential

Example application files structure

Let us see the PHP example code created for this article to upload a file to Google Drive. The following figure shows the file structure of this example.

google drive file upload example

Application config file

This PHP file contains the constants used in this example. The API credentials and the endpoints are stored as PHP constants with this file.

The endpoint URI configured in this file is to hit the Google Drive API for the following purpose.

  • To set scope during OAuth redirect.
  • To get the access token after authentication with the API credentials GOOGLE_WEB_CLIENT_ID and GOOGLE_WEB_CLIENT_SECRET.
  • To upload file to Drive
  • To add metadata to the uploaded file

The AUTHORIZED_REDIRECT_URI is to set the callback. The API will call this URI with the access code to proceed with file upload after authentication.

lib/Config.php

<?php class Config
{ const GOOGLE_WEB_CLIENT_ID = 'add client id'; const GOOGLE_WEB_CLIENT_SECRET = 'add client secret'; const GOOGLE_ACCESS_SCOPE = 'https://www.googleapis.com/auth/drive'; const AUTHORIZED_REDIRECT_URI = 'https://domain-name/php-google-drive-upload/callback.php'; const GOOGLE_OAUTH2_TOKEN_URI = 'https://oauth2.googleapis.com/token'; const GOOGLE_DRIVE_FILE_UPLOAD_URI = 'https://www.googleapis.com/upload/drive/v3/files'; const GOOGLE_DRIVE_FILE_META_URI = 'https://www.googleapis.com/drive/v3/files/';
} ?>

Landing form with file upload option

This is a simple HTML form that calls the PHP endpoint upload.php on submitting. The file data is posted to this PHP file to upload to a local directory and to Google Drive.

I have just managed field validation by using HTML5 required attribute. You can also add exclusive JavaScript validation for this file upload form.

We have already seen code for doing server-side file validation in PHP.

index.php

<?php
session_start(); ?>
<html>
<head>
<title>How to upload file to Google drive</title>
<link rel="stylesheet" type="text/css" href="css/style.css" />
<link rel="stylesheet" type="text/css" href="css/form.css" />
<style>
input.btn-submit { background: #ffc72c url("google-drive-icon.png") no-repeat center left 45px; text-align: right; padding-right: 45px;
}
</style>
</head>
<body> <div class="phppot-container tile-container"> <form method="post" action="upload.php" class="form" enctype="multipart/form-data">
<?php if(!empty($_SESSION['responseMessage'])){ ?> <div id="phppot-message" class="<?php echo $_SESSION['responseMessage']['messageType']; ?>"> <?php echo $_SESSION['responseMessage']['message']; ?> </div>
<?php $_SESSION['responseMessage'] = "";
}
?>
<h2 class="text-center">Upload file to drive</h2> <div> <div class="row"> <label class="inline-block">Select file to upload</label> <input type="file" name="file" class="full-width" required> </div> <div class="row"> <input type="submit" name="submit" value="Upload to Drive" class="btn-submit full-width"> </div> </div> </form> </div>
</body>
</html>

PHP code upload file to a directory, save to database and redirect to Google

This HTML form action endpoint performs file upload to a directory. It saves the file path to the database and redirects to the Google OAuth URI.

This URI sets the scope, app client id and redirect path (callback.php) to get the access code from the Google Drive API endpoint.

In case of error occurrence, it calls application utils to acknowledge and guide users properly.

upload.php

<?php
session_start();
require_once __DIR__ . '/lib/Util.php';
$util = new Util(); if (! empty($_POST['submit'])) { require_once __DIR__ . '/lib/Config.php'; require_once __DIR__ . '/lib/FileModel.php'; $fileModel = new FileModel(); if (! empty($_FILES["file"]["name"])) { $fileName = basename($_FILES["file"]["name"]); $targetFilePath = "data/" . $fileName; if (move_uploaded_file($_FILES["file"]["tmp_name"], $targetFilePath)) { $fileInsertId = $fileModel->insertFile($fileName); if ($fileInsertId) { $_SESSION['fileInsertId'] = $fileInsertId; $googleOAuthURI = 'https://accounts.google.com/o/oauth2/auth?scope=' . urlencode(Config::GOOGLE_ACCESS_SCOPE) . '&redirect_uri=' . Config::AUTHORIZED_REDIRECT_URI . '&response_type=code&client_id=' . Config::GOOGLE_WEB_CLIENT_ID . '&access_type=online'; header("Location: $googleOAuthURI"); exit(); } else { $util->redirect("error", 'Failed to insert into the database.'); } } else { $util->redirect("error", 'Failed to upload file.'); } } else { $util->redirect("error", 'Choose file to upload.'); }
} else { $util->redirect("error", 'Failed to find the form data.');
}
?>

Callback action to get access token and proceed file upload to Google Drive

This page is called by Google API after performing the OAuth request. The API sends a code parameter while calling this redirect URL.

It calls the getAccessToken() a function defined in the service class. It passes API credentials to get the access token.

When the token is received, this file builds the file content and file meta to be uploaded to Google Drive via cURL request.

The uploadFileToGoogleDrive() accepts access token and the file information to set the cURL options. It returns the file id of the uploaded file to Google Drive.

Then, the addFileMeta() PHP function accepts the array of file metadata. It returns the Google Drive file meta data received as a cURL response.

This metadata id is used in the success response to allow users to view the uploaded file in Google Drive.

callback.php

<?php
session_start();
require_once __DIR__ . '/lib/Util.php';
$util = new Util();
if (isset($_GET['code'])) { require_once __DIR__ . '/lib/Config.php'; require_once __DIR__ . '/lib/GoogleDriveUploadService.php'; $googleDriveUploadService = new GoogleDriveUploadService(); $googleResponse = $googleDriveUploadService->getAccessToken(Config::GOOGLE_WEB_CLIENT_ID, Config::AUTHORIZED_REDIRECT_URI, Config::GOOGLE_WEB_CLIENT_SECRET, $_GET['code']); $accessToken = $googleResponse['access_token']; if (! empty($accessToken)) { require_once __DIR__ . '/lib/FileModel.php'; $fileModel = new FileModel(); $fileId = $_SESSION['fileInsertId']; if (! empty($fileId)) { $fileResult = $fileModel->getFileRecordById($fileId); if (! empty($fileResult)) { $fileName = $fileResult[0]['file_base_name']; $filePath = 'data/' . $fileName; $fileContent = file_get_contents($filePath); $fileSize = filesize($filePath); $filetype = mime_content_type($filePath); try { // Move file to Google Drive via cURL $googleDriveFileId = $googleDriveUploadService->uploadFileToGoogleDrive($accessToken, $fileContent, $filetype, $fileSize); if ($googleDriveFileId) { $fileMeta = array( 'name' => basename($fileName) ); // Add file metadata via Google Drive API $googleDriveMeta = $googleDriveUploadService->addFileMeta($accessToken, $googleDriveFileId, $fileMeta); if ($googleDriveMeta) { $fileModel->updateFile($googleDriveFileId, $fileId); $_SESSION['fileInsertId'] = ''; $driveLink = '<a href="https://drive.google.com/open?id=' . $googleDriveMeta['id'] . '" target="_blank"><b>Open in Google Drive</b></a>.'; $util->redirect("success", 'File uploaded. ' . $driveLink); } } } catch (Exception $e) { $util->redirect("error", $e->getMessage()); } } else { $util->redirect("error", 'Failed to get the file content.'); } } else { $util->redirect("error", 'File id not found.'); } } else { $util->redirect("error", 'Something went wrong. Access forbidden.'); }
}
?>

PHP service class to prepare requests and hit Google Drive API via cURL

The service class contains functions that build the PHP cURL request to hit the Google Drive API.

All the cURL requests use POST methods to submit parameters to the API endpoints.

It gets the response code and the data in the specified format. In case of a cURL error or getting a response code other than 200, it throws exceptions.

On getting the status code 200, it receives the Google Drive file reference and metadata JSON response appropriately.

lib/GoogleDriveUploadService.php

<?php
require_once __DIR__ . '/Config.php'; class GoogleDriveUploadService
{ public function getAccessToken($clientId, $authorizedRedirectURI, $clientSecret, $code) { $curlPost = 'client_id=' . $clientId . '&redirect_uri=' . $authorizedRedirectURI . '&client_secret=' . $clientSecret . '&code=' . $code . '&grant_type=authorization_code'; $curl = curl_init(); curl_setopt($curl, CURLOPT_URL, Config::GOOGLE_OAUTH2_TOKEN_URI); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); curl_setopt($curl, CURLOPT_POST, 1); curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE); curl_setopt($curl, CURLOPT_POSTFIELDS, $curlPost); $curlResponse = json_decode(curl_exec($curl), true); $responseCode = curl_getinfo($curl, CURLINFO_HTTP_CODE); if ($responseCode != 200) { $errorMessage = 'Problem in getting access token'; if (curl_errno($curl)) { $errorMessage = curl_error($curl); } throw new Exception('Error: ' . $responseCode . ': ' . $errorMessage); } return $curlResponse; } public function uploadFileToGoogleDrive($accessToken, $fileContent, $filetype, $fileSize) { $curl = curl_init(); curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false); curl_setopt($curl, CURLOPT_URL, Config::GOOGLE_DRIVE_FILE_UPLOAD_URI . '?uploadType=media'); curl_setopt($curl, CURLOPT_BINARYTRANSFER, 1); curl_setopt($curl, CURLOPT_POST, 1); curl_setopt($curl, CURLOPT_POSTFIELDS, $fileContent); curl_setopt($curl, CURLOPT_RETURNTRANSFER, true); curl_setopt($curl, CURLOPT_HTTPHEADER, array( 'Content-Type: ' . $filetype, 'Content-Length: ' . $fileSize, 'Authorization: Bearer ' . $accessToken )); $curlResponse = json_decode(curl_exec($curl), true); $responseCode = curl_getinfo($curl, CURLINFO_HTTP_CODE); if ($responseCode != 200) { $errorMessage = 'Failed to upload file to drive'; if (curl_errno($curl)) { $errorMessage = curl_error($curl); } throw new Exception('Error ' . $responseCode . ': ' . $errorMessage); } curl_close($curl); return $curlResponse['id']; } public function addFileMeta($accessToken, $googleDriveFileId, $fileMeta) { $curl = curl_init(); curl_setopt($curl, CURLOPT_URL, Config::GOOGLE_DRIVE_FILE_META_URI . $googleDriveFileId); curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); curl_setopt($curl, CURLOPT_POST, 1); curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false); curl_setopt($curl, CURLOPT_HTTPHEADER, array( 'Content-Type: application/json', 'Authorization: Bearer ' . $accessToken )); curl_setopt($curl, CURLOPT_CUSTOMREQUEST, 'PATCH'); curl_setopt($curl, CURLOPT_POSTFIELDS, json_encode($fileMeta)); $curlResponse = json_decode(curl_exec($curl), true); $responseCode = curl_getinfo($curl, CURLINFO_HTTP_CODE); if ($responseCode != 200) { $errorMessage = 'Failed to add file metadata'; if (curl_errno($curl)) { $errorMessage = curl_error($curl); } throw new Exception('Error ' . $responseCode . ': ' . $errorMessage); } curl_close($curl); return $curlResponse; }
}
?>

PHP model class to build queries and parameters to insert, read and update file data log

This PHP model class defines functions to keep track of the database log for the uploaded file.

In the callback, it writes the Google Drive file id with the reference of the last inserted id in the session.

lib/FileModel.php

<?php
require_once __DIR__ . '/DataSource.php'; class FileModel extends DataSource
{ function insertFile($fileBaseName) { $query = "INSERT INTO google_drive_upload_response_log (file_base_name, create_at) VALUES (?, NOW())"; $paramType = 's'; $paramValue = array( $fileBaseName ); $insertId = $this->insert($query, $paramType, $paramValue); return $insertId; } function getFileRecordById($fileId) { $query = "SELECT * FROM google_drive_upload_response_log WHERE id = ?"; $paramType = 'i'; $paramValue = array( $fileId ); $result = $this->select($query, $paramType, $paramValue); return $result; } function updateFile($googleFileId, $fileId) { $query = "UPDATE google_drive_upload_response_log SET google_file_id=? WHERE id=?"; $paramType = 'si'; $paramValue = array( $googleFileId, $fileId ); $this->update($query, $paramType, $paramValue); }
}
?>

This file is a simple PHP Util class having only a redirect function as of now.

We can enhance this function by adding more utils. For example, it can have JSON encode decode to convert the cURL response into an array.

lib/Util.php

<?php class Util
{ function redirect($type, $message) { $_SESSION['responseMessage'] = array( 'messageType' => $type, 'message' => $message ); header("Location: index.php"); exit(); }
}
?>

Installation steps

Before running this example to upload a file to Google Drive, do the following steps. It will let the development environment be ready with the required configurations and resources.

  1. Configure database details with lib/DataSource.php. The source code includes this file.
  2. Configure Google API keys with lib/Config.php. Also, provide the domain and subfolder for setting the callback with AUTHORIZED_REDIRECT_URI.
  3. Import the below SQL script into your target database.

sql/structure.sql

--
-- Table structure for table `google_drive_upload_response_log`
-- CREATE TABLE `google_drive_upload_response_log` ( `id` int NOT NULL, `google_file_id` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL, `file_base_name` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL, `create_at` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci; --
-- Indexes for dumped tables
-- --
-- Indexes for table `google_drive_upload_response_log`
--
ALTER TABLE `google_drive_upload_response_log` ADD PRIMARY KEY (`id`); --
-- AUTO_INCREMENT for dumped tables
-- --
-- AUTO_INCREMENT for table `google_drive_upload_response_log`
--
ALTER TABLE `google_drive_upload_response_log` MODIFY `id` int NOT NULL AUTO_INCREMENT;

Download

↑ Back to Top

Posted on Leave a comment

How to Convert Epoch Time to Date Time in Python

5/5 – (1 vote)

Problem Formulation and Solution Overview

In this article, you’ll learn how to convert Epoch Time to a Date Time representation using Python.

On January 1st, 1970, Epoch Time, aka Time 0 for UNIX systems, started as a date in history to remember. This date is relevant, not only due to this event but because it redefined how dates are calculated!

To make it more fun, we will calculate the time elapsed in Epoch Time from its inception on January 1, 1970, to January 1, 1985, when the first mobile phone call was made in Britain by Ernie Wise to Vodafone. This will then be converted to a Date Time representation.


đź’¬ Question: How would we write code to convert an Epoch Date to a Date Time representation?

We can accomplish this task by one of the following options:


Method 1: Use fromtimestamp()

This method imports the datetime library and calls the associated datetime.fromtimestamp() function to convert Epoch Time into a Local Date Time representation.

To run this code error-free, install the required library. Click here for installation instructions.

import datetime epoch_time = 473398200
date_conv = datetime.datetime.fromtimestamp(epoch_time)
print(date_conv.strftime('%d-%m-%Y'))

Above, imports the datetime library. This allows the conversion of an Epoch Time integer to a readable Local Date Time format.

The following line declares an Epoch Time integer and saves it to epoch_time.

Next, the highlighted line converts the Epoch Time into a Local Date Time representation and saves it to date_conv. If output to the terminal at this point, it would display as follows:

1985-01-01 00:00:00

Finally, date_conv converts into a string using strftime() and outputs the formatted date to the terminal.

01-01-1985

Method 2: Use time.localtime()

This method imports the time library and calls the associated time.localtime() function to convert Epoch Time into a Local Date Time representation.

import time epoch_time = 473398200
date_conv = time.localtime(epoch_time)
print(date_conv)

Above, imports the time library. This allows the conversion of an Epoch Time to a readable Local Date Time format.

The following line declares an Epoch Time integer and saves it to epoch_time.

Next, the highlighted line converts the Epoch Time into a Local Date Time representation and saves it to date_conv as a Tuple as shown below:

time.struct_time(tm_year=1985, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=1, tm_yday=1, tm_isdst=0)

The appropriate elements will need to be accessed to format a date or time. For this example, we will construct the date.

print(f'0{date_conv[1]}-0{date_conv[2]}-{date_conv[0]}')

The output is as follows:

01-01-1985
YouTube Video

Method 3: Use datetime.utcfromtimestamp

This method imports the datetime library and calls the associated datetime.utcfromtimestamp() function to convert an Epoch Time into a UTC Date Time representation.

To run this code error-free, install the required library. Click here for installation instructions.

import datetime epoch_time = 473398200
date_conv = datetime.datetime.utcfromtimestamp(epoch_time).strftime('%Y-%m-%d %H:%M:%S')
print(date_conv)

Above, imports the datetime library. This allows the conversion of an Epoch Time integer to a readable UTC Date Time format.

The following line declares an Epoch Time integer and saves it to epoch_time.

Next, the highlighted line accomplishes the following:

  • Converts an Epoch Time to a UTC Date Format.
  • Converts to a Date string (strftime()) into the stated format.
  • Saves the result to date_conv.

The output is sent to the terminal.

1985-01-01 03:30:00

đź’ˇNote: Universal Time (UTC) is the primary standard 24-hour time clock by which the World regulates clocks and time.

YouTube Video

Method 4: Use time.localtime() and time.strftime()

This method imports the time library in conjunction with the time.localtime()and time.strftime() functions to convert Epoch Time into a Local Date Time representation.

import time epoch_time = 473398200
date_conv = time.strftime('%c', time.localtime(epoch_time))
print('Formatted Date:', date_conv)

Above, imports the time library. This allows the conversion of an Epoch Time to a readable Local Date Time format.

The following line declares an Epoch Time integer and saves it to epoch_time.

Next, the highlighted line converts the Epoch Time into a Local Date Time representation, converts to a string (strftime()) format and saves it to date_conv.

The output is sent to the terminal.

Formatted Date: Tue Jan 1 00:00:00 1985

Summary

These four (4) methods of converting an Epoch Time to a Date Time representation should give you enough information to select the best one for your coding requirements.

Good Luck & Happy Coding!


Programmer Humor – Blockchain

“Blockchains are like grappling hooks, in that it’s extremely cool when you encounter a problem for which they’re the right solution, but it happens way too rarely in real life.” source xkcd
Posted on Leave a comment

Blockchain Basics of Smart Contracts and Solidity

5/5 – (1 vote)

This article will give you an overview of the blockchain basics, transactions, and blocks.

In the previous article of this series, we looked at how to create your own token using Solidity and, specifically, events for transactions.

YouTube Video

Blockchain Basics

Blockchain technology enables us to write programs without having to think about the details of the underlying communication infrastructure.

However, just to be aware of some of the keywords which are commonly used when studying the infrastructure, we’ll name just a few among others (borrowed from the Solidity documentation):

  • mining,
  • elliptic-curve cryptography,
  • peer-to-peer network.

Although it’s interesting to see how these technologies work “under the blockchain’s hood”, the beneficial fact is that you don’t have to be closely familiar with them. You can just implicitly utilize them and maybe even forget they’re there. However, they represent some of the key elements which make up Web3.

đź’ˇ Note: Here, we need to establish a firm distinction between the terms Web3 in the blockchain context that is the subject of our interest when discussing Solidity, and the Semantic Web, sometimes known as Web 3.0, which is an extension of the World Wide Web, intended to make Internet data machine-readable.

Transactions

One of the main roles of a blockchain is to preserve the data and make it temper-resistant. In that sense, we can easily consider a blockchain as a globally shared, transactional database.

A blockchain network is globally shared because any party in the network can access and read its contents.

It is transactional because any change to the blockchain has to be accepted by almost all, or at least the majority of other members, depending on the blockchain implementation.

In database theory and practice having a transactional property means two things: the change to the database is either applied completely or not at all (see here); no other transaction can modify the effect of a transaction being executed.

We regularly ensure the equivalent behavior of our smart contracts by using error handling and control structures available in Solidity (docs).

Blockchain Security

There is also a strong security property that is inherent in the way how blockchain works: each new block header includes the hash of the previous block.

To alter a block in a blockchain, an attacker would have to re-mine the targeted block and all the following blocks, therefore creating a chain fork.

Also, an attacker would need to invest more total work than was invested in the original chain segment to get his chain fork accepted by the rest of the network (source).

The latter would require immense computing power and energy, making the entire effort unfeasible in theory.

However, there have been successful attacks on blockchain networks, mostly due to the smaller scale of a particular network, where a fraudulent consensus (the verification process) was less difficult to fabricate, block creation errors, or insufficient security measures (source).

Example Application

An example to the story above is already given in part by the example we did in the previous article: a smart contract for (simulated) currency transfer between any two parties.

There was a list of accounts holding balances in a cryptocurrency, Wei, and our smart contract supported transfers of a given amount of currency from the sender to the receiver.

What’s important in the context of such transactions is that the same amount of currency should always be “simultaneously” deducted from the sender’s account and added to the receiver’s account.

What we mean by “simultaneously” is not a matter of happening at the same moment, but happening with the same, but the opposite consequence, i.e. if the amount gets successfully deducted from the sender’s account, it has to be added to the receiver’s account.

If an error occurs after the amount is decreased from the sender’s account, but before the amount is added to the receiver’s account, the operation should revert to the smart contract’s previous state, as it was before the transaction started.

This way, the blockchain behaves in a consistent, transactional manner and warrants that the transaction will be done entirely or not at all.

In the context of everything said so far, with our subcurrency example in mind, we may ask ourselves:

đź’¬ Question: How would we enable an account owner to transfer the currency only from the account in his ownership?

The answer to the question is relatively simple:

đź’ˇ Answer: A transaction always bears the sender’s cryptographic signature, which serves as a seal in granting access to precisely defined modifications of (operations on) the database, such as currency transfer from the account owner originally holding the currency, to the account owner – receiver of the currency.

Blocks

There is no story about blockchain without actually mentioning the block itself. We will definitively return to talk about a block from some other angles, but with a security perspective in mind, we will focus on overcoming a significant obstacle known (in Bitcoin terms) a “double-spend attack” (source).

Some paragraphs ago, we mentioned a blockchain attack implying the majority of the network members’ acceptance, popularly known as the “51% attack”.

Let’s assume there are two transactions in the blockchain network, and each of them attempts to transfer all the account’s currency to another account, i.e. they will both attempt to empty the account.

Since there can be only one accepted, confirmed transaction (in contrast to two or more possible unconfirmed transactions), the transaction that gets confirmed first will end up bundled in a block. It is not under our control or, for that matter, not even a subject of our concern which block will be the first one.

What matters is that the second block will be rejected and the contention between the blocks/transactions will be resolved automatically, and the likelihood of the double-spend attack problem will decrease with each additional confirmation (source).

Blocks are sequentially added to a blockchain, ordered by the time of their arrival. Adding a block to the blockchain is mostly done in regular intervals, which last about 17 seconds for the Ethereum network.

Nonetheless, the blockchain tip can sometimes be reverted during the order selection mechanism. In that case, a confirmed block will get discarded and become a stale block, and the stale block’s successor blocks just get returned to the memory pool.

đź’ˇ Note: discarded or stale blocks are popularly and wrongly called orphan blocks. (source)

For block reversal to happen, two conditions should occur.

(1) First, multiple blocks should get created from the same parent block P simultaneously (by different miners) and get confirmed by the network, forming a fork with multiple subchains/branches at the tip (block) P of the blockchain.

As there are multiple successor block candidates (e.g. we will assume three blocks, A, B, and C), each block has an equal chance of becoming a permanent part of the blockchain.

Because blocks propagate through the blockchain network differently, miners will receive one of the blocks (A, B, or C) before the other blocks and start mining a new block on the block they received first.

(2) Second, one of the miners will mine and broadcast a new block, e.g.  C1 to the network before the other miners mine blocks A1 or B1 (these don’t exist as yet).

Since the fastest miner first received block C and then produced C1, it will extend the branch P-C, effectively making the branch P-C-C1 the longest one. Once the network detects there is a chain longer than the chains P-A and P-B, it will disqualify the candidate blocks A and B as stale blocks, thus resolving the contention.

All the miners who chose a different blockchain from the fork, e.g. P-A or P-B, and started building their blockchains on them, will experience a block reversal, as their chains will also get updated by the network with the blockchain P-C-C1 as the longest one.

The blockchain tip reversal gets less likely to happen as more blocks are confirmed and added to the blockchain. A common number of confirmations after we can be virtually certain that our block became a permanent part of a blockchain is six confirmations (source).

Conclusion

In this article, we first shortly made a short detour (sorry about that) to a missing part about Solidity events, and then boldly stepped into the direction of blockchain basics, transactions, and blocks.

First, we made friends with an event listener example based on web3.js library.

Second, we glanced at the blockchain basics. You already know the works: some basic terms, dry notes, etc. It’s just a scratch, really.

Third, we learned about how blockchain treats transactions and how it makes them feel secure,  shared and cared for. That’s why we mentioned some properties that make blockchain look like a database.

Fourth, we went into some light details on what a block is, what is its role in the blockchain ecosystem and what are some of the risks present in a blockchain network.


Learn Solidity Course

Solidity is the programming language of the future.

It gives you the rare and sought-after superpower to program against the “Internet Computer”, i.e., against decentralized Blockchains such as Ethereum, Binance Smart Chain, Ethereum Classic, Tron, and Avalanche – to mention just a few Blockchain infrastructures that support Solidity.

In particular, Solidity allows you to create smart contracts, i.e., pieces of code that automatically execute on specific conditions in a completely decentralized environment. For example, smart contracts empower you to create your own decentralized autonomous organizations (DAOs) that run on Blockchains without being subject to centralized control.

NFTs, DeFi, DAOs, and Blockchain-based games are all based on smart contracts.

This course is a simple, low-friction introduction to creating your first smart contract using the Remix IDE on the Ethereum testnet – without fluff, significant upfront costs to purchase ETH, or unnecessary complexity.

Posted on Leave a comment

How to Convert Multiple Text Files to a Single CSV in Python?

5/5 – (1 vote)

You can merge multiple text files to a single CSV file in Python by using the glob.glob('./*.txt') expression to filter out all path names of text files in a given folder. Then iterate over all those path names and use the open() function to read the file contents and write append them to the CSV.

Example: merge those files

Here’s the simple example:

import glob with open('my_file.csv', 'a') as csv_file: for path in glob.glob('./*.txt'): with open(path) as txt_file: txt = txt_file.read() + '\n' csv_file.write(txt) 

The resulting output CSV file shows that all text files have been merged:

You can replace the separator (e.g., from single empty space to comma) by using the txt.replace(' ', ',') function before writing it in the CSV:

import glob with open('my_file.csv', 'a') as csv_file: for path in glob.glob('./*.txt'): with open(path) as txt_file: txt = txt_file.read() + '\n' txt = txt.replace(' ', ',') csv_file.write(txt) 

The resulting CSV is neatly separated with comma characters:

In case you need some more advanced ways to convert the text files to the CSV, you may want to check out the Pandas read_csv() function to read the CSV into a DataFrame.

As soon as you have it as a DataFrame, you can do advanced processing such as merging, column selection, slicing, etc.

🌍 Related Tutorial: How to Read a CSV to a DataFrame?

Posted on Leave a comment

How to Convert Avro to CSV in Python?

4/5 – (1 vote)

đź’¬ Question: How to convert an .avro file to a .csv file in Python?

Solution:

To convert an Avro file my_file.avro to a CSV file my_file.csv, create a CSV writer using the csv module and iterate over all rows using the iterator returned by fastavro.reader(). Then write each row to a file using the writerow() function.

Here’s an example:

from fastavro import reader
import csv with open('my_file.avro', 'rb') as file_object: csv_file = csv.writer(open("my_file.csv", "w+")) head = True for x in reader(file_object): if head: # write header header = emp.keys() csv_file.writerow(header) head = False # write normal row csv_file.writerow(emp.values())

Related: This code is a modified and improved version of this source.

đź’ˇ Avro is a data serialization framework for RPCs (remote procedure calls) that uses JSON and binary format to serialize data.

đź’ˇ CSV stands for comma-separated values, so you have a row-based file format where values are separated by commas, and the file is named using the suffix .csv.

Posted on Leave a comment

For Loop with Two Variables (for i j in python)

5/5 – (1 vote)

for i j in python

The Python expression for i, j in XXX allows you to iterate over an iterable XXX of pairs of values. For example, to iterate over a list of tuples, this expression captures both tuple values in the loop variables i and j at once in each iteration.

Here’s an example:

for i, j in [('Alice', 18), ('Bob', 22)]: print(i, 'is', j, 'years old')

Output:

Alice is 18 years old
Bob is 22 years old

Notice how the first loop iteration captures i='Alice' and j=18, whereas the second loop iteration captures i='Bob' and j=22.

for i j in enumerate python

The Python expression for i, j in enumerate(iterable) allows you to loop over an iterable using variable i as a counter (0, 1, 2, …) and variable j to capture the iterable elements.

Here’s an example where we assign the counter values i=0,1,2 to the three elements in lst:

lst = ['Alice', 'Bob', 'Carl']
for i, j in enumerate(lst): print(i, j)

Output:

0 Alice
1 Bob
2 Carl

Notice how the loop iteration capture:

  • i=0 and j='Alice',
  • i=1 and j='Bob', and
  • i=2 and j='Carl'.

🌍 Learn More: The enumerate() function in Python.

Python enumerate()

for i j in zip python

The Python expression for i, j in zip(iter_1, iter_2) allows you to align the values of two iterables iter_1 and iter_2 in an ordered manner and iterate over the pairs of elements. We capture the two elements at the same positions in variables i and j.

Here’s an example that zips together the two lists [1,2,3] and [9,8,7,6,5].

for i, j in zip([1,2,3], [9,8,7,6,5]): print(i, j)

Output:

1 9
2 8
3 7

🌍 Learn More: The zip() function in Python.

for i j in list python

The Python expression for i, j in list allows you to iterate over a given list of pairs of elements (list of tuples or list of lists). In each iteration, this expression captures both pairs of elements at once in the loop variables i and j.

Here’s an example:

for i, j in [(1,9), (2,8), (3,7)]: print(i, j)

Output:

1 9
2 8
3 7

for i j k python

The Python expression for i, j, k in iterable allows you to iterate over a given list of triplets of elements (list of tuples or list of lists). In each iteration, this expression captures all three elements at once in the loop variables i and j.

Here’s an example:

for i, j, k in [(1,2,3), (4,5,6), (7,8,9)]: print(i, j, k)

Output:

1 2 3
4 5 6
7 8 9

for i j in a b python

Given two lists a and b, you can iterate over both lists at once by using the expression for i,j in zip(a,b).

Given one list ['a', 'b'], you can use the expression for i,j in enumerate(['a', 'b']) to iterate over the pairs of (identifier, list element) tuples.

Summary

The Python for loop is a powerful method to iterate over multiple iterables at once, usually with the help of the zip() or enumerate() functions.

for i, j in zip(range(10), range(10)): # (0,0), (1,1), ..., (9,9)

If a list element is an iterable by itself, you can capture all iterable elements using a comma-separated list when defining the loop variables.

for i,j,k in [(1,2,3), (4,5,6)]: # Do Something
Posted on Leave a comment

Python RegEx – Match Whitespace But Not Newline

5/5 – (1 vote)

Problem Formulation

đź’¬ Challenge: How to design a regular expression pattern that matches whitespace characters such as the empty space ' ' and the tabular character '\t', but not the newline character '\n'?

An example of this would be to replace all whitespaces (except newlines) between a space-delimited file with commas to obtain a CSV.

Method 1: Use Character Class

The character class pattern [ \t] matches one empty space ' ' or a tabular character '\t', but not a newline character. If you want to match an arbitrary number of empty spaces except for newlines, append the plus quantifier to the pattern like so: [ \t]+.

Here’s an example where you replace all separating whitespace (except newline) with a comma to receive a CSV formatted output:

import re txt = 'a \t b c\nd e f'
csv_txt = re.sub('[ \t]+', ',', txt)
print(csv_txt)

Output:

a,b,c
d,e,f

Why the space in the pattern [ \t]?

The reason there’s a space in the pattern is to match the empty space. The character class essentially is an OR relationship, i.e., one item within the character class is matched. For the given pattern, it matches either the empty space ' ' or the tabular character '\t'.

🌍 Learn More: Character Class (Character Set) — The Ultimate Guide for Python

Method 2: Match Individual Different Whitespace Characters

The previous method only matches the horizontal tab (U+0009) and breaking space (U+0020) characters. If you want more fine-grained control about which whitespace characters to match and which not, you can use the following baseline approach.

The following list of Unicode whitespace characters UNICODE_WHITESPACES contains all major whitespace variants you may want to check your string for. You can generate a character class using the string expression '[' + ''.join(UNICODE_WHITESPACES) + ']'.

Here’s a variant that finds all matches of whitespace characters in a given text:

import re UNICODE_WHITESPACES = [ "\u0009", # character tabulation "\u000a", # line feed "\u000b", # line tabulation "\u000c", # form feed "\u000d", # carriage return "\u0020", # space "\u0085", # next line "\u00a0", # no-break space "\u1680", # ogham space mark "\u2000", # en quad "\u2001", # em quad "\u2002", # en space "\u2003", # em space "\u2004", # three-per-em space "\u2005", # four-per-em space "\u2006", # six-per-em space "\u2007", # figure space "\u2008", # punctuation space "\u2009", # thin space "\u200A", # hair space "\u2028", # line separator "\u2029", # paragraph separator "\u202f", # narrow no-break space "\u205f", # medium mathematical space "\u3000", # ideographic space
] txt = ' \t\n\r'
pattern = '[' + ''.join(UNICODE_WHITESPACES) + ']'
matches = re.findall(pattern, txt)
print(matches)
# [' ', '\t', '\n', '\r']

Of course, you can restrict this to only contain whitespaces that are not newline-related.

Method 3: Match Individual Different Whitespaces Except Newlines

The following code snippet uses the UNICODE_WHITESPACES constant but comments out the newline whitespaces so that newline-related characters such as '\n' and '\r' are not matched anymore!

import re UNICODE_WHITESPACES = [ "\u0009", # character tabulation # "\u000a", # line feed "\u000b", # line tabulation "\u000c", # form feed # "\u000d", # carriage return "\u0020", # space # "\u0085", # next line "\u00a0", # no-break space "\u1680", # ogham space mark "\u2000", # en quad "\u2001", # em quad "\u2002", # en space "\u2003", # em space "\u2004", # three-per-em space "\u2005", # four-per-em space "\u2006", # six-per-em space "\u2007", # figure space "\u2008", # punctuation space "\u2009", # thin space "\u200A", # hair space # "\u2028", # line separator # "\u2029", # paragraph separator "\u202f", # narrow no-break space "\u205f", # medium mathematical space "\u3000", # ideographic space
] txt = ' \t\n\r'
pattern = '[' + ''.join(UNICODE_WHITESPACES) + ']'
matches = re.findall(pattern, txt)
print(matches)
# [' ', '\t']

Of course, you can comment out the individual whitespace Unicode characters you don’t want to match as required by your own application.