+- Sick Gaming (https://www.sickgaming.net)
+-- Forum: Programming (https://www.sickgaming.net/forum-76.html)
+--- Forum: Python (https://www.sickgaming.net/forum-83.html)
+--- Thread: [Tut] My Journey to Help Build a P2P Social Network – Database Code Structure (/thread-100845.html)
[Tut] My Journey to Help Build a P2P Social Network – Database Code Structure - xSicKxBot - 03-08-2023
My Journey to Help Build a P2P Social Network – Database Code Structure
<div>
<div class="kk-star-ratings kksr-auto kksr-align-left kksr-valign-top" data-payload='{"align":"left","id":"1188982","slug":"default","valign":"top","ignore":"","reference":"auto","class":"","count":"1","legendonly":"","readonly":"","score":"5","starsonly":"","best":"5","gap":"5","greet":"Rate this post","legend":"5\/5 - (1 vote)","size":"24","width":"142.5","_legend":"{score}\/{best} - ({count} {votes})","font_factor":"1.25"}'>
<div class="kksr-stars">
<div class="kksr-stars-inactive">
<div class="kksr-star" data-star="1" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" data-star="2" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" data-star="3" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" data-star="4" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" data-star="5" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
</p></div>
<div class="kksr-stars-active" style="width: 142.5px;">
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
</p></div>
</div>
<div class="kksr-legend" style="font-size: 19.2px;"> 5/5 – (1 vote) </div>
</p></div>
<p>Welcome to part 3 of this series, and thank you for sticking around! </p>
<ul>
<li><strong>Part 1</strong>: <a href="https://blog.finxter.com/how-i-started-my-journey-to-help-build-a-p2p-social-network-part-1-deciding-on-a-framework/" data-type="URL" data-id="https://blog.finxter.com/how-i-started-my-journey-to-help-build-a-p2p-social-network-part-1-deciding-on-a-framework/" target="_blank" rel="noreferrer noopener">The P2P Bootstrapping Problem – Starting My Journey To Help Build a P2P Social Network</a></li>
<li><strong>Part 2</strong>: <a href="https://blog.finxter.com/brainwaves-p2p-social-network-how-i-created-a-basic-server/" data-type="URL" data-id="https://blog.finxter.com/brainwaves-p2p-social-network-how-i-created-a-basic-server/" target="_blank" rel="noreferrer noopener">BrainWaves P2P Social Network – How I Created a Basic Server</a></li>
</ul>
<p>I’ve come to realize this might become a rather long series. The main reason for that is that it documents two things. This is the birth of an application and my personal journey in developing that application. I know parts 1 and 2 have been very wordy. This will change now. I promise that you will see a lot of code in this episode :-).</p>
<h2>Database Code</h2>
<p>So after that slight philosophical tidbit, it is time to dive into the actual database code. As I mentioned in the <a href="https://blog.finxter.com/brainwaves-p2p-social-network-how-i-created-a-basic-server/" data-type="URL" data-id="https://blog.finxter.com/brainwaves-p2p-social-network-how-i-created-a-basic-server/" target="_blank" rel="noreferrer noopener">previous article</a>, I chose to use <strong>Deta Space</strong> as the database provider. There are two reasons for this. The first is the ease of use and the second are its similarities to my favorite NoSQL database MongoDB. </p>
<p class="has-base-background-color has-background"><img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f4a1.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <strong>Recommended</strong>: Please <a href="https://blog.finxter.com/how-i-created-a-weekly-shopping-list-and-recipe-app-in-streamlit/" data-type="post" data-id="1096859" target="_blank" rel="noreferrer noopener">check my article on creating a shopping list</a> in Streamlit on how to set it up. It takes only a few minutes.</p>
<p>For reference, the server directory with all the code to get the FastAPI server working looks as follows:</p>
<pre class="wp-block-preformatted"><code>server
├── db.py
├── main.py
├── models.py
├── requirements.txt
├── .env</code></pre>
<p>All the database code will live in the <code>db.py</code> file. For the Pydantic models, I’ll use <code>models.py</code>.</p>
<h2>Database Functions</h2>
<p>The database functions are roughly divided into three parts. </p>
<ul>
<li>We first need functionality for everything related to <em>users</em>. </li>
<li>Next, we need to code that handles everything related to adding and managing <em>friends</em>. </li>
<li>The third part applies to all the code for managing <em>thoughts</em>. Thoughts are Peerbrain’s equal of messages/tweets.</li>
</ul>
<p>The file will also contain some helper functions to aid with managing the public keys of users. I’ll go into a lot more detail on this in the article about encryption.</p>
<p>To set up our <code>db.py</code> file we first need to import everything needed. As before, I’ll show the entire list and then explain what everything does once we write code that uses it.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">"""This file will contain all the database logic for our server module. It will leverage the Deta Base NoSQL database api.""" from datetime import datetime
import math
from typing import Union
import os
import logging
from pprint import pprint #pylint: disable=unused-import
from uuid import uuid4
from deta import Deta
from dotenv import load_dotenv
from passlib.context import CryptContext</pre>
<p>The <code>#pylint</code> comment you can see above is used to make sure <code>pylint</code> skips this import. I use <code><a href="https://blog.finxter.com/a-simple-guide-to-the-pprint-module-in-python/" data-type="post" data-id="508127" target="_blank" rel="noreferrer noopener">pprint</a></code> for displaying dictionaries in a readable way when testing. As I don’t use it anywhere in the actual code, <code>pylint</code> would start to fuss otherwise.</p>
<p class="has-base-background-color has-background"><img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f4a1.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <strong>Tip</strong>: <em>For those interested, </em><a href="https://pypi.org/project/pylint/" target="_blank" rel="noreferrer noopener"><em>pylint</em></a><em> is a great tool to check your code for consistency, errors and, code style. It is static, so it can’t detect errors occurring at runtime. I like it even so</em><img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f642.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" /><em>.</em></p>
<p>After having imported everything, I first initialize the database. The <code>load_dotenv()</code> below, first will load all my environment variables from the <code>.env</code> file. </p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">load_dotenv() #---DB INIT---#
DETA_KEY = os.getenv("DETA_KEY")
deta = Deta(DETA_KEY)
#---#
USERS = deta.Base("users")
THOUGHTS = deta.Base("thoughts")
KEYS = deta.Base("keys_db")
</pre>
<p>Once the variables are accessible, I can use the Deta API key to initialize Deta. Creating Bases in Deta is as easy as defining them with <code>deta.Base</code>. I can now call the variable names to perform CRUD operations when needed.</p>
<h2>Generate Password Hash</h2>
<p>The next part is very important. It will generate our password hash so the password is never readable. Even if someone has control of the database itself, they will not be able to use it. <code>Cryptcontext</code> itself is part of the <a href="https://passlib.readthedocs.io/en/stable/" target="_blank" rel="noreferrer noopener"><code>passlib</code> library</a>. This library can hash passwords in multiple ways.<img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f642.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" />.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">#---PW ENCRYPT INIT---#
pwd_context = CryptContext(schemes =["bcrypt"], deprecated="auto")
#---#
def gen_pw_hash(pw:str)->str: """Function that will use the CryptContext module to generate and return a hashed version of our password""" return pwd_context.hash(pw)</pre>
<h2>User Functions</h2>
<p>The first function of the user functions is the easiest. It uses Deta’s <code>fetch</code> method to retrieve all objects from a certain Base, <code>deta.users</code>, in our case.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">#---USER FUNCTIONS---#
def get_users() -> dict: """Function to return all users from our database""" try: return {user["username"]: user for user in USERS.fetch().items} except Exception as e: # Log the error or handle it appropriately print(f"Error fetching users: {e}") return {}</pre>
<p>The fact that the function returns the found users as a dictionary makes them easy to use with FastAPI. As we contact a database in this function and all the others in this block, a <code>try</code>–<code>except</code> block is necessary. </p>
<p>The next two functions are doing the same thing but with different parameters. They accept either a <code>username</code> or an <code>email</code>. </p>
<p>I am aware that these two could be combined into a single function with an <code>if</code><em>-statement</em>. I still do prefer the two separate functions, as I find them easier to use. Another argument I will make is also that the email search function is primarily an end user function. I plan to use searching by username in the background as a helper function for other functionality.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">def get_user_by_username(username:str)->Union[dict, None]: """Function that returns a User object if it is in the database. If not it returns a JSON object with the message no user exists for that username""" try: if (USERS.fetch({"username" : username}).items) == []: return {"Username" : "No user with username found"} else: return USERS.fetch({"username" : username}).items[0] except Exception as error_message: logging.exception(error_message) return None def get_user_by_email(email:str)->Union[dict, None]: """Function that returns a User object if it is in the database. If not it returns a JSON object with the message no user exists for that email address""" try: if (USERS.fetch({"email" : email}).items) == []: return {"Email" : "No user with email found"} else: return USERS.fetch({"email" : email}).items[0] except Exception as error_message: logging.exception(error_message) return None
</pre>
<p>The functions above both take a parameter that they use to filter the <code>fetch</code> request to the Deta Base <code>users</code>. </p>
<p>If that filtering results in an <a rel="noreferrer noopener" href="https://blog.finxter.com/how-to-create-an-empty-list-in-python/" data-type="post" data-id="453870" target="_blank">empty list</a> a proper message is returned. If the returned list is not empty, we use the <code>.items</code> method on the fetch object and return the first item of that <a rel="noreferrer noopener" href="https://blog.finxter.com/python-lists/" data-type="post" data-id="7332" target="_blank">list</a>. In both cases, this will be the<em> </em><code>user</code> object that contains the query string (email or username).</p>
<p>The entire sequence is run inside a <em>try-except</em> block as we are trying to contact a database.</p>
<h2>Reset User Password</h2>
<p>When working with user creation and databases, a function to reset a user’s password is required. The next function will take care of that.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">def change_password(username, pw_to_hash): """Function that takes a username and a password in plaintext. It will then hash that password> After that it creates a dictionary and tries to match the username to users in the database. If successful it overwrites the previous password hash. If not it returns a JSON message stating no user could be found for the username provided.""" hashed_pw = gen_pw_hash(pw_to_hash) update= {"hashed_pw": hashed_pw } try: user = get_user_by_username(username) user_key = user["key"] if not username in get_users(): return {"Username" : "Not Found"} else: return USERS.update(update, user_key), f"User {username} password changed!" except Exception as error_message: logging.exception(error_message) return None </pre>
<p>This function will take a username and a new password. It will first hash that password and then <a href="https://blog.finxter.com/how-to-create-a-dictionary-from-two-lists/" data-type="post" data-id="316802" target="_blank" rel="noreferrer noopener">create a dictionary</a>. Updates to a Deta Base are always performed by calling the update method with a dictionary. As in the previous functions, we always check if the username in question exists before calling the update. Also, don’t forget the <em>try-except</em> block!</p>
<h2>Create User</h2>
<p>The last function is our most important one :-). You can’t perform any operations on user objects if you have no way to create them! Take a look below to check out how we’ll handle that.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">def create_user(username:str, email:str, pw_to_hash:str)->None: """Function to create a new user. It takes three strings and inputs these into the new_user dictionary. The function then attempts to put this dictionary in the database""" new_user = {"username" : username, "key" : str(uuid4()), "hashed_pw" : gen_pw_hash(pw_to_hash), "email" : email, "friends" : [], "disabled" : False} try: return USERS.put(new_user) except Exception as error_message: logging.exception(error_message) return None
</pre>
<p>The user creation function will take a username, email, and password for now. It will probably become more complex in the future, but it serves our purposes for now. Like the Deta update method, creating a new item in the database requires a <a href="https://blog.finxter.com/python-dictionary/" data-type="post" data-id="5232" target="_blank" rel="noreferrer noopener">dictionary</a>. Some of the necessary attributes for the dictionary are generated inside the function. </p>
<p>The key needs to be unique, so we use Python’s <code>uuid4</code> module. The friend’s attribute will contain the usernames of other users but starts as an empty list. The disabled attribute, finally, is set to false. </p>
<p>After finishing the initialization, creating the object is a matter of calling the Deta <code>put </code>method. I hear some of you thinking that we don’t do any checks if the username or email already exists in the database. You are right, but I will perform these checks on the endpoint receiving the post request for user creation.</p>
<h2>Some Coding Thoughts and Learnings</h2>
<div class="wp-block-image">
<figure class="aligncenter size-full"><img loading="lazy" decoding="async" width="894" height="590" src="https://blog.finxter.com/wp-content/uploads/2023/03/image-84.png" alt="" class="wp-image-1189002" srcset="https://blog.finxter.com/wp-content/uploads/2023/03/image-84.png 894w, https://blog.finxter.com/wp-content/uploads/2023/03/image-84-300x198.png 300w, https://blog.finxter.com/wp-content/uploads/2023/03/image-84-768x507.png 768w" sizes="(max-width: 894px) 100vw, 894px" /><figcaption class="wp-element-caption"><a href="https://github.com/shandralor/PeerBrain" data-type="URL" data-id="https://github.com/shandralor/PeerBrain" target="_blank" rel="noreferrer noopener">GitHub</a> <img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f448.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" /> Join the open-source PeerBrain development community!</figcaption></figure>
</div>
<p>One thing that never ceases to amaze me is the amount of documentation I like to add. I do this first in the form of <strong>docstrings </strong>as it helps me keep track of what function does what. I find it boring most of the time, but in the end, it helps a lot! </p>
<p>The other part of documenting that I like is <strong>type hints</strong>. I admit they sometimes confuse me still, but I can see the merit they have when an application keeps growing. </p>
<p>We will handle the rest of the database function in the next article. See you there!</p>
<h2>Participate in Building the Decentralized Social Brain Network <img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f447.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" /></h2>
<p>As before, I state that I am completely self-taught. This means I’ll make mistakes. If you spot them, please <a href="https://discord.com/invite/Cn7jWhTckP" data-type="URL" data-id="https://discord.com/invite/Cn7jWhTckP" target="_blank" rel="noreferrer noopener">post them on Discord</a> so I can remedy them <img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f642.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" />. </p>
<p>As always, feel free to ask me questions or pass suggestions! And check out the GitHub repository for participation!</p>
<p class="has-base-background-color has-background"><img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f449.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <strong>GitHub</strong>: <a rel="noreferrer noopener" href="https://github.com/shandralor/PeerBrain" target="_blank">https://github.com/shandralor/PeerBrain</a></p></p>
</div>
https://www.sickgaming.net/blog/2023/03/07/my-journey-to-help-build-a-p2p-social-network-database-code-structure/