Category: Programming

Problem Formulation and Solution Overview

In this article, you’ll learn how to ignore (upper/lower/title) case characters when dealing with Strings in Python.

The main reason for ignoring the case of a String is to perform accurate String comparisons or to return accurate search results.

For example, the following three (3) Strings are not identical. If compared with each other, they would return False.

when life gives you lemons, make lemonade. WHEN LIFE GIVES YOU LEMONS, MAKE LEMONADE. When Life Gives You Lemons, Make Lemonade.

This is because each character in the ASCII Table assigns different numeric values for each key or key combination on the keyboard.

This article outlines various ways to ignore the case of Strings.

---

Question: How would we write code to compare Strings?

We can accomplish this task by one of the following options:

• Method 1: Use a lower() and a lambda
• Method 2: Use upper() and List Comprehension
• Method 3: Use title()
• Method 4: Use casefold()
• Method 5: Use lower() and Dictionary Comprehension
• Bonus: Convert CSV of Full Names to title case

---

Method 1: Use lower() and a lambda

This method uses lower() and a lambda to convert a List of Strings to lower case to search for an Employee.

staff = ['Andy', 'MAci', 'SteVe', 'DarCY', 'Harvey']
staff_lower = list((map(lambda x: x.lower(), staff)))
print(staff_lower)

Above declares a List of Rivers Clothing Employees. If we were searching for an Employee, such as Darcy, using this List, no results would return as Darcy and DarCY are not the same. Let’s fix this issue.

Info: A lambda function is a one-liner that, in this case, converts each List element in staff to lower case. Then, converts to a map() object and back to a List. The output saves to staff_lower and is output to the terminal.

['andy', 'maci', 'steve', 'darcy', 'harvey']

If a comparison was made now, it would find the correct Employee.

if ('Darcy'.lower() == staff_lower[3]): print('Match Found!')

Match Found!

---

Method 2: Use upper() and List Comprehension

This method uses upper() and a list comprehension to convert a List of Strings to upper case and search for an Employee.

brands = ['ChaNnel', 'CLINique', 'DIOR', 'Lancome', 'MurAD']
brands_upper = [x.upper() for x in brands]
print(brands_upper)

Above declares a List of the Top five (5) Beauty Brands in the world. If we were searching for a Company, such as Channel, using this List, no results would return as Channel and ChaNnel are not the same. Let’s fix this issue.

A List Comprehension is called and converts each element in brands to upper case. The results are saved to brands_upper and output to the terminal.

['CHANEL', 'CLINIQUE', 'DIOR', 'LANCOME', 'MURAD']

If a comparison was made now, it would find the correct Brand.

my_fav = 'Murad'
if (my_fav.upper() == brands_upper[4]): print(f'{my_fav} Found!')

Murad Found!

---

Method 3: Use title()

This method uses title() to convert each word in a String to upper case. Great use of this is to convert a Full Name to title case. This ensures both the First, Middle and Last Names are as expected.

full_name = 'jessica a. barker'.title()
print(full_name)

Above accepts a string, appends the title() method to the string, saves it to full_name and outputs to the terminal.

Jessica A. Barker

---

Method 4: Use casefold() and a Dictionary

This method uses casefold() to convert a String to lower case. This method does more than lower(): it uses Unicode Case Folding Rules.

phrase = 'Nichts ist unmöglich für den Unwilligen!'
cfold = phrase.casefold()
print(cfold)

Above creates a quote in German and saves it to phrase. This ensures all characters are correctly converted to lower case, thus making a caseless match.

nichts ist unmöglich für den unwilligen!

What does this say?

---

Method 5: Use lower() and Dictionary Comprehension

This method uses Use lower() to convert Dictionary values into lower case using Dictionary Comprehension.

employees = {'Sandy': 'Coder', 'Kevin': 'Network Specialist', 'Amy': 'Designer'}
result = {k: v.lower() for k, v in employees.items()}
print(result)

Above creates a Dictionary containing Rivers Employees Names and associated Job Title. This saves to employees.

Next, Dictionary Comprehension is used to loop through the values in the key:value Dictionary pairs and convert the values to lower case. The output saves to result and is output to the terminal.

{'Sandy': 'coder', 'Kevin': 'network specialist', 'Amy': 'designer'}

---

Bonus: CSV Column to title() case

In this Bonus section, a CSV file you are working with contains a column of Full Names. You want to ensure all data entered in this column is in title() case. This can be accomplished by running the following code.

Contents of finxter_users.csv file

FID	Full_Name
30022145	sally jenkins
30022192	joyce hamilton
30022331	allan thompson
30022345	harry stiller
30022157	ben hawkins

import pandas as pd users = pd.read_csv('finxter_users.csv')
users['Full_Name'] = users['Full_Name'].str.title()
print(users)

Above imports the Pandas library. For installation instructions, click here.

Next, the finxter_users.csv file (contents shown above) is read and saved to the DataFrame users.

The highlighted line directly accesses all values in the users['Full_Name'] column and converts each entry to title() case. This saves back to users['Full_Name'] and is output to the terminal.

	FID	Full_Name
0	30022145	Sally Jenkins
1	30022192	Joyce Hamilton
2	30022331	Allan Thompson
3	30022345	Harry Stiller
4	30022157	Ben Hawkins

Note: Don’t forget to save the DataFrame to keep the changes.

---

Summary

These six (6) methods of ignoring cases in Strings should give you enough information to select the best one for your coding requirements.

Programmer Humor

Question: Why do programmers always mix up Halloween and Christmas?
Answer: Because Oct 31 equals Dec 25.

(If you didn’t get this, read our articles on the oct() and int() Python built-in functions!)

What is KML?

Definition: The Keyhole Markup Language (KML) is a file format for displaying geographic data in Google Earth or other so-called “Earth Browsers”. Similarly to XML, KML uses a tag-based structure with nested elements and attributes.

How to Convert KML to CSV in Python?

You can convert a .kml to a .csv file in Python by using the BeautifulSoup and the csv libraries. You use the former to read the XML-structured KML file and the latter to write the CSV file row by row.

Here’s the code example inspired but modified from this GitHub repository. You can copy&paste it in the directory where your KML file resides and change the input and output filenames at the beginning to convert your own KML to a CSV in Python:

from bs4 import BeautifulSoup
import csv infile = 'my_file.kml'
outfile = 'my_file.csv' with open(infile, 'r') as f: s = BeautifulSoup(f, 'xml') with open(outfile, 'wb') as csvfile: writer = csv.writer(csvfile) for coords in s.find_all('coordinates'): # Take coordinate string from KML and break it up into [Lat,Lon,Lat,Lon...] to get CSV row space_splits = coords.string.split(" ") row = [] for split in space_splits[1:]: # Note: because of the space between <coordinates>" "-80.123, we slice [1:] comma_split = split.split(',') # lattitude row.append(comma_split[1]) # longitude row.append(comma_split[0]) writer.writerow(row)

Example Conversion

We use the following sample KML file as 'my_file.kml':

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2"> <Document> <name>KML Samples</name> <open>1</open> <description>Unleash your creativity with the help of these examples!</description> <Style id="downArrowIcon"> <IconStyle> <Icon> <href>http://maps.google.com/mapfiles/kml/pal4/icon28.png</href> </Icon> </IconStyle> </Style> <Style id="globeIcon"> <IconStyle> <Icon> <href>http://maps.google.com/mapfiles/kml/pal3/icon19.png</href> </Icon> </IconStyle> <LineStyle> <width>2</width> </LineStyle> </Style> <Style id="transPurpleLineGreenPoly"> <LineStyle> <color>7fff00ff</color> <width>4</width> </LineStyle> <PolyStyle> <color>7f00ff00</color> </PolyStyle> </Style> <Style id="yellowLineGreenPoly"> <LineStyle> <color>7f00ffff</color> <width>4</width> </LineStyle> <PolyStyle> <color>7f00ff00</color> </PolyStyle> </Style> <Style id="thickBlackLine"> <LineStyle> <color>87000000</color> <width>10</width> </LineStyle> </Style> <Style id="redLineBluePoly"> <LineStyle> <color>ff0000ff</color> </LineStyle> <PolyStyle> <color>ffff0000</color> </PolyStyle> </Style> <Style id="blueLineRedPoly"> <LineStyle> <color>ffff0000</color> </LineStyle> <PolyStyle> <color>ff0000ff</color> </PolyStyle> </Style> <Style id="transRedPoly"> <LineStyle> <width>1.5</width> </LineStyle> <PolyStyle> <color>7d0000ff</color> </PolyStyle> </Style> <Style id="transBluePoly"> <LineStyle> <width>1.5</width> </LineStyle> <PolyStyle> <color>7dff0000</color> </PolyStyle> </Style> <Style id="transGreenPoly"> <LineStyle> <width>1.5</width> </LineStyle> <PolyStyle> <color>7d00ff00</color> </PolyStyle> </Style> <Style id="transYellowPoly"> <LineStyle> <width>1.5</width> </LineStyle> <PolyStyle> <color>7d00ffff</color> </PolyStyle> </Style> <Style id="noDrivingDirections"> <BalloonStyle> <text><![CDATA[ <b>$[name]</b> <br /><br /> $[description] ]]></text> </BalloonStyle> </Style> <Folder> <name>Placemarks</name> <description>These are just some of the different kinds of placemarks with which you can mark your favorite places</description> <LookAt> <longitude>-122.0839597145766</longitude> <latitude>37.42222904525232</latitude> <altitude>0</altitude> <heading>-148.4122922628044</heading> <tilt>40.5575073395506</tilt> <range>500.6566641072245</range> </LookAt> <Placemark> <name>Simple placemark</name> <description>Attached to the ground. Intelligently places itself at the height of the underlying terrain.</description> <Point> <coordinates>-122.0822035425683,37.42228990140251,0</coordinates> </Point> </Placemark> <Placemark> <name>Floating placemark</name> <visibility>0</visibility> <description>Floats a defined distance above the ground.</description> <LookAt> <longitude>-122.0839597145766</longitude> <latitude>37.42222904525232</latitude> <altitude>0</altitude> <heading>-148.4122922628044</heading> <tilt>40.5575073395506</tilt> <range>500.6566641072245</range> </LookAt> <styleUrl>#downArrowIcon</styleUrl> <Point> <altitudeMode>relativeToGround</altitudeMode> <coordinates>-122.084075,37.4220033612141,50</coordinates> </Point> </Placemark> <Placemark> <name>Extruded placemark</name> <visibility>0</visibility> <description>Tethered to the ground by a customizable &quot;tail&quot;</description> <LookAt> <longitude>-122.0845787421525</longitude> <latitude>37.42215078737763</latitude> <altitude>0</altitude> <heading>-148.4126684946234</heading> <tilt>40.55750733918048</tilt> <range>365.2646606980322</range> </LookAt> <styleUrl>#globeIcon</styleUrl> <Point> <extrude>1</extrude> <altitudeMode>relativeToGround</altitudeMode> <coordinates>-122.0857667006183,37.42156927867553,50</coordinates> </Point> </Placemark> </Folder> <Folder> <name>Styles and Markup</name> <visibility>0</visibility> <description>With KML it is easy to create rich, descriptive markup to annotate and enrich your placemarks</description> <LookAt> <longitude>-122.0845787422371</longitude> <latitude>37.42215078726837</latitude> <altitude>0</altitude> <heading>-148.4126777488172</heading> <tilt>40.55750733930874</tilt> <range>365.2646826292919</range> </LookAt> <styleUrl>#noDrivingDirections</styleUrl> <Document> <name>Highlighted Icon</name> <visibility>0</visibility> <description>Place your mouse over the icon to see it display the new icon</description> <LookAt> <longitude>-122.0856552124024</longitude> <latitude>37.4224281311035</latitude> <altitude>0</altitude> <heading>0</heading> <tilt>0</tilt> <range>265.8520424250024</range> </LookAt> <Style id="highlightPlacemark"> <IconStyle> <Icon> <href>http://maps.google.com/mapfiles/kml/paddle/red-stars.png</href> </Icon> </IconStyle> </Style> <Style id="normalPlacemark"> <IconStyle> <Icon> <href>http://maps.google.com/mapfiles/kml/paddle/wht-blank.png</href> </Icon> </IconStyle> </Style> <StyleMap id="exampleStyleMap"> <Pair> <key>normal</key> <styleUrl>#normalPlacemark</styleUrl> </Pair> <Pair> <key>highlight</key> <styleUrl>#highlightPlacemark</styleUrl> </Pair> </StyleMap> <Placemark> <name>Roll over this icon</name> <visibility>0</visibility> <styleUrl>#exampleStyleMap</styleUrl> <Point> <coordinates>-122.0856545755255,37.42243077405461,0</coordinates> </Point> </Placemark> </Document> <Placemark> <name>Descriptive HTML</name> <visibility>0</visibility> <description><![CDATA[Click on the blue link!<br><br>
Placemark descriptions can be enriched by using many standard HTML tags.<br>
For example:
<hr>
Styles:<br>
<i>Italics</i>, <b>Bold</b>, <u>Underlined</u>, <s>Strike Out</s>, subscript<sub>subscript</sub>, superscript<sup>superscript</sup>, <big>Big</big>, <small>Small</small>, <tt>Typewriter</tt>, <em>Emphasized</em>, <strong>Strong</strong>, <code>Code</code>
<hr>
Fonts:<br> <font color="red">red by name</font>, <font color="#408010">leaf green by hexadecimal RGB</font>
<br>
<font size=1>size 1</font>, <font size=2>size 2</font>, <font size=3>size 3</font>, <font size=4>size 4</font>, <font size=5>size 5</font>, <font size=6>size 6</font>, <font size=7>size 7</font>
<br>
<font face=times>Times</font>, <font face=verdana>Verdana</font>, <font face=arial>Arial</font><br>
<hr>
Links: <br>
<a href="http://earth.google.com/">Google Earth!</a>
<br> or: Check out our website at www.google.com
<hr>
Alignment:<br>
<p align=left>left</p>
<p align=center>center</p>
<p align=right>right</p>
<hr>
Ordered Lists:<br>
<ol><li>First</li><li>Second</li><li>Third</li></ol>
<ol type="a"><li>First</li><li>Second</li><li>Third</li></ol>
<ol type="A"><li>First</li><li>Second</li><li>Third</li></ol>
<hr>
Unordered Lists:<br>
<ul><li>A</li><li>B</li><li>C</li></ul>
<ul type="circle"><li>A</li><li>B</li><li>C</li></ul>
<ul type="square"><li>A</li><li>B</li><li>C</li></ul>
<hr>
Definitions:<br>
<dl>
<dt>Google:</dt><dd>The best thing since sliced bread</dd>
</dl>
<hr>
Centered:<br><center>
Time present and time past<br>
Are both perhaps present in time future,<br>
And time future contained in time past.<br>
If all time is eternally present<br>
All time is unredeemable.<br>
</center>
<hr>
Block Quote:
<br>
<blockquote>
We shall not cease from exploration<br>
And the end of all our exploring<br>
Will be to arrive where we started<br>
And know the place for the first time.<br>
<i> – T.S. Eliot</i>
</blockquote>
<br>
<hr>
Headings:<br>
<h1>Header 1</h1>
<h2>Header 2</h2>
<h3>Header 3</h3>
<h3>Header 4</h4>
<h3>Header 5</h5>
<hr>
Images:<br>
<i>Remote image</i><br>
<img src="//developers.google.com/kml/documentation/images/googleSample.png"><br>
<i>Scaled image</i><br>
<img src="//developers.google.com/kml/documentation/images/googleSample.png" width=100><br>
<hr>
Simple Tables:<br>
<table border="1" padding="1">
<tr><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td></tr>
<tr><td>a</td><td>b</td><td>c</td><td>d</td><td>e</td></tr>
</table>
<br>
[Did you notice that double-clicking on the placemark doesn't cause the viewer to take you anywhere? This is because it is possible to directly author a "placeless placemark". If you look at the code for this example, you will see that it has neither a point coordinate nor a LookAt element.]]]></description> </Placemark> </Folder> <Folder> <name>Ground Overlays</name> <visibility>0</visibility> <description>Examples of ground overlays</description> <GroundOverlay> <name>Large-scale overlay on terrain</name> <visibility>0</visibility> <description>Overlay shows Mount Etna erupting on July 13th, 2001.</description> <LookAt> <longitude>15.02468937557116</longitude> <latitude>37.67395167941667</latitude> <altitude>0</altitude> <heading>-16.5581842842829</heading> <tilt>58.31228652890705</tilt> <range>30350.36838438907</range> </LookAt> <Icon> <href>http://developers.google.com/kml/documentation/images/etna.jpg</href> </Icon> <LatLonBox> <north>37.91904192681665</north> <south>37.46543388598137</south> <east>15.35832653742206</east> <west>14.60128369746704</west> <rotation>-0.1556640799496235</rotation> </LatLonBox> </GroundOverlay> </Folder> <Folder> <name>Screen Overlays</name> <visibility>0</visibility> <description>Screen overlays have to be authored directly in KML. These examples illustrate absolute and dynamic positioning in screen space.</description> <ScreenOverlay> <name>Simple crosshairs</name> <visibility>0</visibility> <description>This screen overlay uses fractional positioning to put the image in the exact center of the screen</description> <Icon> <href>http://developers.google.com/kml/documentation/images/crosshairs.png</href> </Icon> <overlayXY x="0.5" y="0.5" xunits="fraction" yunits="fraction"/> <screenXY x="0.5" y="0.5" xunits="fraction" yunits="fraction"/> <rotationXY x="0.5" y="0.5" xunits="fraction" yunits="fraction"/> <size x="0" y="0" xunits="pixels" yunits="pixels"/> </ScreenOverlay> <ScreenOverlay> <name>Absolute Positioning: Top left</name> <visibility>0</visibility> <Icon> <href>http://developers.google.com/kml/documentation/images/top_left.jpg</href> </Icon> <overlayXY x="0" y="1" xunits="fraction" yunits="fraction"/> <screenXY x="0" y="1" xunits="fraction" yunits="fraction"/> <rotationXY x="0" y="0" xunits="fraction" yunits="fraction"/> <size x="0" y="0" xunits="fraction" yunits="fraction"/> </ScreenOverlay> <ScreenOverlay> <name>Absolute Positioning: Top right</name> <visibility>0</visibility> <Icon> <href>http://developers.google.com/kml/documentation/images/top_right.jpg</href> </Icon> <overlayXY x="1" y="1" xunits="fraction" yunits="fraction"/> <screenXY x="1" y="1" xunits="fraction" yunits="fraction"/> <rotationXY x="0" y="0" xunits="fraction" yunits="fraction"/> <size x="0" y="0" xunits="fraction" yunits="fraction"/> </ScreenOverlay> <ScreenOverlay> <name>Absolute Positioning: Bottom left</name> <visibility>0</visibility> <Icon> <href>http://developers.google.com/kml/documentation/images/bottom_left.jpg</href> </Icon> <overlayXY x="0" y="-1" xunits="fraction" yunits="fraction"/> <screenXY x="0" y="0" xunits="fraction" yunits="fraction"/> <rotationXY x="0" y="0" xunits="fraction" yunits="fraction"/> <size x="0" y="0" xunits="fraction" yunits="fraction"/> </ScreenOverlay> <ScreenOverlay> <name>Absolute Positioning: Bottom right</name> <visibility>0</visibility> <Icon> <href>http://developers.google.com/kml/documentation/images/bottom_right.jpg</href> </Icon> <overlayXY x="1" y="-1" xunits="fraction" yunits="fraction"/> <screenXY x="1" y="0" xunits="fraction" yunits="fraction"/> <rotationXY x="0" y="0" xunits="fraction" yunits="fraction"/> <size x="0" y="0" xunits="fraction" yunits="fraction"/> </ScreenOverlay> <ScreenOverlay> <name>Dynamic Positioning: Top of screen</name> <visibility>0</visibility> <Icon> <href>http://developers.google.com/kml/documentation/images/dynamic_screenoverlay.jpg</href> </Icon> <overlayXY x="0" y="1" xunits="fraction" yunits="fraction"/> <screenXY x="0" y="1" xunits="fraction" yunits="fraction"/> <rotationXY x="0" y="0" xunits="fraction" yunits="fraction"/> <size x="1" y="0.2" xunits="fraction" yunits="fraction"/> </ScreenOverlay> <ScreenOverlay> <name>Dynamic Positioning: Right of screen</name> <visibility>0</visibility> <Icon> <href>http://developers.google.com/kml/documentation/images/dynamic_right.jpg</href> </Icon> <overlayXY x="1" y="1" xunits="fraction" yunits="fraction"/> <screenXY x="1" y="1" xunits="fraction" yunits="fraction"/> <rotationXY x="0" y="0" xunits="fraction" yunits="fraction"/> <size x="0" y="1" xunits="fraction" yunits="fraction"/> </ScreenOverlay> </Folder> <Folder> <name>Paths</name> <visibility>0</visibility> <description>Examples of paths. Note that the tessellate tag is by default set to 0. If you want to create tessellated lines, they must be authored (or edited) directly in KML.</description> <Placemark> <name>Tessellated</name> <visibility>0</visibility> <description><![CDATA[If the <tessellate> tag has a value of 1, the line will contour to the underlying terrain]]></description> <LookAt> <longitude>-112.0822680013139</longitude> <latitude>36.09825589333556</latitude> <altitude>0</altitude> <heading>103.8120432044965</heading> <tilt>62.04855796276328</tilt> <range>2889.145007690472</range> </LookAt> <LineString> <tessellate>1</tessellate> <coordinates> -112.0814237830345,36.10677870477137,0 -112.0870267752693,36.0905099328766,0 </coordinates> </LineString> </Placemark> <Placemark> <name>Untessellated</name> <visibility>0</visibility> <description><![CDATA[If the <tessellate> tag has a value of 0, the line follow a simple straight-line path from point to point]]></description> <LookAt> <longitude>-112.0822680013139</longitude> <latitude>36.09825589333556</latitude> <altitude>0</altitude> <heading>103.8120432044965</heading> <tilt>62.04855796276328</tilt> <range>2889.145007690472</range> </LookAt> <LineString> <tessellate>0</tessellate> <coordinates> -112.080622229595,36.10673460007995,0 -112.085242575315,36.09049598612422,0 </coordinates> </LineString> </Placemark> <Placemark> <name>Absolute</name> <visibility>0</visibility> <description>Transparent purple line</description> <LookAt> <longitude>-112.2719329043177</longitude> <latitude>36.08890633450894</latitude> <altitude>0</altitude> <heading>-106.8161545998597</heading> <tilt>44.60763714063257</tilt> <range>2569.386744398339</range> </LookAt> <styleUrl>#transPurpleLineGreenPoly</styleUrl> <LineString> <tessellate>1</tessellate> <altitudeMode>absolute</altitudeMode> <coordinates> -112.265654928602,36.09447672602546,2357 -112.2660384528238,36.09342608838671,2357 -112.2668139013453,36.09251058776881,2357 -112.2677826834445,36.09189827357996,2357 -112.2688557510952,36.0913137941187,2357 -112.2694810717219,36.0903677207521,2357 -112.2695268555611,36.08932171487285,2357 -112.2690144567276,36.08850916060472,2357 -112.2681528815339,36.08753813597956,2357 -112.2670588176031,36.08682685262568,2357 -112.2657374587321,36.08646312301303,2357 </coordinates> </LineString> </Placemark> <Placemark> <name>Absolute Extruded</name> <visibility>0</visibility> <description>Transparent green wall with yellow outlines</description> <LookAt> <longitude>-112.2643334742529</longitude> <latitude>36.08563154742419</latitude> <altitude>0</altitude> <heading>-125.7518698668815</heading> <tilt>44.61038665812578</tilt> <range>4451.842204068102</range> </LookAt> <styleUrl>#yellowLineGreenPoly</styleUrl> <LineString> <extrude>1</extrude> <tessellate>1</tessellate> <altitudeMode>absolute</altitudeMode> <coordinates> -112.2550785337791,36.07954952145647,2357 -112.2549277039738,36.08117083492122,2357 -112.2552505069063,36.08260761307279,2357 -112.2564540158376,36.08395660588506,2357 -112.2580238976449,36.08511401044813,2357 -112.2595218489022,36.08584355239394,2357 -112.2608216347552,36.08612634548589,2357 -112.262073428656,36.08626019085147,2357 -112.2633204928495,36.08621519860091,2357 -112.2644963846444,36.08627897945274,2357 -112.2656969554589,36.08649599090644,2357 </coordinates> </LineString> </Placemark> <Placemark> <name>Relative</name> <visibility>0</visibility> <description>Black line (10 pixels wide), height tracks terrain</description> <LookAt> <longitude>-112.2580438551384</longitude> <latitude>36.1072674824385</latitude> <altitude>0</altitude> <heading>4.947421249553717</heading> <tilt>44.61324882043339</tilt> <range>2927.61105910266</range> </LookAt> <styleUrl>#thickBlackLine</styleUrl> <LineString> <tessellate>1</tessellate> <altitudeMode>relativeToGround</altitudeMode> <coordinates> -112.2532845153347,36.09886943729116,645 -112.2540466121145,36.09919570465255,645 -112.254734666947,36.09984998366178,645 -112.255493345654,36.10051310621746,645 -112.2563157098468,36.10108441943419,645 -112.2568033076439,36.10159722088088,645 -112.257494011321,36.10204323542867,645 -112.2584106072308,36.10229131995655,645 -112.2596588987972,36.10240001286358,645 -112.2610581199487,36.10213176873407,645 -112.2626285262793,36.10157011437219,645 </coordinates> </LineString> </Placemark> <Placemark> <name>Relative Extruded</name> <visibility>0</visibility> <description>Opaque blue walls with red outline, height tracks terrain</description> <LookAt> <longitude>-112.2683594333433</longitude> <latitude>36.09884362144909</latitude> <altitude>0</altitude> <heading>-72.24271551768405</heading> <tilt>44.60855445139561</tilt> <range>2184.193522571467</range> </LookAt> <styleUrl>#redLineBluePoly</styleUrl> <LineString> <extrude>1</extrude> <tessellate>1</tessellate> <altitudeMode>relativeToGround</altitudeMode> <coordinates> -112.2656634181359,36.09445214722695,630 -112.2652238941097,36.09520916122063,630 -112.2645079986395,36.09580763864907,630 -112.2638827428817,36.09628572284063,630 -112.2635746835406,36.09679275951239,630 -112.2635711822407,36.09740038871899,630 -112.2640296531825,36.09804913435539,630 -112.264327720538,36.09880337400301,630 -112.2642436562271,36.09963644790288,630 -112.2639148687042,36.10055381117246,630 -112.2626894973474,36.10149062823369,630 </coordinates> </LineString> </Placemark> </Folder> <Folder> <name>Polygons</name> <visibility>0</visibility> <description>Examples of polygon shapes</description> <Folder> <name>Google Campus</name> <visibility>0</visibility> <description>A collection showing how easy it is to create 3-dimensional buildings</description> <LookAt> <longitude>-122.084120030116</longitude> <latitude>37.42174011925477</latitude> <altitude>0</altitude> <heading>-34.82469740081282</heading> <tilt>53.454348562403</tilt> <range>276.7870053764046</range> </LookAt> <Placemark> <name>Building 40</name> <visibility>0</visibility> <styleUrl>#transRedPoly</styleUrl> <Polygon> <extrude>1</extrude> <altitudeMode>relativeToGround</altitudeMode> <outerBoundaryIs> <LinearRing> <coordinates> -122.0848938459612,37.42257124044786,17 -122.0849580979198,37.42211922626856,17 -122.0847469573047,37.42207183952619,17 -122.0845725380962,37.42209006729676,17 -122.0845954886723,37.42215932700895,17 -122.0838521118269,37.42227278564371,17 -122.083792243335,37.42203539112084,17 -122.0835076656616,37.42209006957106,17 -122.0834709464152,37.42200987395161,17 -122.0831221085748,37.4221046494946,17 -122.0829247374572,37.42226503990386,17 -122.0829339169385,37.42231242843094,17 -122.0833837359737,37.42225046087618,17 -122.0833607854248,37.42234159228745,17 -122.0834204551642,37.42237075460644,17 -122.083659133885,37.42251292011001,17 -122.0839758438952,37.42265873093781,17 -122.0842374743331,37.42265143972521,17 -122.0845036949503,37.4226514386435,17 -122.0848020460801,37.42261133916315,17 -122.0847882750515,37.42256395055121,17 -122.0848938459612,37.42257124044786,17 </coordinates> </LinearRing> </outerBoundaryIs> </Polygon> </Placemark> <Placemark> <name>Building 41</name> <visibility>0</visibility> <styleUrl>#transBluePoly</styleUrl> <Polygon> <extrude>1</extrude> <altitudeMode>relativeToGround</altitudeMode> <outerBoundaryIs> <LinearRing> <coordinates> -122.0857412771483,37.42227033155257,17 -122.0858169768481,37.42231408832346,17 -122.085852582875,37.42230337469744,17 -122.0858799945639,37.42225686138789,17 -122.0858860101409,37.4222311076138,17 -122.0858069157288,37.42220250173855,17 -122.0858379542653,37.42214027058678,17 -122.0856732640519,37.42208690214408,17 -122.0856022926407,37.42214885429042,17 -122.0855902778436,37.422128290487,17 -122.0855841672237,37.42208171967246,17 -122.0854852065741,37.42210455874995,17 -122.0855067264352,37.42214267949824,17 -122.0854430712915,37.42212783846172,17 -122.0850990714904,37.42251282407603,17 -122.0856769818632,37.42281815323651,17 -122.0860162273783,37.42244918858722,17 -122.0857260327004,37.42229239604253,17 -122.0857412771483,37.42227033155257,17 </coordinates> </LinearRing> </outerBoundaryIs> </Polygon> </Placemark> <Placemark> <name>Building 42</name> <visibility>0</visibility> <styleUrl>#transGreenPoly</styleUrl> <Polygon> <extrude>1</extrude> <altitudeMode>relativeToGround</altitudeMode> <outerBoundaryIs> <LinearRing> <coordinates> -122.0857862287242,37.42136208886969,25 -122.0857312990603,37.42136935989481,25 -122.0857312992918,37.42140934910903,25 -122.0856077073679,37.42138390166565,25 -122.0855802426516,37.42137299550869,25 -122.0852186221971,37.42137299504316,25 -122.0852277765639,37.42161656508265,25 -122.0852598189347,37.42160565894403,25 -122.0852598185499,37.42168200156,25 -122.0852369311478,37.42170017860346,25 -122.0852643957828,37.42176197982575,25 -122.0853239032746,37.42176198013907,25 -122.0853559454324,37.421852864452,25 -122.0854108752463,37.42188921823734,25 -122.0854795379357,37.42189285337048,25 -122.0855436229819,37.42188921797546,25 -122.0856260178042,37.42186013499926,25 -122.085937287963,37.42186013453605,25 -122.0859428718666,37.42160898590042,25 -122.0859655469861,37.42157992759144,25 -122.0858640462341,37.42147115002957,25 -122.0858548911215,37.42140571326184,25 -122.0858091162768,37.4214057134039,25 -122.0857862287242,37.42136208886969,25 </coordinates> </LinearRing> </outerBoundaryIs> </Polygon> </Placemark> <Placemark> <name>Building 43</name> <visibility>0</visibility> <styleUrl>#transYellowPoly</styleUrl> <Polygon> <extrude>1</extrude> <altitudeMode>relativeToGround</altitudeMode> <outerBoundaryIs> <LinearRing> <coordinates> -122.0844371128284,37.42177253003091,19 -122.0845118855746,37.42191111542896,19 -122.0850470999805,37.42178755121535,19 -122.0850719913391,37.42143663023161,19 -122.084916406232,37.42137237822116,19 -122.0842193868167,37.42137237801626,19 -122.08421938659,37.42147617161496,19 -122.0838086419991,37.4214613409357,19 -122.0837899728564,37.42131306410796,19 -122.0832796534698,37.42129328840593,19 -122.0832609819207,37.42139213944298,19 -122.0829373621737,37.42137236399876,19 -122.0829062425667,37.42151569778871,19 -122.0828502269665,37.42176282576465,19 -122.0829435788635,37.42176776969635,19 -122.083217411188,37.42179248552686,19 -122.0835970430103,37.4217480074456,19 -122.0839455556771,37.42169364237603,19 -122.0840077894637,37.42176283815853,19 -122.084113587521,37.42174801104392,19 -122.0840762473784,37.42171341292375,19 -122.0841447047739,37.42167881534569,19 -122.084144704223,37.42181720660197,19 -122.0842503333074,37.4218170700446,19 -122.0844371128284,37.42177253003091,19 </coordinates> </LinearRing> </outerBoundaryIs> </Polygon> </Placemark> </Folder> <Folder> <name>Extruded Polygon</name> <description>A simple way to model a building</description> <Placemark> <name>The Pentagon</name> <LookAt> <longitude>-77.05580139178142</longitude> <latitude>38.870832443487</latitude> <heading>59.88865561738225</heading> <tilt>48.09646074797388</tilt> <range>742.0552506670548</range> </LookAt> <Polygon> <extrude>1</extrude> <altitudeMode>relativeToGround</altitudeMode> <outerBoundaryIs> <LinearRing> <coordinates> -77.05788457660967,38.87253259892824,100 -77.05465973756702,38.87291016281703,100 -77.05315536854791,38.87053267794386,100 -77.05552622493516,38.868757801256,100 -77.05844056290393,38.86996206506943,100 -77.05788457660967,38.87253259892824,100 </coordinates> </LinearRing> </outerBoundaryIs> <innerBoundaryIs> <LinearRing> <coordinates> -77.05668055019126,38.87154239798456,100 -77.05542625960818,38.87167890344077,100 -77.05485125901024,38.87076535397792,100 -77.05577677433152,38.87008686581446,100 -77.05691162017543,38.87054446963351,100 -77.05668055019126,38.87154239798456,100 </coordinates> </LinearRing> </innerBoundaryIs> </Polygon> </Placemark> </Folder> <Folder> <name>Absolute and Relative</name> <visibility>0</visibility> <description>Four structures whose roofs meet exactly. Turn on/off terrain to see the difference between relative and absolute positioning.</description> <LookAt> <longitude>-112.3348969157552</longitude> <latitude>36.14845533214919</latitude> <altitude>0</altitude> <heading>-86.91235037566909</heading> <tilt>49.30695423894192</tilt> <range>990.6761201087104</range> </LookAt> <Placemark> <name>Absolute</name> <visibility>0</visibility> <styleUrl>#transBluePoly</styleUrl> <Polygon> <tessellate>1</tessellate> <altitudeMode>absolute</altitudeMode> <outerBoundaryIs> <LinearRing> <coordinates> -112.3372510731295,36.14888505105317,1784 -112.3356128688403,36.14781540589019,1784 -112.3368169371048,36.14658677734382,1784 -112.3384408457543,36.14762778914076,1784 -112.3372510731295,36.14888505105317,1784 </coordinates> </LinearRing> </outerBoundaryIs> </Polygon> </Placemark> <Placemark> <name>Absolute Extruded</name> <visibility>0</visibility> <styleUrl>#transRedPoly</styleUrl> <Polygon> <extrude>1</extrude> <tessellate>1</tessellate> <altitudeMode>absolute</altitudeMode> <outerBoundaryIs> <LinearRing> <coordinates> -112.3396586818843,36.14637618647505,1784 -112.3380597654315,36.14531751871353,1784 -112.3368254237788,36.14659596244607,1784 -112.3384555043203,36.14762621763982,1784 -112.3396586818843,36.14637618647505,1784 </coordinates> </LinearRing> </outerBoundaryIs> </Polygon> </Placemark> <Placemark> <name>Relative</name> <visibility>0</visibility> <LookAt> <longitude>-112.3350152490417</longitude> <latitude>36.14943123077423</latitude> <altitude>0</altitude> <heading>-118.9214100848499</heading> <tilt>37.92486261093203</tilt> <range>345.5169113679813</range> </LookAt> <styleUrl>#transGreenPoly</styleUrl> <Polygon> <tessellate>1</tessellate> <altitudeMode>relativeToGround</altitudeMode> <outerBoundaryIs> <LinearRing> <coordinates> -112.3349463145932,36.14988705767721,100 -112.3354019540677,36.14941108398372,100 -112.3344428289146,36.14878490381308,100 -112.3331289492913,36.14780840132443,100 -112.3317019516947,36.14680755678357,100 -112.331131440106,36.1474173426228,100 -112.332616324338,36.14845453364654,100 -112.3339876620524,36.14926570522069,100 -112.3349463145932,36.14988705767721,100 </coordinates> </LinearRing> </outerBoundaryIs> </Polygon> </Placemark> <Placemark> <name>Relative Extruded</name> <visibility>0</visibility> <LookAt> <longitude>-112.3351587892382</longitude> <latitude>36.14979247129029</latitude> <altitude>0</altitude> <heading>-55.42811560891606</heading> <tilt>56.10280503739589</tilt> <range>401.0997279712519</range> </LookAt> <styleUrl>#transYellowPoly</styleUrl> <Polygon> <extrude>1</extrude> <tessellate>1</tessellate> <altitudeMode>relativeToGround</altitudeMode> <outerBoundaryIs> <LinearRing> <coordinates> -112.3348783983763,36.1514008468736,100 -112.3372535345629,36.14888517553886,100 -112.3356068927954,36.14781612679284,100 -112.3350034807972,36.14846469024177,100 -112.3358353861232,36.1489624162954,100 -112.3345888301373,36.15026229372507,100 -112.3337937856278,36.14978096026463,100 -112.3331798208424,36.1504472788618,100 -112.3348783983763,36.1514008468736,100 </coordinates> </LinearRing> </outerBoundaryIs> </Polygon> </Placemark> </Folder> </Folder> </Document>
</kml>

The following is the resulting CSV after running the above code snippet (new CSV file: 'my_file.csv'):

36.10677870477137,-112.0814237830345,36.0905099328766,-112.0870267752693
36.10673460007995,-112.080622229595,36.09049598612422,-112.085242575315
36.09447672602546,-112.265654928602,36.09342608838671,-112.2660384528238,36.09251058776881,-112.2668139013453,36.09189827357996,-112.2677826834445,36.0913137941187,-112.2688557510952,36.0903677207521,-112.2694810717219,36.08932171487285,-112.2695268555611,36.08850916060472,-112.2690144567276,36.08753813597956,-112.2681528815339,36.08682685262568,-112.2670588176031,36.08646312301303,-112.2657374587321
36.07954952145647,-112.2550785337791,36.08117083492122,-112.2549277039738,36.08260761307279,-112.2552505069063,36.08395660588506,-112.2564540158376,36.08511401044813,-112.2580238976449,36.08584355239394,-112.2595218489022,36.08612634548589,-112.2608216347552,36.08626019085147,-112.262073428656,36.08621519860091,-112.2633204928495,36.08627897945274,-112.2644963846444,36.08649599090644,-112.2656969554589
36.09886943729116,-112.2532845153347,36.09919570465255,-112.2540466121145,36.09984998366178,-112.254734666947,36.10051310621746,-112.255493345654,36.10108441943419,-112.2563157098468,36.10159722088088,-112.2568033076439,36.10204323542867,-112.257494011321,36.10229131995655,-112.2584106072308,36.10240001286358,-112.2596588987972,36.10213176873407,-112.2610581199487,36.10157011437219,-112.2626285262793
36.09445214722695,-112.2656634181359,36.09520916122063,-112.2652238941097,36.09580763864907,-112.2645079986395,36.09628572284063,-112.2638827428817,36.09679275951239,-112.2635746835406,36.09740038871899,-112.2635711822407,36.09804913435539,-112.2640296531825,36.09880337400301,-112.264327720538,36.09963644790288,-112.2642436562271,36.10055381117246,-112.2639148687042,36.10149062823369,-112.2626894973474
37.42257124044786,-122.0848938459612,37.42211922626856,-122.0849580979198,37.42207183952619,-122.0847469573047,37.42209006729676,-122.0845725380962,37.42215932700895,-122.0845954886723,37.42227278564371,-122.0838521118269,37.42203539112084,-122.083792243335,37.42209006957106,-122.0835076656616,37.42200987395161,-122.0834709464152,37.4221046494946,-122.0831221085748,37.42226503990386,-122.0829247374572,37.42231242843094,-122.0829339169385,37.42225046087618,-122.0833837359737,37.42234159228745,-122.0833607854248,37.42237075460644,-122.0834204551642,37.42251292011001,-122.083659133885,37.42265873093781,-122.0839758438952,37.42265143972521,-122.0842374743331,37.4226514386435,-122.0845036949503,37.42261133916315,-122.0848020460801,37.42256395055121,-122.0847882750515,37.42257124044786,-122.0848938459612
37.42227033155257,-122.0857412771483,37.42231408832346,-122.0858169768481,37.42230337469744,-122.085852582875,37.42225686138789,-122.0858799945639,37.4222311076138,-122.0858860101409,37.42220250173855,-122.0858069157288,37.42214027058678,-122.0858379542653,37.42208690214408,-122.0856732640519,37.42214885429042,-122.0856022926407,37.422128290487,-122.0855902778436,37.42208171967246,-122.0855841672237,37.42210455874995,-122.0854852065741,37.42214267949824,-122.0855067264352,37.42212783846172,-122.0854430712915,37.42251282407603,-122.0850990714904,37.42281815323651,-122.0856769818632,37.42244918858722,-122.0860162273783,37.42229239604253,-122.0857260327004,37.42227033155257,-122.0857412771483
37.42136208886969,-122.0857862287242,37.42136935989481,-122.0857312990603,37.42140934910903,-122.0857312992918,37.42138390166565,-122.0856077073679,37.42137299550869,-122.0855802426516,37.42137299504316,-122.0852186221971,37.42161656508265,-122.0852277765639,37.42160565894403,-122.0852598189347,37.42168200156,-122.0852598185499,37.42170017860346,-122.0852369311478,37.42176197982575,-122.0852643957828,37.42176198013907,-122.0853239032746,37.421852864452,-122.0853559454324,37.42188921823734,-122.0854108752463,37.42189285337048,-122.0854795379357,37.42188921797546,-122.0855436229819,37.42186013499926,-122.0856260178042,37.42186013453605,-122.085937287963,37.42160898590042,-122.0859428718666,37.42157992759144,-122.0859655469861,37.42147115002957,-122.0858640462341,37.42140571326184,-122.0858548911215,37.4214057134039,-122.0858091162768,37.42136208886969,-122.0857862287242
37.42177253003091,-122.0844371128284,37.42191111542896,-122.0845118855746,37.42178755121535,-122.0850470999805,37.42143663023161,-122.0850719913391,37.42137237822116,-122.084916406232,37.42137237801626,-122.0842193868167,37.42147617161496,-122.08421938659,37.4214613409357,-122.0838086419991,37.42131306410796,-122.0837899728564,37.42129328840593,-122.0832796534698,37.42139213944298,-122.0832609819207,37.42137236399876,-122.0829373621737,37.42151569778871,-122.0829062425667,37.42176282576465,-122.0828502269665,37.42176776969635,-122.0829435788635,37.42179248552686,-122.083217411188,37.4217480074456,-122.0835970430103,37.42169364237603,-122.0839455556771,37.42176283815853,-122.0840077894637,37.42174801104392,-122.084113587521,37.42171341292375,-122.0840762473784,37.42167881534569,-122.0841447047739,37.42181720660197,-122.084144704223,37.4218170700446,-122.0842503333074,37.42177253003091,-122.0844371128284
38.87253259892824,-77.05788457660967,38.87291016281703,-77.05465973756702,38.87053267794386,-77.05315536854791,38.868757801256,-77.05552622493516,38.86996206506943,-77.05844056290393,38.87253259892824,-77.05788457660967
38.87154239798456,-77.05668055019126,38.87167890344077,-77.05542625960818,38.87076535397792,-77.05485125901024,38.87008686581446,-77.05577677433152,38.87054446963351,-77.05691162017543,38.87154239798456,-77.05668055019126
36.14888505105317,-112.3372510731295,36.14781540589019,-112.3356128688403,36.14658677734382,-112.3368169371048,36.14762778914076,-112.3384408457543,36.14888505105317,-112.3372510731295
36.14637618647505,-112.3396586818843,36.14531751871353,-112.3380597654315,36.14659596244607,-112.3368254237788,36.14762621763982,-112.3384555043203,36.14637618647505,-112.3396586818843
36.14988705767721,-112.3349463145932,36.14941108398372,-112.3354019540677,36.14878490381308,-112.3344428289146,36.14780840132443,-112.3331289492913,36.14680755678357,-112.3317019516947,36.1474173426228,-112.331131440106,36.14845453364654,-112.332616324338,36.14926570522069,-112.3339876620524,36.14988705767721,-112.3349463145932
36.1514008468736,-112.3348783983763,36.14888517553886,-112.3372535345629,36.14781612679284,-112.3356068927954,36.14846469024177,-112.3350034807972,36.1489624162954,-112.3358353861232,36.15026229372507,-112.3345888301373,36.14978096026463,-112.3337937856278,36.1504472788618,-112.3331798208424,36.1514008468736,-112.3348783983763

What is GeoJSON?

GeoJSON is an RFC standardized data format to encode geographic data structures such as Point, LineString, Polygon, MultiPoint, MultiLineString, and MultiPolygon. GeoJSON is based on the JavaScript Object Notation (JSON).

Example GeoJSON to CSV

Say, you have the following GeoJSON snippet:

{ "type": "FeatureCollection", "features": [ { "type": "Feature", "geometry": {"type": "Point", "coordinates": [-75.343, 39.984]}, "properties": { "name": "Location A", "category": "Store" } }, { "type": "Feature", "geometry": {"type": "Point", "coordinates": [-80.24, 40.12]}, "properties": { "name": "Location B", "category": "House" } }, { "type": "Feature", "geometry": {"type": "Point", "coordinates": [ -77.2, 41.427]}, "properties": { "name": "Location C", "category": "Office" } } ] }

You want to convert it to the following CSV format:

latitude,longitude,altitude,geometry,name,category
39.984,-75.343,,Point,Location A,Store
40.12,-80.24,,Point,Location B,House
41.427,-77.2,,Point,Location C,Office

Python GeoJSON to CSV Conversion

The Python code to convert a GeoJSON to a CSV in Python uses a combination of the json and csv packages.

import json
import csv geo_filename = 'my_file.json'
csv_filename = 'my_file.csv' def feature_to_row(feature, header): l = [] for k in header: l.append(feature['properties'][k]) coords = feature['geometry']['coordinates'] assert(len(coords)==2) l.extend(coords) return l with open(geo_filename, 'r') as geo_file: with open(csv_filename, 'w', newline='') as csv_file: geojson_data = json.load(geo_file) features = geojson_data['features'] csv_writer = csv.writer(csv_file) is_header = True header = [] for feature in features: if is_header: is_header = False header = list(feature['properties'].keys()) header.extend(['px','py']) csv_writer.writerow(header) csv_writer.writerow(feature_to_row(feature, feature['properties'].keys())) 

You can either copy&paste this code and run it in the same folder as your GeoJSON (of course, after renaming the input and output filenames.

Or you can check out this excellent GitHub to get a more “scriptable” variant to be used in the command line. This code is inspired by the GitHub but simplified significantly.

Example input:

{ "type": "FeatureCollection", "features": [ { "type": "Feature", "geometry": {"type": "Point", "coordinates": [-75.343, 39.984]}, "properties": { "name": "Location A", "category": "Store" } }, { "type": "Feature", "geometry": {"type": "Point", "coordinates": [-80.24, 40.12]}, "properties": { "name": "Location B", "category": "House" } }, { "type": "Feature", "geometry": {"type": "Point", "coordinates": [ -77.2, 41.427]}, "properties": { "name": "Location C", "category": "Office" } } ] }

Example output:

GeoJSON to CSV in QGIS

In QGIS, if you have a map like this one (source):

You can convert GEOJSON to CSV right within QGIS by clicking Export, then Save Feature As and select the Comma Separated Value [CSV] selector in the first dropdown menu.

GeoJSON to CSV Online Converter

You can easily convert specific GeoJSON snippets to CSV using the following online converter:

Problem Formulation and Solution Overview

In this article, you’ll learn how to iterate over a 1D, 2D and 3D NumPy array using Python.

When working with the NumPy library, you will encounter situations where you will need to iterate through a 1D, 2D and even a 3D array. This article will show you how to accomplish this task.

---

Question: How would we write code to iterate through a 1D, 2D or 3D NumPy array?

We can accomplish this task by one of the following options:

• Method 1: Use a For loop and np.array()
• Method 2: Use a For loop and np.nditer()
• Method 3: Use a For loop and itertools
• Method 4: Use a While loop and np.size
• Method 5: Use a For loop and np.ndenumerate()
• Method 6: Use a For Loop and range()
• Bonus: CSV to np.array()

---

Preparation

Before moving forward, please ensure the NumPy library is installed. Click here if you require instructions.

Then, add the following code to the top of each script. This snippet will allow the code in this article to run error-free.

import numpy as np 

After importing the NumPy library, this library is referenced by calling the shortcode (np).

---

Method 1: Use a For Loop and np.array()

This method uses a For loop combined with np.array() to iterate through a 1D NumPy array. The first five (5) Atomic Numbers from the Periodic Table are generated and displayed for this example.

atomic_els = np.array(np.arange(1,6)) for el in atomic_els: print(el, end=' ')

Above, calls the np.array() function and passes it np.arange(1,6) with two (2) arguments: a start position of one (1) and stop position of five (5) or (stop-1). The result is an array of integers and saves to atomic_els.

[1 2 3 4 5]

Next, a For loop is instantiated and iterates through each element of the 1D NumPy array atomic_els.

The output is sent to the terminal on one (1) line as an additional argument was passed to the print statement (end=' '). This argument replaces the default newline character with a blank space.

1 2 3 4 5

---

Method 2: Uses a For Loop and nditer()

This method uses a for loop combined with np.nditer() to iterate through a NumPy array. The first five (5) Atomic Numbers and Number of Neutrons data from the Periodic Table display for this example.

atomic_data = np.array([np.arange(1,6), [0, 2, 4, 5, 6]]) for dim in atomic_data: for d in dim: print(d, end=' ')

Above, calls the np.array() function and passes np.arange(1,6) as the first argument and an array of associated Number of Neutrons values as the second.

Next, an outer for loop is instantiated. This iterates through each array dimension.

Inside this loop, another for loop is instantiated. The inside loop iterates through each element of the above dimensions.

The output is sent to the terminal on one (1) line as an additional argument is passed to the print statement (end=' '). This argument replaces the default newline character with a blank space.

1 2 3 4 5 0 2 4 5 6

Note: The first dimension is highlighted to easily differentiate the dimensions.

---

Method 3: Use a For Loop and itertools

This method uses a for loop and Python’s built-in itertools library to iterate through a NumPy array. The first three (3) Atomic Numbers, Phase, and Group data from the Periodic Table display for this example.

import itertools atomic_num = np.arange(1,4)
atomic_phase = ['gas', 'gas', 'solid']
atomic_group = [1, 18, 1] for (a, b, c) in itertools.zip_longest(atomic_num, atomic_phase, atomic_group, fillvalue=0): print (a, b, c)

Above, imports the itertools library to use the zip_longest() function.

Then, three (3) arrays are declared containing the relevant data for the first three (3) elements from the Periodic Table. They save respectively to atomic_num, atomic_phase, and atomic_group.

Next, a for loop is instantiated and passed three (3) arguments inside the brackets (a, b, c) which reference the arguments inside the zip_longest(atomic_num, atomic_phase, atomic_group) function.

An additional argument (fillvalue=0) is also passed. This fills in any missing values for lists of unequal lengths with the stated value.

The loop iterates until the end of the arrays are reached, outputting to the terminal for each iteration.

1 gas 1 2 gas 18 3 solid 1

---

Method 4: Use a While Loop and Size

This method uses a while loop combined with np.size to iterate through a NumPy array. The first five (5) Atomic Numbers, Number of Neutrons, and Phase data from the Periodic Table display for this example.

atomic_names = np.array(['Hydrogen', 'Helium', 'Lithium', 'Beryllium', 'Boron'])
el_count = 0 while el_count < atomic_names.size: print(atomic_names[el_count], end=' ') el_count += 1

Above, calls the np.array() function and passes a list containing the first five (5) element names of the Periodic Table. These results save to atomic_names.

Then, a variable, el_count, is declared with a start value of zero (0). This will be a counter variable and lets the While loop know when to stop iterating.

Next, a while loop is instantiated and iterates through the elements of atomic_names. This will continue while the value of el_count remains less than the value of atomic_names.size.

The output is sent to the terminal on one (1) line as an additional argument is passed to the print statement (end=' '). This argument replaces the default newline character with a blank space.

Hydrogen Helium Lithium Beryllium Boron

---

Method 5: Use a For Loop and np.ndenumerate()

This method uses a For loop and np.ndenumerate() to iterate through a NumPy array. The first five (5) Element Names from the Periodic Table display for this example.

atomic_names = np.array(['Hydrogen', 'Helium', 'Lithium', 'Beryllium', 'Boron']) for idx, x in np.ndenumerate(atomic_names): print(idx, x)

Above, calls the np.array() function and passes it an array containing the first five (5) element names from the Periodic Table. The results save to atomic_names.

Next, a for loop is instantiated referencing idx which is the index of the array, and x, which is the array element’s value.

The output is sent to the terminal. Then idx displays a Tuple containing the index value of the array and then the value of x for each iteration.

(0,) Hydrogen (1,) Helium (2,) Lithium (3,) Beryllium (4,) Boron

---

Method 6: Use a For Loop and range()

This method uses a for loop and range()) to iterate through a 3D NumPy array.

nums = np.array([[[1, 2], [3, 4], [5, 6]], [[7, 8], [9, 10], [11, 12]], [[13, 14], [15, 16], [17, 18]], [[19, 20], [21, 22], [23, 23]], [[24, 25], [26, 27], [28, 29]]]) for x in range(0, 5): for y in range(0, 3): for z in range(0, 2): print(nums[x][y][z])

Above declares a 3D NumPy array containing consecutive numbers from 1-29 inclusive. These save to nums.

Next, three (3) for loops are instantiated to loop through and output the contents of nums to the terminal one (1) number per line.

The range() function for each loop is based on the dimensions of the 3D Numpy array, for example:

• The first loop (range(0,5) identifies the total number of tows in the array.

• The next loop (range(0,3) identifies the number of columns in the array.

• The final loop (range(0,2) identifies the elements in each column in the array.

Finally, the output is sent to the terminal (snippet only).

1 2 3 4 5 6 7 8 9 10 ...

---

Bonus: Convert CSV to np.array()

This example reads in a snippet of the Periodic Table as a CSV file. This data is converted to a NumPy array and is output to the terminal.

The CSV file below contains the values of the Atomic Number, Atomic Mass, Number of Neutrons, and the Number of Electrons for the first seven (7) elements from the Periodic Table.

Contents of pt_sample.csv

1,1.007,0 2,4.002,2 3,6.941,4 4,9.012,5 5,10.811,6 6,12.011,6 7,14.007,7

To follow along, save this file as pt_sample.csv and move it into the current working directory.

Before moving forward, please ensure the Pandas library is installed. Click here if you require instructions.

import pandas as pd
import csv file_name = 'pt_sample.csv'
df_data = np.array(list(csv.reader(open(file_name, 'r'), delimiter=','))).astype("float")
print(df_data)

Above, imports the Pandas library and the CSV library. This is needed to work with DataFrames and read in the CSV file.

Then, a filename is declared and saves to file_name.

On the highlighted line, the np.array() function is called and passed the following arguments.

• The csv.reader() function is passed as an argument and this argument passes the open() method to open the specified CSV file in read (r) mode, including the field delimiter (csv.reader(open(file_name, 'r'), delimiter=',')).
• The contents of the CSV file converts to a list.
• This list converts to an np.array().

The results save to df_data and output to the terminal.

[[ 1. 1.007 0. ] [ 2. 4.002 2. ] [ 3. 6.941 4. ] [ 4. 9.012 5. ] [ 5. 10.811 6. ] [ 6. 12.011 6. ] [ 7. 14.007 7. ]]

---

Summary

These seven (7) methods of iterating a NumPy array should give you enough information to select the best one for your coding requirements.

Good Luck & Happy Coding!

---

Programmer Humor – Blockchain

Python Append Row to CSV

To append a row (=dictionary) to an existing CSV, open the file object in append mode using open('my_file.csv', 'a', newline=''). Then create a csv.DictWriter() to append a dict row using DictWriter.writerow(my_dict).

Given the following file 'my_file.csv':

You can append a row (dict) to the CSV file via this code snippet:

import csv # Create the dictionary (=row)
row = {'A':'Y1', 'B':'Y2', 'C':'Y3'} # Open the CSV file in "append" mode
with open('my_file.csv', 'a', newline='') as f: # Create a dictionary writer with the dict keys as column fieldnames writer = csv.DictWriter(f, fieldnames=row.keys()) # Append single row to CSV writer.writerow(row)

After running the code in the same folder as your original 'my_file.csv', you’ll see the following result:

Append Multiple Rows to CSV

Given the following CSV file:

To add multiple rows (i.e., dicts) to an old existing CSV file, iterate over the rows and write each row by calling csv.DictWriter.writerow(row) on the initially created DictWriter object.

Here’s an example (major changes highlighted):

import csv # Create the dictionary (=row)
rows = [{'A':'Z1', 'B':'Z2', 'C':'Z3'}, {'A':'ZZ1', 'B':'ZZ2', 'C':'ZZ3'}, {'A':'ZZZ1', 'B':'ZZZ2', 'C':'ZZZ3'}] # Open the CSV file in "append" mode
with open('my_file.csv', 'a', newline='') as f: # Create a dictionary writer with the dict keys as column fieldnames writer = csv.DictWriter(f, fieldnames=rows[0].keys()) # Append multiple rows to CSV for row in rows: writer.writerow(row)

The resulting CSV file has all three rows added to the first row:

Python Add Row to CSV Pandas

To add a row to an existing CSV using Pandas, you can set the write mode argument to append 'a' in the pandas DataFrame to_csv() method like so: df.to_csv('my_csv.csv', mode='a', header=False).

df.to_csv('my_csv.csv', mode='a', header=False)

For a full example, check out this code snippet:

import pandas as pd # Create the initial CSV data
rows = [{'A':'Z1', 'B':'Z2', 'C':'Z3'}, {'A':'ZZ1', 'B':'ZZ2', 'C':'ZZ3'}, {'A':'ZZZ1', 'B':'ZZZ2', 'C':'ZZZ3'}] # Create a DataFrame and write to CSV
df = pd.DataFrame(rows)
df.to_csv('my_file.csv', header=False, index=False) # Create another row and append row (as df) to existing CSV
row = [{'A':'X1', 'B':'X2', 'C':'X3'}]
df = pd.DataFrame(row)
df.to_csv('my_file.csv', mode='a', header=False, index=False)

The output file looks like this (new row highlighted):

Alternatively, you can open the file in append mode using normal open() function with the append 'a' argument and pass it into the pandas DataFrame to_csv() method.

Here’s an example snippet for copy&paste:

with open('my_csv.csv', 'a') as f: df.to_csv(f, header=False)

Learn More: If you want to learn about 7 Best Ways to Convert Dict to CSV in Python, check out this Finxter tutorial.

Question: How to convert a dictionary to a CSV in Python?

In Python, convert a dictionary to a CSV file using the DictWriter() method from the csv module. The csv.DictWriter() method allows you to insert a dictionary-formatted row (keys=column names; values=row elements) into the CSV file using its DictWriter.writerow() method.

Learn More: If you want to learn about converting a list of dictionaries to a CSV, check out this Finxter tutorial.

Method 1: Python Dict to CSV in Pandas

First, convert a list of dictionaries to a Pandas DataFrame that provides you with powerful capabilities such as the

Second, convert the Pandas DataFrame to a CSV file using the DataFrame’s to_csv() method with the filename as argument.

salary = [{'Name':'Alice', 'Job':'Data Scientist', 'Salary':122000}, {'Name':'Bob', 'Job':'Engineer', 'Salary':77000}, {'Name':'Carl', 'Job':'Manager', 'Salary':119000}] # Method 1
import pandas as pd
df = pd.DataFrame(salary)
df.to_csv('my_file.csv', index=False, header=True)

Output:

Name,Job,Salary
Alice,Data Scientist,122000
Bob,Engineer,77000
Carl,Manager,119000

You create a Pandas DataFrame—which is Python’s default representation of tabular data. Think of it as an Excel spreadsheet within your code (with rows and columns).

The DataFrame is a very powerful data structure that allows you to perform various methods. One of those is the to_csv() method that allows you to write its contents into a CSV file.

• You set the index argument of the to_csv() method to False because Pandas, per default, adds integer row and column indices 0, 1, 2, …. Again, think of them as the row and column indices in your Excel spreadsheet. You don’t want them to appear in the CSV file so you set the arguments to False.
• You set the and header argument to True because you want the dict keys to be used as headers of the CSV.

If you want to customize the CSV output, you’ve got a lot of special arguments to play with. Check out this Finxter Tutorial for a comprehensive list of all arguments.

Related article: Pandas Cheat Sheets to Pin to Your Wall

Method 2: Dict to CSV File

In Python, convert a dictionary to a CSV file using the DictWriter() method from the csv module. The csv.DictWriter() method allows you to insert data into the CSV file using its DictWriter.writerow() method.

The following example writes the dictionary to a CSV using the keys as column names and the values as row values.

import csv data = {'A':'X1', 'B':'X2', 'C':'X3'} with open('my_file.csv', 'w', newline='') as f: writer = csv.DictWriter(f, fieldnames=data.keys()) writer.writeheader() writer.writerow(data)

The resulting file 'my_file.csv' looks like this:

The csv library may not yet installed on your machine. To check if it is installed, follow these instructions. If it is not installed, fix it by running pip install csv in your shell or terminal.

Method 3: Dict to CSV String (in Memory)

To convert a list of dicts to a CSV string in memory, i.e., returning a CSV string instead of writing in a CSV file, use the pandas.DataFrame.to_csv() function without file path argument. The return value is a CSV string representation of the dictionary.

Here’s an example:

import pandas as pd data = [{'A':'X1', 'B':'X2', 'C':'X3'}, {'A':'Y1', 'B':'Y2', 'C':'Y3'}] df = pd.DataFrame(data)
my_csv_string = df.to_csv(index=False) print(my_csv_string) '''
A,B,C
X1,X2,X3
Y1,Y2,Y3 '''

We pass index=False because we don’t want an index 0, 1, 2 in front of each row.

Method 4: Dict to CSV Append Line

To append a dictionary to an existing CSV, you can open the file object in append mode using open('my_file.csv', 'a', newline='') and using the csv.DictWriter() to append a dict row using DictWriter.writerow(my_dict).

Given the following file 'my_file.csv':

You can append a row (dict) to the CSV file via this code snippet:

import csv row = {'A':'Y1', 'B':'Y2', 'C':'Y3'} with open('my_file.csv', 'a', newline='') as f: writer = csv.DictWriter(f, fieldnames=row.keys()) writer.writerow(row)

After running the code in the same folder as your original 'my_file.csv', you’ll see the following result:

Method 5: Dict to CSV Columns

To write a Python dictionary in a CSV file as a column, i.e., a single (key, value) pair per row, use the following three steps:

• Open the file in writing mode and using the newline='' argument to prevent blank lines.
• Create a CSV writer object.
• Iterate over the (key, value) pairs of the dictionary using the dict.items() method.
• Write one (key, value) tuple at a time by passing it in the writer.writerow() method.

Here’s the code example:

import csv data = {'A':42, 'B':41, 'C':40} with open('my_file.csv', 'w', newline='') as f: writer = csv.writer(f) for row in data.items(): writer.writerow(row)

Your output CSV file (column dict) looks like this:

Method 6: Dict to CSV with Header

Convert a Python dictionary to a CSV file with header using the csv.DictWriter(fileobject, fieldnames) method to create a writer object used for writing the header via writer.writeheader() without argument. This writes the list of column names passed as fieldnames, e.g., the dictionary keys obtained via dict.keys().

To write the rows, you can then call the DictWriter.writerow() method.

The following example writes the dictionary to a CSV using the keys as column names and the values as row values.

import csv data = {'A':'X1', 'B':'X2', 'C':'X3'} with open('my_file.csv', 'w', newline='') as f: writer = csv.DictWriter(f, fieldnames=data.keys()) writer.writeheader() writer.writerow(data)

The resulting file 'my_file.csv' looks like this:

Where to Go From Here

If you haven’t found your solution, yet, you may want to check out my in-depth guide on how to write a list of dicts to a CSV:

Guide: Write List of Dicts to CSV in Python

Bonus Method 7: Vanilla Python

If you don’t want to import any library and still convert a list of dicts into a CSV file, you can use standard Python implementation as well: it’s not complicated and very efficient.

Finxter Recommended: Join the Finxter community and download your 8+ Python cheat sheets to refresh your code understanding.

This method is best if you won’t or cannot use external dependencies.

• Open the file f in writing mode using the standard open() function.
• Write the first dictionary’s keys in the file using the one-liner expression f.write(','.join(salary[0].keys())).
• Iterate over the list of dicts and write the values in the CSV using the expression f.write(','.join(str(x) for x in row.values())).

Here’s the concrete code example:

salary = [{'Name':'Alice', 'Job':'Data Scientist', 'Salary':122000}, {'Name':'Bob', 'Job':'Engineer', 'Salary':77000}, {'Name':'Carl', 'Job':'Manager', 'Salary':119000}] # Method 3
with open('my_file.csv','w') as f: f.write(','.join(salary[0].keys())) f.write('n') for row in salary: f.write(','.join(str(x) for x in row.values())) f.write('n')

Output:

Name,Job,Salary
Alice,Data Scientist,122000
Bob,Engineer,77000
Carl,Manager,119000

In the code, you first open the file object f. Then you iterate over each row and each element in the row and write the element to the file—one by one. After each element, you place the comma to generate the CSV file format. After each row, you place the newline character 'n'.

Note: to get rid of the trailing comma, you can check if the element x is the last element in the row within the loop body and skip writing the comma if it is.

Related Tutorial: How to Convert a List to a CSV File in Python [5 Ways]

Programming Humor – Python

Where to Go From Here?

Enough theory. Let’s get some practice!

Coders get paid six figures and more because they can solve problems more effectively using machine intelligence and automation.

To become more successful in coding, solve more real problems for real people. That’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

You build high-value coding skills by working on practical coding projects!

Do you want to stop learning with toy projects and focus on practical code projects that earn you money and solve real problems for people?

If your answer is YES!, consider becoming a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

If you just want to learn about the freelancing opportunity, feel free to watch my free webinar “How to Build Your High-Income Skill Python” and learn how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

Artificial neural networks have become a powerful tool providing many benefits in our modern world. They are used to filter out spam, to perform voice recognition, and are even being developed to drive cars, among many other things.

As remarkable as these tools are, they are readily within the grasp of almost anyone. If you have technical interest and have some experience with computer programming you can build your own neural networks.

But before you learn the hands-on details of building neural networks you should learn some of the fundamentals of how they work. This article will cover one of those fundamentals – how neural networks learn.

Note: This article includes some algebra and calculus. If you’re not comfortable with algebra, you should still be able to understand the content from the graphs and descriptions. The calculus is not done in any detail. Again you should still be able to follow along from the descriptions. You will not learn the details of how the calculations are done. Instead, you will gain an intuitive understanding of what is going on.

Before learning this, you should be familiar with the basics of how neural networks are structured and how they operate. The article “The Magic of Neural Networks: History and Concepts” covers these basics. Still, we offer the following brief refresher.

Basic Fundamentals: How Neural Networks Work

Figure 1 shows an artificial neuron.

Signals from other neurons come in through multiple inputs, each multiplied by its corresponding weight (Weights express the connection strengths between the neuron and each of its upstream neurons.).

A bias is input as well (bias expresses a neuron’s inherent activation, independent of its input from other neurons.). All these inputs add together, and the resulting total signal is then processed through the activation function (A sigmoid function is shown here.).

Figure 2 shows a network of these neurons. Signals are introduced on the input side, and they progress through the network, passing through neurons and along their connections, getting processed by the calculations described above. How the signals are processed, depends on the weights and biases among all the neurons.

The key takeaway is that it is the settings of the weights and biases that establish how the network as a whole computes. In other words, the learning and memory of the network is encoded by the weights and biases.

So how does one program these weights and biases?

They are set by training the network with samples and letting it learn by example. The details of how that is done is the subject of this article.

Overview of How Neural Networks Learn

As mentioned, a neural network’s learning and memory is encoded by the connection weights and biases of the neurons throughout the network.

These weights and biases are set by training the network on examples by following this six-step training procedure:

1. Provide a sample to the network.
2. Since the network is untrained, it will probably get the wrong answer.
3. Compute how far this answer is from the correct answer. This error is known as loss.
4. Calculate what changes in the weights and biases will make the loss smaller.
5. Make adjustments to those weights and biases as determined by those calculations.
6. Repeat this again and again with numerous samples until the network learns to answer the samples correctly.

Presenting Samples and Calculating Loss

Let’s review some of this in more detail while considering a use case.

Imagine we want to train a network to estimate crowd size.

To do this we must first train the network with a large set of images of crowds. For each image the number of people are counted. We then include labels indicating correct crowd size for each picture. This is known as a training set.

The pictures are submitted to the network, which then indicates its crowd estimate for each picture. Since the network is not trained, it surely gets the estimate wrong for each image.

For each image/label pair, the network calculates the loss for that sample.

Multiple possible choices can be used for calculating loss. One can choose any calculation that appropriately expresses how far the network’s answer is from the correct answer.

An appropriate choice for crowd-size loss estimate is the square error:

where:

Suppose we submit an image showing a crowd size of 500 people. Figure 3 shows how the error varies for crowd estimates around the true crowd size of 500 people.

If the Network guesses 350 people the loss is 22500. If the network guesses 600 people the loss is 10000.

Clearly, the loss is minimized when the network guesses the correct crowd size of 500 people.

But recall we said it is the weights and biases in the network that encode its learning and memory, so it is the weights and biases that determine if the network gets the right answer. So we need to adjust the weights and biases so that the network gets closer to the correct answer for this image.

In other words, we need to change the weights and biases to minimize the loss. To do that, we need to figure out how the loss varies when we vary the weights and biases.

Minimizing Loss: Calculus and the Derivative

So how do we calculate how loss changes when we vary weights and biases?

This is where calculus comes in.

(Don’t worry if you don’t know calculus, we’ll show you everything you need to know, and we’ll keep it intuitive.)

Calculus is all about determining how one variable is affected by changes in another variable.

(Strictly speaking there’s more to calculus than that, but this idea is one of the core ideas of calculus.)

The loss L depends on network output y, but y depends on input, and on weights w and biases b. So there is a somewhat long and complicated chain of dependencies we have to go through to figure out how L varies when w and b vary.

However, for the sake of learning, let’s instead start by just examing how L varies when y varies, since this is simpler and will help develop an intuition for calculus.

How L depends on y is somewhat easy – we saw the equation for it earlier, and we saw the graph of that equation in Figure 3. We can tell by looking at the graph that if the network guesses 350 then we need to increase the output y in order to reduce the loss, and that if the network guesses 600 then we need to decrease the output y in order to reduce the loss.

But with neural networks, we never have the luxury of being able to examine the graph of the loss to figure it out.

We can, however, use calculus to get our answer. To do this, we do what is called taking the derivative.

Here is the derivative of the equation for the graph in Figure 3 (note, we will not explain how this is calculated, that is the domain of a calculus course.):

This is typically referred to as “taking the derivative of L with respect to y”. You can read that dL/dy as saying “this is how L changes when y changes”. Now let’s calculate how L changes when y changes at the point y = 350:

So at y = 350, for every bit y increases, L decreases by 300. That implies that when we increase y the loss will decrease.

Now let’s calculate how L changes when y changes at the point y = 600:

So at y = 600, for every bit y increases, L increases by 200. Since we want to decrease L, that means we need to decrease y.

These calculations match what we concluded from looking at the graph.

You can also read dL/dy as saying “this is the slope of the graph”.

This makes sense: at point y = 350 the slope of the graph is -300 (sloping down steeply), while at point y = 600 the slope of the graph is 200 (sloping up, not quite so steeply).

So by using calculus and taking the derivative, we can figure out which way to change y to reduce the loss L, even when we can’t see the graph to figure it out.

Recall, however, that we want to figure out how to change the weights and biases to reduce the loss L. Also recall there is a chain of dependencies, of L depending on y, which itself depends on w and b (for several layers worth of w and b!), and on input.

So a full description could result in some rather complicated equations and some difficult derivatives. For those curious about the math details, the method for figuring out derivatives when there is such dependencies is called the chain rule.

Fortunately, with modern neural network software, the computer takes care of calculating derivatives and keeping track of and resolving the chains of dependencies in the derivatives. Just understand that, even if we can’t see its graph:

• there is some relationship between the loss L and the weights w and biases b (a “graph”)
• there is some set of weights and biases where the loss L is at a minimum for a given input
• we can use calculus to figure out how to adjust the weights and biases to minimize loss

The Loss Surface and Gradient Descent

Let’s consider a very simple case where there are just two weights, w1 and w2, and no biases. The graph of L as a function of w1 and w2 might look like figure 4.

In this example, with two independent weights, we end up with a bowl-shaped surface for the loss graph. In this case, the loss is minimized when w1 = 4 and w2 = 3. In the beginning, when the network is not yet trained the weights (initially set to small random numbers) are almost certainly not at the correct values for the loss to be at a minimum.

We still figure out which direction to change the weights to reduce the loss by taking the derivative.

Only this time, since there are two independent variables, we take the derivative with respect to each independently.

Important: The result is, for any given point on the loss surface, a direction (a vector, or an arrow) pointing in which direction the loss increases the fastest (“uphill”). This is known as the gradient (instead of derivative). Since we want to reduce loss, we move in the opposite direction, the negative of the gradient.

The larger point is we are still using calculus to figure out which direction to change weights to reduce loss. Repeatedly doing this moves the weights closer to the values which make the network give the correct answer for a given input. This is known as gradient descent.

However, most neural networks have many more than two weights, typically dozens for any given layer.

But the same ideas still apply: if we have a layer consisting of 16 weighted connections, the loss is a 16-dimensional surface! You can’t visualize it but it still exists mathematically, and the same principles apply!

You can still calculate the gradient, that is the derivative with respect to all 16 w’s, and figure out which direction to change the w’s to minimize the loss.

So how much do we adjust the weights and biases?

Typically they are adjusted just a small amount. This is because large adjustments can cause problems.

Refer to the loss surface shown in Figure 4. If too large a step is made, you could jump right across the loss surface bowl, even going so far as to make the loss worse!

The adjustment step size is known as the learning rate. Figuring out the best learning rate is one of the tricks to optimizing your network that a neural network engineer has to work out.

Backpropagation

Ultimately all of the weights and biases throughout the network have to be adjusted to minimize loss. This is done back from the loss, working back layer by layer to the beginning of the network, a process called backpropagation.

It has to be done this way because you can’t figure out how the first layer’s weights and biases affect loss until you know how the second layer’s weights and biases affect loss; you can’t tell how the second layer’s weights and biases effect loss until you know how the third layer’s weights and biases effect loss, and so on.

So calculations and adjustments are done starting with the last layer, then working back to the second to the last layer, and so on back to the first layer.

So that’s the core algorithm of training a neural network:

1. Present example image.
2. Calculate the loss.
3. Adjust the network weights and biases through backpropagation, calculating gradient descent, and making adjustments layer by layer.

Batch Size

However, recall that the objective of the training is to adjust the weights and biases for all of the images, not just one.

So how does one train the network, one image at a time, or using the entire set of all training images? Either choice is a possibility.

Ultimately the loss we want to minimize is the loss for the entire set of training samples, so a natural choice might be to run all samples through the network before making adjustments to the weights and biases. This is known as batch processing.

However performing so many calculations before making adjustments can be very demanding on computer resources and can slow the training process down.

How about adjusting weights and biases for each individual training sample? Optimum weights and biases will be different for each training sample, and this variation can introduce large randomness into the gradient descent. This is known as stochastic gradient descent.

To better understand the importance of this refer to the hypothetical loss curve in figure 5:

Notice that there is more than one minimum: there is a local minimum at point B, which is not quite the lowest loss, and a global minimum at point A that is truly the minimum where the loss is lowest.

It is truly possible (even likely) to get loss curves like this, with multiple local minima, and it’s also possible for the network to get stuck in one of these local minima.

The randomness of single sample training can help knock the network out of a local minimum if it gets stuck in one, so there is some benefit to stochastic gradient descent.

However, the randomness can be so extreme that it can actually knock the network out of the true global minimum if it happens to reach it before a training cycle ends. This can slow the training as the network has to work back down to minimize the loss again.

So in practice, it turns out the best approach is to use minibatches. These are batch sizes of perhaps a few hundred samples that are run through the network, and then adjustments are made.

The network runs through mini batch after many batch until the entire set of training samples has been processed. This has enough randomness to it that it has the same benefit as stochastic gradient descent of pushing the network out of local minima, but not so much randomness that the loss can get worse.

Running through the entire set of training samples once is called an epoch.

Typically networks must run through many epochs to become fully trained. Also the ordering and grouping of training samples within and between batches is randomized from epoch to epoch. This is to avoid overfitting.

Overfitting is when the network performs successfully on the training samples, but fails on samples it has not seen before. This is like a person memorizing a set of samples, rather than generalizing characteritics from those samples so that it can be successful on new samples.

After training the network is then tested on a test set. This is a set of samples the network has not seen before. This allows one to assess how well the trained network performs. It checks to see how effective the network is on unknown samples, and checks to make sure overfitting has not occurred.

How Neural Networks Learn

So that is the full process of how neural networks learn:

1. Train the network by presenting it minibatches of samples from the training set.
2. The training algorithm calculates the loss for the minibatch.
3. The algorithm calculates the gradient of the loss.
4. The network adjusts weights and biases according to the gradient calculations, through the process of backpropagation and gradient descent.
5. Running this sequence through all training samples is called an epoch.
6. This is then repeated for multiple epochs, until the network is successfully trained on the training set.
7. Finally the network is tested on a test set to make sure it works successfully and does not suffer from overfitting.

We hope you have found this lesson on how neural networks learn informative.

We wish you happy coding!