Login

Check for NaN Values in Python

<div><h2>Overview</h2>
Problem: How to check if a given value is <code>NaN</code>?
Here’s a quick look at the solutions to follow:
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import math
import numpy as np
import pandas as pd x = float('nan')
print(math.isnan(x))
print(x != x)
print(np.isnan(x))
print(pd.isna(x))
print(not(float('-inf') < x < float('inf')))</pre>
So, what is a <code>NaN</code> value?
<code>NaN</code> is a constant value that indicates that the given value is Not a Number. It’s a floating-point value, hence cannot be converted to any other type other than float. We should know that <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code> and Null are two different things in Python. The Null values indicate something which does not exist, i.e. is empty. But that is not the case with <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code>.
We have to deal with <code>NaN</code> values frequently in Python especially when we deal with array objects or DataFrames. So, without further delay, let us dive into our mission critical question and have a look at the different methods to solve our problem.
<h2>Method 1: Using math.isnan()</h2>
The simplest solution to check for NaN values in Python is to use the mathematical function <code>math.isnan()</code>.
<code>math.isnan()</code> is a function of the math module in Python that checks for <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code> constants in float objects and returns True for every NaN value encountered and returns False otherwise. 
Example: 
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Importing the math module
import math # Function to check for NaN values
def isNaN(a): # Using math.isnan() if math.isnan(a): print("NaN value encountered!") else: print("Type of Given Value: ", type(a)) # NaN value
x = float('NaN')
isNaN(x)
# Floating value
y = float("5.78")
isNaN(y)</pre>
Output:
<pre class="wp-block-code"><code>NaN value encountered!
Type of Given Value: <class 'float'></code></pre>
In the above example, since <code>x</code> represents a NaN value, hence, the <code>isNaN</code> method returns <code>True</code> but in case of <code>y</code> , <code>isNan</code> returns <code>False</code> and prints the type of the variable <code>y</code> as an output. 
<h2>Method 2: Hack NaN Using != Operator</h2>
The most unique thing about <code>NaN</code> values is that they are constantly shapeshifting. This means we cannot compare the <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code> value even against itself. Hence, we can use the <code>!=</code> (not equal to) operator to check for the <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code> values. Thus, the idea is to check if the given variable is equal to itself. If we consider any object other than <code>NaN</code>, the expression <code>(x == x)</code> will always return <code data-enlighter-language="generic" class="EnlighterJSRAW">True</code>. If it’s not equal, then it is a <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code> value. 
Example 1:
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">print(5 == 5)
# True
print(['a', 'b'] == ['a', 'b'])
# True
print([] == [])
# True
print(float("nan") == float("nan"))
# False
print(float("nan") != float("nan"))
# True</pre>
Example 2:
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Function to check for NaN values
def not_a_number(x): # Using != operator if x != x: print("Not a Number!") else: print(f'Type of {x} is {type(x)}') # Floating value
x = float("7.8")
not_a_number(x)
# NaN value
y = float("NaN")
not_a_number(y)</pre>
Output:
<pre class="wp-block-code"><code>Type of 7.8 is <class 'float'>
Not a Number!</code></pre>
<h2>Method 3: Using <a href="https://blog.finxter.com/numpy-tutorial/" target="_blank" rel="noreferrer noopener">numpy</a>.isnan()</h2>
We can also use the <code>NumPy</code> library to check whether the given value is <code>NaN</code> or not. We just need to ensure that we import the library at the start of the program and then use its <code>np.isnan(x)</code> method. 
The <code>np.isnan(number)</code> function checks whether the element in a Numpy array is <code>NaN</code> or not. It then returns the result as a boolean array.
Example: In the following example we have a Numpy Array and then we will check the type of each value. We will also check if it is a <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code> value or not.
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import numpy as np arr = np.array([10, 20, np.nan, 40, np.nan])
for x in arr: if np.isnan(x): print("Not a Number!") else: print(x, ":", type(x))</pre>
Output:
<pre class="wp-block-code"><code>10.0 : <class 'numpy.float64'>
20.0 : <class 'numpy.float64'>
Not a Number!
40.0 : <class 'numpy.float64'>
Not a Number!</code></pre>
<img src="https://s.w.org/images/core/emoji/13.1.0/72x72/1f4a1.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" />TRIVIA
Let us try to perform some basic functions on an numpy array that involves <code>NaN</code> values and find out what happens to it.
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import numpy as np arr = np.array([10, 20, np.nan, 40, np.nan])
print(arr.sum())
print(arr.max())</pre>
Output:
<pre class="wp-block-code"><code>nan
nan</code></pre>
Now this can be a problem in many cases. So, do we have a way to eliminate the NaN values from our array object and then perform the mathematical operations upon the array elements? Yes! Numpy facilitates us with methods like <code>np.nansum()</code> and <code>np.nanmax()</code> that help us to calculate the sum and maximum values in the array by ignoring the presence of <code>NaN</code> values in the array.
Example:
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import numpy as np arr = np.array([10, 20, np.nan, 40, np.nan])
print(np.nansum(arr))
print(np.nanmax(arr))</pre>
Output:
<pre class="wp-block-code"><code>70.0
40.0</code></pre>
<h2>Method 4: Using <a href="https://blog.finxter.com/pandas-quickstart/" target="_blank" rel="noreferrer noopener">pandas</a>.isna()</h2>
Another way to solve our problem is to use the <code>isna()</code> method of the Pandas module. <code>pandas.isna()</code> is a function that detects missing values in an array-like object. It returns True if any <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN</code> value is encountered. 
Example 1:
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import pandas as pd x = float("nan")
y = 25.75
print(pd.isna(x))
print(pd.isna(y))</pre>
Output:
<pre class="wp-block-code"><code>True
False</code></pre>
Example 2: In the following example we will have a look at a Pandas DataFrame and detect the presence of NaN values in the DataFrame.
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import pandas as pd df = pd.DataFrame([['Mercury', 'Venus', 'Earth'], ['1', float('nan'), '2']])
print(pd.isna(df))</pre>
Output:
<pre class="wp-block-code"><code> 0 1 2
0 False False False
1 False True False</code></pre>
<h2>Method 5: By Checking The Range</h2>
We can check for the <code>NaN</code> values by using another NaN special property: limited range. The range of all the floating-point values falls within negative infinity to infinity. However, <code>NaN</code> values do not fall within this range. 
Hence, the idea is to check whether a given value lies within the range of <code>-inf</code> and <code>inf</code>. If yes , then it is not a <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN </code>value else it is a <code data-enlighter-language="generic" class="EnlighterJSRAW">NaN </code>value. 
Example:
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">li = [25.87, float('nan')]
for i in li: if float('-inf') < float(i) < float('inf'): print(i) else: print("Not a Number!")</pre>
Output:
<pre class="wp-block-code"><code>25.87
Not a Number!</code></pre>
Recommended read: <a href="https://blog.finxter.com/python-infinity/" target="_blank" rel="noreferrer noopener">Python Infinity</a>
<h2>Conclusion</h2>
In this article, we learned how we can use the various methods and modules (<code>pandas</code>, <code>NumPy</code>, and <code>math</code>) in Python to check for the <code>NaN</code> values. I hope this article was able to answer your queries. Please <a rel="noreferrer noopener" href="https://blog.finxter.com" target="_blank">stay tuned</a> and <a rel="noreferrer noopener" href="https://blog.finxter.com/subscribe/" target="_blank">subscribe</a> for more such articles. 
Authors: SHUBHAM SAYON and RASHI AGARWAL
<hr class="wp-block-separator" />
Do you want to become a NumPy master? Check out our interactive puzzle book <a href="https://amzn.to/39dEykm" target="_blank" rel="noreferrer noopener" title="https://amzn.to/39dEykm">Coffee Break NumPy</a> and boost your data science skills! (Amazon link opens in new tab.)
<div class="wp-block-image">
<figure class="aligncenter size-medium"><a href="https://amzn.to/39dEykm" target="_blank" rel="noopener noreferrer"><img loading="lazy" width="200" height="300" src="https://blog.finxter.com/wp-content/uploads/2019/04/Cover_Coffee_Break_NumPy-200x300.jpg" alt="Coffee Break NumPy" class="wp-image-2766"/></a></figure>
</div>
</div>

https://www.sickgaming.net/blog/2022/03/...in-python/

xSicKxBot