Pandas: Search for String in All Columns of DataFrame

---

You can use the following syntax to search for a particular string in each column of a pandas DataFrame and filter for rows that contain the string in at least one column:

#define filter
mask = np.column_stack([df[col].str.contains(r"my_string", na=False) for col in df])

#filter for rows where any column contains 'my_string'
df.loc[mask.any(axis=1)]

The following example shows how to use this syntax in practice.

Example: Search for String in All Columns of Pandas DataFrame

Suppose we have the following pandas DataFrame that contains information about the first role and second role of various basketball players on a team:

import pandas as pd

#create DataFrame
df = pd.DataFrame({'player': ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H'],
                   'first_role': ['P Guard', 'P Guard', 'S Guard', 'S Forward',
                                  'P Forward', 'Center', 'Center', 'Center'],
                   'second_role': ['S Guard', 'S Guard', 'Forward', 'S Guard',
                                   'S Guard', 'S Forward', 'P Forward', 'P Forward']})

#view DataFrame
print(df)

  player first_role second_role
0      A    P Guard     S Guard
1      B    P Guard     S Guard
2      C    S Guard     Forward
3      D  S Forward     S Guard
4      E  P Forward     S Guard
5      F     Center   S Forward
6      G     Center   P Forward
7      H     Center   P Forward

The following code shows how to filter the pandas DataFrame for rows where the string “Guard” occurs in any column:

import numpy as np

#define filter
mask = np.column_stack([df[col].str.contains(r"Guard", na=False) for col in df])

#filter for rows where any column contains 'Guard'
df.loc[mask.any(axis=1)]

        player	first_role  second_role
0	A	P Guard	    S Guard
1	B	P Guard	    S Guard
2	C	S Guard	    Forward
3	D	S Forward   S Guard
4	E	P Forward   S Guard

Notice that each row in the resulting DataFrame contains the string “Guard” in at least one column.

You could also filter for rows where one of several strings occurs in at least one column by using the “OR” ( | ) operator in pandas.

For example, the following code shows how to filter for rows where either “P Guard” or “Center” occurs in at least one column:

import numpy as np

#define filter
mask = np.column_stack([df[col].str.contains(r"P Guard|Center", na=False) for col in df])

#filter for rows where any column contains 'P Guard' or 'Center'
df.loc[mask.any(axis=1)]

        player	first_role  second_role
0	A	P Guard	    S Guard
1	B	P Guard	    S Guard
5	F	Center	    S Forward
6	G	Center	    P Forward
7	H	Center	    P Forward

Notice that each row in the resulting DataFrame contains “P Guard” or “Center” in at least one column.

Note: It’s important to include the argument na=False within the contains() function or else you will encounter an error if NaN values are present in the DataFrame.

Additional Resources

The following tutorials explain how to perform other common filtering operations in pandas:

How to Filter a Pandas DataFrame by Column Values
How to Filter Pandas DataFrame Rows by Date
How to Filter a Pandas DataFrame on Multiple Conditions