Don’t Use This Popular Python Function — It’s Slower Than You Think

Think twice before using it — your code might be crawling without you knowing.

Don’t Use This Popular Python Function — It’s Slower Than You Think

The Convenience Trap in Python

We’ve all done it.
You’re writing Python code, want something quick, and instinctively reach for that familiar, built-in function everyone seems to use. It works. It’s clean. It even makes you feel like a more “Pythonic” developer.

But there’s a catch — it’s not always the fastest option. In fact, for large datasets or performance-critical parts of your code, this function can become a hidden bottleneck.

The culprit? in checks on lists.

Yes, the humble if item in my_list: pattern — a darling of beginner tutorials and Stack Overflow answers — is far slower than you might expect when the list grows.

Let’s unpack why, and what to do instead.

Why in on Lists Is Slower Than You Think

When you check if an element exists in a Python list, Python has to scan through each item until it finds a match (or reaches the end).

That means:

Time complexity: O(n) — the longer your list, the longer it takes.

Worst-case scenario: The element is at the end or doesn’t exist, forcing Python to check every single item.

For small lists, you won’t notice the difference. But once you start working with thousands, hundreds of thousands, or millions of elements, it adds up.

Example:

# Checking membership in a list 
names = ["Alice", "Bob", "Charlie", "David"] * 1_000_000  # 4M items 
if "Zara" in names:  # O(n) check 
    print("Found!")

On my machine, this simple check can take hundreds of milliseconds — a lifetime in code that runs thousands of times.

The Faster Alternative: Sets and Dictionaries

If you care about lookup speed, use a set or dictionary instead.

Why?
Both use hash tables under the hood, giving you O(1) average lookup time — that’s constant time, regardless of collection size.

Example:

# Using a set for faster lookups 
names_set = {"Alice", "Bob", "Charlie", "David"} 
if "Zara" in names_set:  # O(1) check 
    print("Found!")

Even with millions of elements, set lookups are blazingly fast.

Real-World Impact

Imagine you’re processing a massive CSV file with millions of rows, and for each row, you check whether a certain value exists in a list of “blocked” IDs.

With a list:

blocked_ids = [123, 456, 789, ...]  # millions of entries 
for row in rows: 
    if row.id in blocked_ids:  # O(n) each time 
        ...

With a set:

blocked_ids = {123, 456, 789, ...}  # millions of entries 
for row in rows: 
    if row.id in blocked_ids:  # O(1) each time 
        ...

Switching from list to set here can reduce execution time from minutes to seconds.

Benchmark: List vs Set Lookup

Let’s put numbers to it:

import time 
 
N = 5_000_000 
target = N - 1 
 
# Setup 
nums_list = list(range(N)) 
nums_set = set(range(N)) 
 
# List lookup 
start = time.perf_counter() 
target in nums_list 
end = time.perf_counter() 
print(f"List lookup: {end - start:.6f} seconds") 
 
# Set lookup 
start = time.perf_counter() 
target in nums_set 
end = time.perf_counter() 
print(f"Set lookup: {end - start:.6f} seconds")

Example output on my machine:

List lookup: 0.402381 seconds 
Set lookup: 0.000002 seconds

That’s 200,000× faster for the set lookup in this example.

“But Lists Are More Flexible!”

True — lists maintain order and allow duplicates, which sets do not. But in most membership-check scenarios, you don’t care about order or duplicates — you care about speed.

When to use a list for in:

Very small datasets (tens or maybe hundreds of elements).

When you need order and will be iterating through it anyway.

When to use a set or dict for in:

Large datasets.

Frequent membership checks.

Performance-critical loops.

The Hybrid Approach

Sometimes you need both — the order of a list and the fast lookups of a set.

Solution: keep both structures in sync.

items_list = [] 
items_set = set() 
 
def add_item(item): 
    if item not in items_set: 
        items_list.append(item) 
        items_set.add(item)

This way, you can quickly check membership with the set and still preserve insertion order in the list.

Other Hidden “Slow by Default” Functions

in on lists is just one example. Other built-in patterns that can be slow if misused:

str concatenation in loops — use join() instead.

sorted() without a key function when comparing complex objects.

filter() and map() with large data when list comprehensions might be faster (due to better optimization in CPython).

The key takeaway: Know the complexity of the tools you use.

Final Thoughts

Python makes it easy to write clear, readable code — but that doesn’t mean all default patterns are fast.

The in operator on lists is a silent performance killer in large datasets.
If speed matters, switch to a set or dictionary for membership checks.

Your future self — and your users — will thank you.

Write Python like you pay for every millisecond — because in many cases, you do.