pip - pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 200000

 

pip install apache-airflow-providers-apache-spark

Got below error when ran the above command:

pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 200000

 

That error is a pip dependency resolver loop, not a Spark or Airflow bug.

It usually happens when:

• Your pip version is old

• Your Airflow version is incompatible with the provider

• You installed Airflow without constraints

• Conflicting Python package versions exist

Below are proven fixes, starting from the safest.

---

✅ Fix 1 (MOST COMMON): Upgrade pip first

Old pip versions trigger this exact error.

(airflow-venv) frank@ZZHPC:~/download$ pip install --upgrade pip setuptools wheel
Requirement already satisfied: pip in /home/frank/airflow-venv/lib/python3.12/site-packages (24.0)
Collecting pip
  Downloading pip-25.3-py3-none-any.whl.metadata (4.7 kB)
Collecting setuptools
  Using cached setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB)
Collecting wheel
  Downloading wheel-0.45.1-py3-none-any.whl.metadata (2.3 kB)
Downloading pip-25.3-py3-none-any.whl (1.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 1.9 MB/s eta 0:00:00
Using cached setuptools-80.9.0-py3-none-any.whl (1.2 MB)
Downloading wheel-0.45.1-py3-none-any.whl (72 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 72.5/72.5 kB 5.6 MB/s eta 0:00:00
Installing collected packages: wheel, setuptools, pip
  Attempting uninstall: pip
    Found existing installation: pip 24.0
    Uninstalling pip-24.0:
      Successfully uninstalled pip-24.0
Successfully installed pip-25.3 setuptools-80.9.0 wheel-0.45.1

Then retry:

(airflow-venv) frank@ZZHPC:~/download$ pip install apache-airflow-providers-apache-spark
......
Successfully installed apache-airflow-providers-apache-spark-5.4.1 apache-airflow-providers-common-compat-1.10.1 grpcio-status-1.62.3