Default Service Account ... is missing required permissions: [dataproc.agents.create, ...] | The Compute Engine default service account lacks roles/dataproc.worker. Grant it along with roles/storage.objectAdmin, then wait 60 seconds for IAM to propagate. |
UNAVAILABLE ... zone does not have enough resources | Zone capacity shortage. Omit --zone so Dataproc retries across zones, try e2-standard-8 instead of n2-standard-8, or switch region. |
ALREADY_EXISTS: Failed to create cluster | A previous failed attempt left the cluster in ERROR state. Delete it before retrying. |
| Console shows only Cluster on GKE | Use the gcloud CLI. Creation works through the API regardless of what the console displays. |
| HTTP 502 on the JupyterLab link | Jupyter has not finished starting — wait 2–3 minutes after RUNNING. If it persists after a stop/start cycle, delete and recreate the cluster. |
Network is unreachable during pip install | The cluster has no outbound internet. Stage the wheel in GCS and install from the local file. |
ModuleNotFoundError: No module named 'zingg' | The open-source base package is missing, or the kernel was not restarted after installing. Install all three wheels, then Kernel → Restart Kernel. |
NameError: name 'ECsvPipe' is not defined | Missing import. Each Enterprise class comes from its own module — see Step 6. |
Cannot call methods on a stopped SparkContext | Two phases were run in separate cells, each creating a new context. Use trainMatch, or restart the kernel between phases. |
ConnectionRefusedError: [Errno 111] | The JVM is gone — usually a consequence of the error above, or a driver OOM. Restart the kernel and re-run from Step 5. |
java.lang.OutOfMemoryError: Java heap space | Driver heap exhausted building the query plan. Recreate on a *-standard-8 node with spark.driver.memory=12g and confirm the checkpoint directory is set. |
ClassNotFoundException on the first Zingg call | A path in spark.jars is wrong. The cluster builds fine regardless — verify paths against gcloud storage ls. |