用ipython 写spark
Spark ipython
2023-09-14 09:09:29 时间
只需要在pyspark文件里做出如下修改就行:
# Default to standard python interpreter unless told otherwise
if [[ -z "$PYSPARK_DRIVER_PYTHON" ]]; then
PYSPARK_DRIVER_PYTHON="${PYSPARK_PYTHON:-"ipython"}"
fi