/
githubmirror
/
spark
Обзор
Документация
Войти
/
githubmirror
/
spark
Код
Запросы
0
Пакеты
0
Релизы
0
Аналитика
Безопасность
master
bin/spark-pipelines
40 строк
1 KB
anishm-db
[SPARK-53498][SDP] Correctly Reference `pyspark/pipelines/cli.py` from `spark-pipelines` Binary
12 сен 2025, 05:41
Не верифицирован
12 сен 2025, 05:41
faa1aaa
Код
Авторство
О чём код?
#!/usr/bin/env bash # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # Default to standard python3 interpreter unless told otherwise if [[ -z "$PYSPARK_PYTHON" ]]; then PYSPARK_PYTHON=python3 fi if [ -z "${SPARK_HOME}" ]; then source "$(dirname "$0")"/find-spark-home fi # Add the PySpark classes to the Python path: export PYTHONPATH="${SPARK_HOME}/python/:$PYTHONPATH" export PYTHONPATH="${SPARK_HOME}/python/lib/py4j-0.10.9.9-src.zip:$PYTHONPATH" SDP_CLI_PY_FILE_PATH=$("${PYSPARK_PYTHON}" - <<'EOF' import pyspark, os from pathlib import Path print(Path(os.path.dirname(pyspark.__file__)) / "pipelines" / "cli.py") EOF ) exec "${SPARK_HOME}"/bin/spark-class org.apache.spark.deploy.SparkPipelines "$SDP_CLI_PY_FILE_PATH" "$@"