v4.2.0
[SPARK-55474][PYTHON][TESTS] Remove test files from ignore list of ruff
[SPARK-54950][PYTHON][TEST] Move unittest magic to unittestutils
[SPARK-55986][PYTHON] Upgrade black to 26.3.1
[SPARK-54944][PYTHON][TEST] Add tests for pa.Array.to_pandas with default arguments
[SPARK-37022][PYTHON] Use black as a formatter for PySpark
[SPARK-56607][PYTHON][FOLLOWUP] Use pyspark.sql.DataFrame to support connect-only
[SPARK-56463][PYTHON] Disallow unpickling UDT
[SPARK-55306] Add ability to run Kafka tests in Python
[SPARK-54906][PYTHON][TESTS][FOLLOWUP] Try to use a real module name instead of `__main__`
[SPARK-54090][PYTHON] Fix cascading diff in assertDataFrameEqual when row counts differ