/usr/lib/spark/bin/docker-image-tool.sh

Usage: /usr/lib/spark/bin/docker-image-tool.sh [options] [command]
Builds or pushes the built-in Spark Docker image.

Commands:
  build       Build image. Requires a repository address to be provided if the image will be
              pushed to a different registry.
  push        Push a pre-built image to a registry. Requires a repository address to be provided.

Options:
  -f file               (Optional) Dockerfile to build for JVM based Jobs. By default builds the Dockerfile shipped with Spark.
  -p file               (Optional) Dockerfile to build for PySpark Jobs. Builds Python dependencies and ships with Spark.
                        Skips building PySpark docker image if not specified.
  -R file               (Optional) Dockerfile to build for SparkR Jobs. Builds R dependencies and ships with Spark.
                        Skips building SparkR docker image if not specified.
  -r repo               Repository address.
  -t tag                Tag to apply to the built image, or to identify the image to be pushed.
  -m                    Use minikube's Docker daemon.
  -n                    Build docker image with --no-cache
  -u uid                UID to use in the USER directive to set the user the main Spark process runs as inside the
                        resulting container
  -X                    Use docker buildx to cross build. Automatically pushes.
                        See https://docs.docker.com/buildx/working-with-buildx/ for steps to setup buildx.
  -b arg                Build arg to build or push the image. For multiple build args, this option needs to
                        be used separately for each build arg.

Using minikube when building images will do so directly into minikube's Docker daemon.
There is no need to push the images into minikube in that case, they'll be automatically
available when running applications inside the minikube cluster.

Check the following documentation for more information on using the minikube Docker daemon:

  https://kubernetes.io/docs/getting-started-guides/minikube/#reusing-the-docker-daemon

Examples:
  - Build image in minikube with tag "testing"
    /usr/lib/spark/bin/docker-image-tool.sh -m -t testing build

  - Build PySpark docker image
    /usr/lib/spark/bin/docker-image-tool.sh -r docker.io/myrepo -t v4.0.0 -p kubernetes/dockerfiles/spark/bindings/python/Dockerfile build

  - Build and push image with tag "v4.0.0" to docker.io/myrepo
    /usr/lib/spark/bin/docker-image-tool.sh -r docker.io/myrepo -t v4.0.0 build
    /usr/lib/spark/bin/docker-image-tool.sh -r docker.io/myrepo -t v4.0.0 push

  - Build and push Java17-based image with tag "v4.0.0" to docker.io/myrepo
    /usr/lib/spark/bin/docker-image-tool.sh -r docker.io/myrepo -t v4.0.0 -b java_image_tag=17 build
    /usr/lib/spark/bin/docker-image-tool.sh -r docker.io/myrepo -t v4.0.0 push

  - Build and push image for multiple archs to docker.io/myrepo
    /usr/lib/spark/bin/docker-image-tool.sh -r docker.io/myrepo -t v4.0.0 -X build
    # Note: buildx, which does cross building, needs to do the push during build
    # So there is no separate push step with -X