/usr/bin/vtbackup

vtbackup is a batch command to perform a single pass of backup maintenance for a shard.

When run periodically for each shard, vtbackup can ensure these configurable policies:
 * There is always a recent backup for the shard.

 * Old backups for the shard are removed.

Whatever system launches vtbackup is responsible for the following:
 - Running vtbackup with similar flags that would be used for a vttablet and
   mysqlctld in the target shard to be backed up.

 - Provisioning as much disk space for vtbackup as would be given to vttablet.
   The data directory MUST be empty at startup. Do NOT reuse a persistent disk.

 - Running vtbackup periodically for each shard, for each backup storage location.

 - Ensuring that at most one instance runs at a time for a given pair of shard
   and backup storage location.

 - Retrying vtbackup if it fails.

 - Alerting human operators if the failure is persistent.

The process vtbackup follows to take a new backup has the following steps:
 1. Restore from the most recent backup.
 2. Start a mysqld instance (but no vttablet) from the restored data.
 3. Instruct mysqld to connect to the current shard primary and replicate any
    transactions that are new since the last backup.
 4. Ask the primary for its current replication position and set that as the goal
    for catching up on replication before taking the backup, so the goalposts
    don't move.
 5. Wait until replication is caught up to the goal position or beyond.
 6. Stop mysqld and take a new backup.

Aside from additional replication load while vtbackup's mysqld catches up on
new transactions, the shard should be otherwise unaffected. Existing tablets
will continue to serve, and no new tablets will appear in topology, meaning no
query traffic will ever be routed to vtbackup's mysqld. This silent operation
mode helps make backups minimally disruptive to serving capacity and orthogonal
to the handling of the query path.

The command-line parameters to vtbackup specify a policy for when a new backup
is needed, and when old backups should be removed. If the existing backups
already satisfy the policy, then vtbackup will do nothing and return success
immediately.

Usage:
  vtbackup [flags]

Flags:
      --allow-first-backup                                          Allow this job to take the first backup of an existing shard.
      --alsologtostderr                                             log to standard error as well as files
      --azblob-backup-account-key-file string                       Path to a file containing the Azure Storage account key; if this flag is unset, the environment variable VT_AZBLOB_ACCOUNT_KEY will be used as the key itself (NOT a file path).
      --azblob-backup-account-name string                           Azure Storage Account name for backups; if this flag is unset, the environment variable VT_AZBLOB_ACCOUNT_NAME will be used.
      --azblob-backup-buffer-size int                               The memory buffer size to use in bytes, per file or stripe, when streaming to Azure Blob Service. (default 104857600)
      --azblob-backup-container-name string                         Azure Blob Container Name.
      --azblob-backup-parallelism int                               Azure Blob operation parallelism (requires extra memory when increased -- a multiple of azblob-backup-buffer-size). (default 1)
      --azblob-backup-storage-root string                           Root prefix for all backup-related Azure Blobs; this should exclude both initial and trailing '/' (e.g. just 'a/b' not '/a/b/').
      --backup-engine-implementation string                         Specifies which implementation to use for creating new backups (builtin or xtrabackup). Restores will always be done with whichever engine created a given backup. (default "builtin")
      --backup-storage-block-size int                               if backup-storage-compress is true, backup-storage-block-size sets the byte size for each block while compressing (default is 250000). (default 250000)
      --backup-storage-compress                                     if set, the backup files will be compressed. (default true)
      --backup-storage-implementation string                        Which backup storage implementation to use for creating and restoring backups.
      --backup-storage-number-blocks int                            if backup-storage-compress is true, backup-storage-number-blocks sets the number of blocks that can be processed, in parallel, before the writer blocks, during compression (default is 2). It should be equal to the number of CPUs available for compression. (default 2)
      --bind-address string                                         Bind address for the server. If empty, the server will listen on all available unicast and anycast IP addresses of the local system.
      --builtinbackup-file-read-buffer-size uint                    read files using an IO buffer of this many bytes. Golang defaults are used when set to 0.
      --builtinbackup-file-write-buffer-size uint                   write files using an IO buffer of this many bytes. Golang defaults are used when set to 0. (default 2097152)
      --builtinbackup-incremental-restore-path string               the directory where incremental restore files, namely binlog files, are extracted to. In k8s environments, this should be set to a directory that is shared between the vttablet and mysqld pods. The path should exist. When empty, the default OS temp dir is assumed.
      --builtinbackup-mysqld-timeout duration                       how long to wait for mysqld to shutdown at the start of the backup. (default 10m0s)
      --builtinbackup-progress duration                             how often to send progress updates when backing up large files. (default 5s)
      --ceph-backup-storage-config string                           Path to JSON config file for ceph backup storage. (default "ceph_backup_config.json")
      --compression-engine-name string                              compressor engine used for compression. (default "pargzip")
      --compression-level int                                       what level to pass to the compressor. (default 1)
      --concurrency int                                             (init restore parameter) how many concurrent files to restore at once (default 4)
      --config-file string                                          Full path of the config file (with extension) to use. If set, --config-path, --config-type, and --config-name are ignored.
      --config-file-not-found-handling ConfigFileNotFoundHandling   Behavior when a config file is not found. (Options: error, exit, ignore, warn) (default warn)
      --config-name string                                          Name of the config file (without extension) to search for. (default "vtconfig")
      --config-path strings                                         Paths to search for config files in. (default [/tmp])
      --config-persistence-min-interval duration                    minimum interval between persisting dynamic config changes back to disk (if no change has occurred, nothing is done). (default 1s)
      --config-type string                                          Config file type (omit to infer config type from file extension).
      --consul-auth-static-file string                              JSON File to read the topos/tokens from.
      --db-allprivs-password string                                 db allprivs password
      --db-allprivs-use-ssl                                         Set this flag to false to make the allprivs connection to not use ssl (default true)
      --db-allprivs-user string                                     db allprivs user userKey (default "vt_allprivs")
      --db-app-password string                                      db app password
      --db-app-use-ssl                                              Set this flag to false to make the app connection to not use ssl (default true)
      --db-app-user string                                          db app user userKey (default "vt_app")
      --db-appdebug-password string                                 db appdebug password
      --db-appdebug-use-ssl                                         Set this flag to false to make the appdebug connection to not use ssl (default true)
      --db-appdebug-user string                                     db appdebug user userKey (default "vt_appdebug")
      --db-charset string                                           Character set/collation used for this tablet. Make sure to configure this to a charset/collation supported by the lowest MySQL version in your environment. (default "utf8mb4")
      --db-conn-query-info                                          enable parsing and processing of QUERY_OK info fields
      --db-connect-timeout-ms int                                   connection timeout to mysqld in milliseconds (0 for no timeout)
      --db-credentials-file string                                  db credentials file; send SIGHUP to reload this file
      --db-credentials-server string                                db credentials server type ('file' - file implementation; 'vault' - HashiCorp Vault implementation) (default "file")
      --db-credentials-vault-addr string                            URL to Vault server
      --db-credentials-vault-path string                            Vault path to credentials JSON blob, e.g.: secret/data/prod/dbcreds
      --db-credentials-vault-role-mountpoint string                 Vault AppRole mountpoint; can also be passed using VAULT_MOUNTPOINT environment variable (default "approle")
      --db-credentials-vault-role-secretidfile string               Path to file containing Vault AppRole secret_id; can also be passed using VAULT_SECRETID environment variable
      --db-credentials-vault-roleid string                          Vault AppRole id; can also be passed using VAULT_ROLEID environment variable
      --db-credentials-vault-timeout duration                       Timeout for vault API operations (default 10s)
      --db-credentials-vault-tls-ca string                          Path to CA PEM for validating Vault server certificate
      --db-credentials-vault-tokenfile string                       Path to file containing Vault auth token; token can also be passed using VAULT_TOKEN environment variable
      --db-credentials-vault-ttl duration                           How long to cache DB credentials from the Vault server (default 30m0s)
      --db-dba-password string                                      db dba password
      --db-dba-use-ssl                                              Set this flag to false to make the dba connection to not use ssl (default true)
      --db-dba-user string                                          db dba user userKey (default "vt_dba")
      --db-erepl-password string                                    db erepl password
      --db-erepl-use-ssl                                            Set this flag to false to make the erepl connection to not use ssl (default true)
      --db-erepl-user string                                        db erepl user userKey (default "vt_erepl")
      --db-filtered-password string                                 db filtered password
      --db-filtered-use-ssl                                         Set this flag to false to make the filtered connection to not use ssl (default true)
      --db-filtered-user string                                     db filtered user userKey (default "vt_filtered")
      --db-flags uint                                               Flag values as defined by MySQL.
      --db-flavor string                                            Flavor overrid. Valid value is FilePos.
      --db-host string                                              The host name for the tcp connection.
      --db-port int                                                 tcp port
      --db-repl-password string                                     db repl password
      --db-repl-use-ssl                                             Set this flag to false to make the repl connection to not use ssl (default true)
      --db-repl-user string                                         db repl user userKey (default "vt_repl")
      --db-server-name string                                       server name of the DB we are connecting to.
      --db-socket string                                            The unix socket to connect on. If this is specified, host and port will not be used.
      --db-ssl-ca string                                            connection ssl ca
      --db-ssl-ca-path string                                       connection ssl ca path
      --db-ssl-cert string                                          connection ssl certificate
      --db-ssl-key string                                           connection ssl key
      --db-ssl-mode SslMode                                         SSL mode to connect with. One of disabled, preferred, required, verify_ca & verify_identity.
      --db-tls-min-version string                                   Configures the minimal TLS version negotiated when SSL is enabled. Defaults to TLSv1.2. Options: TLSv1.0, TLSv1.1, TLSv1.2, TLSv1.3.
      --detach                                                      detached mode - run backups detached from the terminal
      --disable-redo-log                                            Disable InnoDB redo log during replication-from-primary phase of backup.
      --emit-stats                                                  If set, emit stats to push-based monitoring and stats backends
      --external-compressor string                                  command with arguments to use when compressing a backup.
      --external-compressor-extension string                        extension to use when using an external compressor.
      --external-decompressor string                                command with arguments to use when decompressing a backup.
      --external-decompressor-use-manifest                          allows the decompressor command stored in the backup manifest to be used at restore time. Enabling this is a security risk: an attacker with write access to the backup storage could modify the manifest to execute arbitrary commands on the tablet as the Vitess user. NOT RECOMMENDED.
      --file-backup-storage-root string                             Root directory for the file backup storage.
      --gcs-backup-storage-bucket string                            Google Cloud Storage bucket to use for backups.
      --gcs-backup-storage-root string                              Root prefix for all backup-related object names.
      --grpc-auth-static-client-creds string                        When using grpc_static_auth in the server, this file provides the credentials to use to authenticate with server.
      --grpc-compression string                                     Which protocol to use for compressing gRPC. Default: nothing. Supported: snappy
      --grpc-dial-concurrency-limit int                             Maximum concurrency of grpc dial operations. This should be less than the golang max thread limit of 10000. (default 1024)
      --grpc-enable-tracing                                         Enable gRPC tracing.
      --grpc-initial-conn-window-size int                           gRPC initial connection window size
      --grpc-initial-window-size int                                gRPC initial window size
      --grpc-keepalive-time duration                                After a duration of this time, if the client doesn't see any activity, it pings the server to see if the transport is still alive. (default 10s)
      --grpc-keepalive-timeout duration                             After having pinged for keepalive check, the client waits for a duration of Timeout and if no activity is seen even after that the connection is closed. (default 10s)
      --grpc-max-message-size int                                   Maximum allowed RPC message size. Larger messages will be rejected by gRPC with the error 'exceeding the max size'. (default 16777216)
      --grpc-prometheus                                             Enable gRPC monitoring with Prometheus.
  -h, --help                                                        help for vtbackup
      --incremental-from-pos string                                 Position, or name of backup from which to create an incremental backup. Default: empty. If given, then this backup becomes an incremental backup from given position or given backup. If value is 'auto', this backup will be taken from the last successful backup position.
      --init-db-name-override string                                (init parameter) override the name of the db used by vttablet
      --init-db-sql-file string                                     path to .sql file to run after mysql_install_db
      --init-keyspace string                                        (init parameter) keyspace to use for this tablet
      --init-shard string                                           (init parameter) shard to use for this tablet
      --initial-backup                                              Instead of restoring from backup, initialize an empty database with the provided init-db-sql-file and upload a backup of that for the shard, if the shard has no backups yet. This can be used to seed a brand new shard with an initial, empty backup. If any backups already exist for the shard, this will be considered a successful no-op. This can only be done before the shard exists in topology (i.e. before any tablets are deployed).
      --keep-alive-timeout duration                                 Wait until timeout elapses after a successful backup before shutting down.
      --keep-logs duration                                          keep logs for this long (using ctime) (zero to keep forever)
      --keep-logs-by-mtime duration                                 keep logs for this long (using mtime) (zero to keep forever)
      --lock-timeout duration                                       Maximum time to wait when attempting to acquire a lock from the topo server (default 45s)
      --log-err-stacks                                              log stack traces for errors
      --log-rotate-max-size uint                                    size in bytes at which logs are rotated (glog.MaxSize) (default 1887436800)
      --log_backtrace_at traceLocations                             when logging hits line file:N, emit a stack trace
      --log_dir string                                              If non-empty, write log files in this directory
      --logtostderr                                                 log to standard error instead of files
      --manifest-external-decompressor string                       command with arguments to store in the backup manifest when compressing a backup with an external compression engine.
      --min-backup-interval duration                                Only take a new backup if it's been at least this long since the most recent backup.
      --min-retention-count int                                     Always keep at least this many of the most recent backups in this backup storage location, even if some are older than the min_retention_time. This must be at least 1 since a backup must always exist to allow new backups to be made (default 1)
      --min-retention-time duration                                 Keep each old backup for at least this long before removing it. Set to 0 to disable pruning of old backups.
      --mycnf-bin-log-path string                                   mysql binlog path
      --mycnf-data-dir string                                       data directory for mysql
      --mycnf-error-log-path string                                 mysql error log path
      --mycnf-file string                                           path to my.cnf, if reading all config params from there
      --mycnf-general-log-path string                               mysql general log path
      --mycnf-innodb-data-home-dir string                           Innodb data home directory
      --mycnf-innodb-log-group-home-dir string                      Innodb log group home directory
      --mycnf-master-info-file string                               mysql master.info file
      --mycnf-mysql-port int                                        port mysql is listening on
      --mycnf-pid-file string                                       mysql pid file
      --mycnf-relay-log-index-path string                           mysql relay log index path
      --mycnf-relay-log-info-path string                            mysql relay log info path
      --mycnf-relay-log-path string                                 mysql relay log path
      --mycnf-secure-file-priv string                               mysql path for loading secure files
      --mycnf-server-id int                                         mysql server id of the server (if specified, mycnf-file will be ignored)
      --mycnf-slow-log-path string                                  mysql slow query log path
      --mycnf-socket-file string                                    mysql socket file
      --mycnf-tmp-dir string                                        mysql tmp directory
      --mysql-port int                                              MySQL port (default 3306)
      --mysql-server-version string                                 MySQL server version to advertise. (default "8.4.6-Vitess")
      --mysql-shell-backup-location string                          location where the backup will be stored
      --mysql-shell-dump-flags string                               flags to pass to mysql shell dump utility. This should be a JSON string and will be saved in the MANIFEST (default "{\"threads\": 4}")
      --mysql-shell-flags string                                    execution flags to pass to mysqlsh binary to be used during dump/load (default "--defaults-file=/dev/null --js -h localhost")
      --mysql-shell-load-flags string                               flags to pass to mysql shell load utility. This should be a JSON string (default "{\"threads\": 4, \"loadUsers\": true, \"updateGtidSet\": \"replace\", \"skipBinlog\": true, \"progressFile\": \"\"}")
      --mysql-shell-should-drain                                    decide if we should drain while taking a backup or continue to serving traffic
      --mysql-shell-speedup-restore                                 speed up restore by disabling redo logging and double write buffer during the restore process
      --mysql-shutdown-timeout duration                             how long to wait for mysqld shutdown (default 5m0s)
      --mysql-socket string                                         Path to the mysqld socket file
      --mysql-timeout duration                                      how long to wait for mysqld startup (default 5m0s)
      --opentsdb-uri string                                         URI of opentsdb /api/put method
      --port int                                                    port for the server
      --pprof strings                                               enable profiling
      --pprof-http                                                  enable pprof http endpoints
      --purge-logs-interval duration                                how often try to remove old logs (default 1h0m0s)
      --remote-operation-timeout duration                           time to wait for a remote operation (default 15s)
      --restart-before-backup                                       Perform a mysqld clean/full restart after applying binlogs, but before taking the backup. Only makes sense to work around xtrabackup bugs.
      --s3-backup-aws-endpoint string                               endpoint of the S3 backend (region must be provided).
      --s3-backup-aws-min-partsize int                              Minimum part size to use, defaults to 5MiB but can be increased due to the dataset size. (default 5242880)
      --s3-backup-aws-region string                                 AWS region to use. (default "us-east-1")
      --s3-backup-aws-retries int                                   AWS request retries. (default -1)
      --s3-backup-force-path-style                                  force the s3 path style.
      --s3-backup-log-level string                                  determine the S3 loglevel to use from LogOff, LogDebug, LogDebugWithSigning, LogDebugWithHTTPBody, LogDebugWithRequestRetries, LogDebugWithRequestErrors. (default "LogOff")
      --s3-backup-server-side-encryption string                     server-side encryption algorithm (e.g., AES256, aws:kms, sse_c:/path/to/key/file).
      --s3-backup-storage-bucket string                             S3 bucket to use for backups.
      --s3-backup-storage-root string                               root prefix for all backup-related object names.
      --s3-backup-tls-skip-verify-cert                              skip the 'certificate is valid' check for SSL connections.
      --security-policy string                                      the name of a registered security policy to use for controlling access to URLs - empty means allow all for anyone (built-in policies: deny-all, read-only)
      --sql-max-length-errors int                                   truncate queries in error logs to the given length (default unlimited)
      --sql-max-length-ui int                                       truncate queries in debug UIs to the given length (default 512) (default 512)
      --stats-backend string                                        The name of the registered push-based monitoring/stats backend to use
      --stats-combine-dimensions string                             List of dimensions to be combined into a single "all" value in exported stats vars
      --stats-common-tags strings                                   Comma-separated list of common tags for the stats backend. It provides both label and values. Example: label1:value1,label2:value2
      --stats-drop-variables string                                 Variables to be dropped from the list of exported variables.
      --stats-emit-period duration                                  Interval between emitting stats to all registered backends (default 1m0s)
      --stderrthreshold severityFlag                                logs at or above this threshold go to stderr (default 1)
      --tablet-manager-grpc-ca string                               the server ca to use to validate servers when connecting
      --tablet-manager-grpc-cert string                             the cert to use to connect
      --tablet-manager-grpc-concurrency int                         concurrency to use to talk to a vttablet server for performance-sensitive RPCs (like ExecuteFetchAs{Dba,App}, CheckThrottler and FullStatus) (default 8)
      --tablet-manager-grpc-connpool-size int                       number of tablets to keep tmclient connections open to (default 100)
      --tablet-manager-grpc-crl string                              the server crl to use to validate server certificates when connecting
      --tablet-manager-grpc-key string                              the key to use to connect
      --tablet-manager-grpc-server-name string                      the server name to use to validate server certificate
      --tablet-manager-protocol string                              Protocol to use to make tabletmanager RPCs to vttablets. (default "grpc")
      --topo-consul-lock-delay duration                             LockDelay for consul session. (default 15s)
      --topo-consul-lock-session-checks string                      List of checks for consul session. (default "serfHealth")
      --topo-consul-lock-session-ttl string                         TTL for consul session.
      --topo-consul-watch-poll-duration duration                    time of the long poll for watch queries. (default 30s)
      --topo-etcd-lease-ttl int                                     Lease TTL for locks and leader election. The client will use KeepAlive to keep the lease going. (default 30)
      --topo-etcd-tls-ca string                                     path to the ca to use to validate the server cert when connecting to the etcd topo server
      --topo-etcd-tls-cert string                                   path to the client cert to use to connect to the etcd topo server, requires topo-etcd-tls-key, enables TLS
      --topo-etcd-tls-key string                                    path to the client key to use to connect to the etcd topo server, enables TLS
      --topo-global-root string                                     the path of the global topology data in the global topology server
      --topo-global-server-address string                           the address of the global topology server
      --topo-implementation string                                  the topology implementation to use
      --topo-read-concurrency int                                   Maximum concurrency of topo reads per global or local cell. (default 32)
      --topo-zk-auth-file string                                    auth to use when connecting to the zk topo server, file contents should be <scheme>:<auth>, e.g., digest:user:pass
      --topo-zk-base-timeout duration                               zk base timeout (see zk.Connect) (default 30s)
      --topo-zk-max-concurrency int                                 maximum number of pending requests to send to a Zookeeper server. (default 64)
      --topo-zk-tls-ca string                                       the server ca to use to validate servers when connecting to the zk topo server
      --topo-zk-tls-cert string                                     the cert to use to connect to the zk topo server, requires topo-zk-tls-key, enables TLS
      --topo-zk-tls-key string                                      the key to use to connect to the zk topo server, enables TLS
      --upgrade-safe                                                Whether to use innodb_fast_shutdown=0 for the backup so it is safe to use for MySQL upgrades.
      --v Level                                                     log level for V logs
  -v, --version                                                     print binary version
      --vmodule vModuleFlag                                         comma-separated list of pattern=N settings for file-filtered logging
      --xbstream-restore-flags string                               Flags to pass to xbstream command during restore. These should be space separated and will be added to the end of the command. These need to match the ones used for backup e.g. --compress / --decompress, --encrypt / --decrypt
      --xtrabackup-backup-flags string                              Flags to pass to backup command. These should be space separated and will be added to the end of the command
      --xtrabackup-prepare-flags string                             Flags to pass to prepare command. These should be space separated and will be added to the end of the command
      --xtrabackup-root-path string                                 Directory location of the xtrabackup and xbstream executables, e.g., /usr/bin
      --xtrabackup-should-drain                                     Decide if we should drain while taking a backup or continue to serving traffic
      --xtrabackup-stream-mode string                               Which mode to use if streaming, valid values are tar and xbstream. Please note that tar is not supported in XtraBackup 8.0 (default "tar")
      --xtrabackup-stripe-block-size uint                           Size in bytes of each block that gets sent to a given stripe before rotating to the next stripe (default 102400)
      --xtrabackup-stripes uint                                     If greater than 0, use data striping across this many destination files to parallelize data transfer and decompression
      --xtrabackup-user string                                      User that xtrabackup will use to connect to the database server. This user must have all necessary privileges. For details, please refer to xtrabackup documentation.