datachain --help
usage: datachain [-h] [-V] command ...

DataChain: Wrangle unstructured AI data at scale.

options:
  -h, --help     show this help message and exit
  -V, --version  show program's version number and exit

Available Commands:
  command        Use `datachain command --help` for command-specific help
    studio       Manage Studio authentication
    job          Manage jobs in Studio

usage: datachain cp [-h] [--aws-endpoint-url AWS_ENDPOINT_URL] [--anon] [-u] [-v] [-q] [-f] [-r] [--no-glob] sources [sources ...] output

Copy data files from the cloud.

positional arguments:
  sources               Data sources - paths to cloud storage dirs
  output                Output

options:
  -h, --help            show this help message and exit
  --aws-endpoint-url AWS_ENDPOINT_URL
                        AWS endpoint URL
  --anon                AWS anon (aka awscli's --no-sign-request)
  -u, --update          Update cache
  -v, --verbose         Verbose
  -q, --quiet           Be quiet
  -f, --force           Force creating outputs
  -r, -R, --recursive   Copy directories recursively
  --no-glob             Do not expand globs (such as * or ?)

usage: datachain clone [-h] [--aws-endpoint-url AWS_ENDPOINT_URL] [--anon] [-u] [-v] [-q] [-f] [-r] [--no-glob] [--no-cp] [--edatachain] [--edatachain-file EDATACHAIN_FILE] sources [sources ...] output

Copy data files from the cloud.

positional arguments:
  sources               Data sources - paths to cloud storage dirs
  output                Output

options:
  -h, --help            show this help message and exit
  --aws-endpoint-url AWS_ENDPOINT_URL
                        AWS endpoint URL
  --anon                AWS anon (aka awscli's --no-sign-request)
  -u, --update          Update cache
  -v, --verbose         Verbose
  -q, --quiet           Be quiet
  -f, --force           Force creating outputs
  -r, -R, --recursive   Copy directories recursively
  --no-glob             Do not expand globs (such as * or ?)
  --no-cp               Do not copy files, just create a dataset
  --edatachain          Create a .edatachain file
  --edatachain-file EDATACHAIN_FILE
                        Use a different filename for the resulting .edatachain file

usage: datachain studio [-h] [--aws-endpoint-url AWS_ENDPOINT_URL] [--anon] [-u] [-v] [-q] {login,logout,team,token,dataset} ...

Manage authentication and settings for Studio. Configure tokens for sharing datasets and using Studio features.

positional arguments:
  {login,logout,team,token,dataset}
                        Use `datachain studio CMD --help` to display command-specific help
    login               Authenticate with Studio
    logout              Log out from Studio
    team                Set default team for Studio operations
    token               View Studio authentication token
    dataset             List available Studio datasets

options:
  -h, --help            show this help message and exit
  --aws-endpoint-url AWS_ENDPOINT_URL
                        AWS endpoint URL
  --anon                AWS anon (aka awscli's --no-sign-request)
  -u, --update          Update cache
  -v, --verbose         Verbose
  -q, --quiet           Be quiet

usage: datachain job [-h] [--aws-endpoint-url AWS_ENDPOINT_URL] [--anon] [-u] [-v] [-q] {run,cancel,logs} ...

Commands to manage job execution in Studio.

positional arguments:
  {run,cancel,logs}     Use `datachain studio CMD --help` to display command-specific help
    run                 Run a job in Studio
    cancel              Cancel a job in Studio
    logs                Show job logs and status in Studio

options:
  -h, --help            show this help message and exit
  --aws-endpoint-url AWS_ENDPOINT_URL
                        AWS endpoint URL
  --anon                AWS anon (aka awscli's --no-sign-request)
  -u, --update          Update cache
  -v, --verbose         Verbose
  -q, --quiet           Be quiet

usage: datachain dataset [-h] [--aws-endpoint-url AWS_ENDPOINT_URL] [--anon] [-u] [-v] [-q] {pull,edit,ls,rm,remove,stats} ...

Commands for managing datasets.

positional arguments:
  {pull,edit,ls,rm,remove,stats}
                        Use `datachain dataset CMD --help` to display command-specific help

options:
  -h, --help            show this help message and exit
  --aws-endpoint-url AWS_ENDPOINT_URL
                        AWS endpoint URL
  --anon                AWS anon (aka awscli's --no-sign-request)
  -u, --update          Update cache
  -v, --verbose         Verbose
  -q, --quiet           Be quiet

usage: datachain ls [-h] [--aws-endpoint-url AWS_ENDPOINT_URL] [--anon] [-u] [-v] [-q] [-l] [--studio] [-L] [-a] [--team TEAM] [sources ...]

List storage contents.

positional arguments:
  sources               Data sources - paths to cloud storage dirs

options:
  -h, --help            show this help message and exit
  --aws-endpoint-url AWS_ENDPOINT_URL
                        AWS endpoint URL
  --anon                AWS anon (aka awscli's --no-sign-request)
  -u, --update          Update cache
  -v, --verbose         Verbose
  -q, --quiet           Be quiet
  -l, --long            List files in long format
  --studio              List the files in the Studio
  -L, --local           List local files only
  -a, --all             List all files including hidden files
  --team TEAM           The team to list datasets for. By default, it will use team from config

usage: datachain du [-h] [--aws-endpoint-url AWS_ENDPOINT_URL] [--anon] [-u] [-v] [-q] [-b] [-d N] [--si] sources [sources ...]

Display space usage.

positional arguments:
  sources               Data sources - paths to cloud storage dirs

options:
  -h, --help            show this help message and exit
  --aws-endpoint-url AWS_ENDPOINT_URL
                        AWS endpoint URL
  --anon                AWS anon (aka awscli's --no-sign-request)
  -u, --update          Update cache
  -v, --verbose         Verbose
  -q, --quiet           Be quiet
  -b, --bytes           Display sizes in bytes instead of human-readable sizes
  -d N, --depth N, --max-depth N
                        Display sizes up to N directory levels deep (default: 0, summarize provided directory only)
  --si                  Display sizes using powers of 1000 not 1024

usage: datachain find [-h] [--aws-endpoint-url AWS_ENDPOINT_URL] [--anon] [-u] [-v] [-q] [--name NAME] [--iname INAME] [--path PATH] [--ipath IPATH] [--size SIZE] [--type TYPE] [-c COLUMNS]
                      sources [sources ...]

Search in a directory hierarchy.

positional arguments:
  sources               Data sources - paths to cloud storage dirs

options:
  -h, --help            show this help message and exit
  --aws-endpoint-url AWS_ENDPOINT_URL
                        AWS endpoint URL
  --anon                AWS anon (aka awscli's --no-sign-request)
  -u, --update          Update cache
  -v, --verbose         Verbose
  -q, --quiet           Be quiet
  --name NAME           Match filename pattern
  --iname INAME         Match filename pattern (case insensitive)
  --path PATH           Path to match pattern
  --ipath IPATH         Like -path but case insensitive
  --size SIZE           Filter by size (+ is greater or equal, - is less or equal). Specified size is in bytes, or use a suffix like K, M, G for kilobytes, megabytes, gigabytes, etc
  --type TYPE           File type: "f" - regular, "d" - directory
  -c COLUMNS, --columns COLUMNS
                        A comma-separated list of columns to print for each result. Options are: du,name,path,size,type (Default: path)

datachain index --help
usage: datachain index [-h] [--aws-endpoint-url AWS_ENDPOINT_URL] [--anon] [-u] [-v] [-q] sources [sources ...]

Index storage location.

positional arguments:
  sources               Data sources - paths to cloud storage dirs

options:
  -h, --help            show this help message and exit
  --aws-endpoint-url AWS_ENDPOINT_URL
                        AWS endpoint URL
  --anon                AWS anon (aka awscli's --no-sign-request)
  -u, --update          Update cache
  -v, --verbose         Verbose
  -q, --quiet           Be quiet

usage: datachain show [-h] [--aws-endpoint-url AWS_ENDPOINT_URL] [--anon] [-u] [-v] [-q] [--version VERSION] [--schema] [--limit LIMIT] [--offset OFFSET] [--columns COLUMNS] [--no-collapse] name

Create a new dataset with a query script.

positional arguments:
  name                  Dataset name

options:
  -h, --help            show this help message and exit
  --aws-endpoint-url AWS_ENDPOINT_URL
                        AWS endpoint URL
  --anon                AWS anon (aka awscli's --no-sign-request)
  -u, --update          Update cache
  -v, --verbose         Verbose
  -q, --quiet           Be quiet
  --version VERSION     Dataset version
  --schema              Show schema
  --limit LIMIT         Number of rows to show
  --offset OFFSET       Number of rows to offset
  --columns COLUMNS     Columns to show
  --no-collapse         Do not collapse the columns

usage: datachain query [-h] [--aws-endpoint-url AWS_ENDPOINT_URL] [--anon] [-u] [-v] [-q] [--parallel [N]] [-p param=value] <script.py>

Create a new dataset with a query script.

positional arguments:
  <script.py>           Filepath for script

options:
  -h, --help            show this help message and exit
  --aws-endpoint-url AWS_ENDPOINT_URL
                        AWS endpoint URL
  --anon                AWS anon (aka awscli's --no-sign-request)
  -u, --update          Update cache
  -v, --verbose         Verbose
  -q, --quiet           Be quiet
  --parallel [N]        Use multiprocessing to run any query script UDFs with N worker processes. N defaults to the CPU count
  -p param=value, --param param=value
                        Query parameters

datachain clear-cache --help
usage: datachain clear-cache [-h] [--aws-endpoint-url AWS_ENDPOINT_URL] [--anon] [-u] [-v] [-q]

Clear the local file cache.

options:
  -h, --help            show this help message and exit
  --aws-endpoint-url AWS_ENDPOINT_URL
                        AWS endpoint URL
  --anon                AWS anon (aka awscli's --no-sign-request)
  -u, --update          Update cache
  -v, --verbose         Verbose
  -q, --quiet           Be quiet

datachain gc --help
usage: datachain gc [-h] [--aws-endpoint-url AWS_ENDPOINT_URL] [--anon] [-u] [-v] [-q]

Garbage collect temporary tables.

options:
  -h, --help            show this help message and exit
  --aws-endpoint-url AWS_ENDPOINT_URL
                        AWS endpoint URL
  --anon                AWS anon (aka awscli's --no-sign-request)
  -u, --update          Update cache
  -v, --verbose         Verbose
  -q, --quiet           Be quiet

datachain completion --help
usage: datachain completion [-h] [--aws-endpoint-url AWS_ENDPOINT_URL] [--anon] [-u] [-v] [-q] [-s {bash,zsh,tcsh}]

Output shell completion script.

options:
  -h, --help            show this help message and exit
  --aws-endpoint-url AWS_ENDPOINT_URL
                        AWS endpoint URL
  --anon                AWS anon (aka awscli's --no-sign-request)
  -u, --update          Update cache
  -v, --verbose         Verbose
  -q, --quiet           Be quiet
  -s {bash,zsh,tcsh}, --shell {bash,zsh,tcsh}
                        Shell syntax for completions