deepr.io.Path

class deepr.io.Path(*args)

Equivalent of pathlib.Path for local and HDFS FileSystem

Automatically opens and closes an HDFS connection if the path is an HDFS path.

Allows you to work with local / HDFS files in an agnostic manner.

Example

path = Path("viewfs://foo", "bar") / "baz"
path.parent.mkdir()
with path.open("r") as file:
    for line in file:
        print(line)
for path in path.glob("*"):
    print(path.is_file())

__init__(*args)

Methods

__init__(*args)
copy_dir(dest[, recursive, filesystem])	Copy current files and directories if recursive to dest.
copy_file(dest[, filesystem])	Copy current file to dest (target directory must exist).
delete([filesystem])	Delete file from filesystem
delete_dir([filesystem])	Delete dir from filesystem
exists([filesystem])	Return True if the path points to an existing file or dir.
glob(pattern)	Retrieve directory content matching pattern
is_dir([filesystem])	Return True if the path points to a regular directory.
is_file([filesystem])	Return True if the path points to a regular file.
iterdir([filesystem])	Retrieve directory content.
mkdir([parents, exist_ok, filesystem])	Create directory
open([mode, encoding, filesystem])	Open file on both HDFS and Local File Systems.

Attributes

is_hdfs	Return True if the path points to an HDFS location
is_local	Return True if the path points to a local file or dir.
name	Final path component.
parent	Path to the parent of the current path
suffix	File extension of the file if any.