UPath¶
Description:
- Universal Pathlib is a python library that aims to extend Python's built-in pathlib.Path api to use a variety of backend filesystems using fsspec
Resources:
Environment variables¶
Example 2 reads its setting from an environment variable. Set it before starting Jupyter:
BOX_JWT_CONFIG_PATH: path to the JWT config JSON of your Box app (seebox/boxfs.ipynb); Example 1 needs no settings
Example 1¶
Use the universal-pathlib UPath class like a pathlib.Path on fsspec's in-memory filesystem. The same API works for s3://, gs://, box:// and other fsspec URLs.
In [ ]:
Copied!
from upath import UPath
path = UPath("memory:///ganymede_demo/notes.txt")
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("hello from upath")
print(path, path.name, path.suffix)
print(path.read_text())
print([p.name for p in path.parent.iterdir()])
from upath import UPath
path = UPath("memory:///ganymede_demo/notes.txt")
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("hello from upath")
print(path, path.name, path.suffix)
print(path.read_text())
print([p.name for p in path.parent.iterdir()])
Example 2¶
List a directory on box.com. This needs the boxfs package and a Box JWT app; set BOX_JWT_CONFIG_PATH to the path of the app's JWT config JSON (see box/boxfs.ipynb) before starting Jupyter.
In [ ]:
Copied!
import os
from boxsdk import JWTAuth
from upath import UPath
# The JWT app config JSON from the Box developer console; keep it outside the repository
oauth = JWTAuth.from_settings_file(os.environ["BOX_JWT_CONFIG_PATH"])
# Extra keyword arguments are passed to the box filesystem
path = UPath("box://Documents", oauth=oauth)
print(path)
print([p.name for p in path.iterdir()])
import os
from boxsdk import JWTAuth
from upath import UPath
# The JWT app config JSON from the Box developer console; keep it outside the repository
oauth = JWTAuth.from_settings_file(os.environ["BOX_JWT_CONFIG_PATH"])
# Extra keyword arguments are passed to the box filesystem
path = UPath("box://Documents", oauth=oauth)
print(path)
print([p.name for p in path.iterdir()])