Managing Files

This notebook shows how you can manage files with the ThanoSQL library. We will cover how to upload files, get the list of files in a certain directory, and delete files. First, initialize a ThanoSQL client with your API token and engine URL.

from thanosql import ThanoSQL

client = ThanoSQL(
    api_token="THANOSQL_API_TOKEN",
    engine_url="THANOSQL_ENGINE_URL"
)

Uploading Files

You can upload files from your local computer to your ThanoSQL workspace. By default, files will be uploaded to the drive directory. You can use the dir parameter to specify another folder location relative to drive/. If the directory does not exist, it will be created automatically.

file_path = "path/to/file.jpeg"

# you can run both or comment one of the following
res = client.file.upload(file_path)  # file will be uploaded to `drive/`
assert res["data"]["file_path"] == "drive/file.jpeg"

res = client.file.upload(path=file_path, dir="target_dir")  # file will be uploaded to `drive/target_dir/`
assert res["data"]["file_path"] == "drive/target_dir/file.jpeg"

By default, uploading file will fail if another file already exists in the target path. You can change this behavior by setting the optional parameter if_exists. There are only two possible values: “fail” (default) or “replace”. If “replace” is used, the new file will be uploaded regardless of whether another file with the same path exists. The old file will be overwritten by the new file.

res = client.file.upload(path=file_path, if_exists="replace")
assert res["data"]["file_path"] == "drive/file.jpeg"

You can also save the file path to a table by setting db_commit, table_name, and column_name. Note that both table_name and column_name must exist beforehand; the API will not create a new table or column and will promptly return an error otherwise. By default, db_commit will be set to False. If db_commit is True, two more values will be returned by the response data: table_name and column_name.

res = client.file.upload(path=file_path, db_commit=True, table="table_name", column="column_name", if_exists="replace")
assert res["data"]["table_name"] == "table_name"
assert res["data"]["column_name"] == "column_name"

Listing Files

You can get the list of files in a directory under drive/ using the list() function. For example, if you have this file structure:

drive/
├── audio/
├── image/
├── others/
├── video/
└── my_file.md

res = client.file.list("drive/*")
print(res["data"]["matched_pathnames"])

The output will look like:

["drive/audio", "drive/image", "drive/others", "drive/video", "drive/my_file.md"]

Note that this function is not recursive; files inside subdirectories will not be listed. For ease of use, it is recommended to use regular expression (regex) pattern as the input.

Deleting Files

You can also delete files from your workspace using the client. If you previously saved the file information in a table, you can also delete the entry in a way similar to uploading it.

# you can run both or comment one of the following
client.file.delete(path="drive/target_dir/file.jpeg")  # without db_commit
client.file.delete(path="drive/file.jpeg", db_commit=True, table="table_name", column="column_name")