Custom file system providers/API spec

Release:	… • 4.6 • 5.0 • 5.1 • 5.2 • 5.3 • 5.4 • 6.0 • 6.1 • 6.2 • 6.3 • 6.4 • 6.5 • • 6.6 • 7.0 • 7.1 • 7.2

New in Analytica 7.2. See also the parent page, Custom file system providers.

Overview

A store service is an ordinary HTTP service that Analytica calls whenever a model uses a «scheme»:// path. A store that is only read from needs three endpoints, and nothing else is required. A store that Analytica may also write to -- saving models, WriteTextFile, FileSystemNewFolder, FileSystemCopy, FileSystemMove, FileSystemDelete, SpreadsheetSave -- implements the write and file-management endpoints as well. No Analytica code is involved, so write the service in whatever language and framework you like, and back it with whatever you like: object storage, a database, a version control system, a document management system, or a REST API of its own.

Analytica is told about the service by one line in FileProviders.config:

FileStore repo = http://mystore.internal:8080

after which every path beginning repo:// becomes a request to that base URL. See Custom file system providers for the rest of the configuration.

The endpoints

These three serve every read. All endpoints, including the write and file-management endpoints, are relative to the configured base URL, including any path prefix in it. With a base URL of http://mystore.internal:8080/api, the first endpoint is http://mystore.internal:8080/api/files/....

Request	Purpose	Success	Not found
`GET {base}/files/{root}/{path}`	Fetch a file	`200`, with the raw file bytes as the body	`404`
`HEAD {base}/files/{root}/{path}`	Existence check, without transferring the file	`200`	`404`
`GET {base}/list/{root}/{path}?recursive=0` or `?recursive=1`	List a folder	`200`, with a JSON array (see below)	`404`, taken as an empty listing

{root}/{path} is everything that followed «scheme»:// in the model's path. For repo://Sales/2026/Q3.csv the engine requests {base}/files/Sales/2026/Q3.csv. The engine attaches no meaning of its own to {root}: it is simply the first segment. It is there so that one service can host several logical roots -- one per customer, per project or per bucket -- and decide from it which backing store to consult.

The recursive query parameter is sent on every /list request and on no /files request.

A health endpoint is a good idea for your own monitoring, but the engine never calls one and does not require it.

Path encoding

Everything after «scheme»:// is the literal path -- unencoded text, as the model author wrote it. Before sending a request, the engine:

replaces every \ with /, and
percent-encodes every byte that is not in the unreserved set A-Z a-z 0-9 - _ . ~ /.

Your service must percent-decode the path. Because the encoding is applied byte by byte to the UTF-8 form of the name, spaces and non-ASCII characters round-trip correctly: repo://models/Ventas Q3.csv arrives as /files/models/Ventas%20Q3.csv.

The engine refuses a .. segment before any request is sent, so a well-behaved client never asks for one. Reject traversal in the service as well; defence in depth is worth the three lines it costs.

Authentication

When a token is configured, with FileStoreToken or FileStoreTokenFile, the engine sends

Authorization: Bearer «token»

on every request, to every endpoint. When no token is configured, no Authorization header is sent at all.

Answer 401 or 403 when the token is missing, expired or wrong. Analytica turns either one into an error that names the scheme, so the user is told the credentials were rejected rather than that the file is missing.

How the engine interprets your status codes

Status	What the engine does
`200`	Success. The body is the file, or the listing.
`404`	Not found. Treated as a missing file, with no error text of its own, so the caller reports a missing file in its usual way. On `/list`, an empty listing.
`401` or `403`	A credentials error naming the scheme: `the repo:// file store rejected the credentials (HTTP 401)`.
`405`	On a write: that store, or that part of it, is read-only. The write fails with `the repo:// file store is read-only and refused the write (HTTP 405)`.
`409`, `412`	On a write: a conflict -- the destination already exists, a folder is not empty, or a file is in the way of a folder. The function that asked reports it in its own words (for example, that the destination exists and «replace» would allow it).
`413`	On a write: the upload is too large. `the repo:// file store refused the upload as too large (HTTP 413)`.
`501`	On a copy or move: the service cannot do it itself. For a single file the engine then does the work with `GET`, `PUT` and `DELETE`; for a folder the operation fails.
anything else	`HTTP «n» from the repo:// file store`.

A connection that cannot be made at all, or that times out, is reported as could not reach the repo:// file store (...), with the transport reason in the parentheses. Neither of those is a status code you control, so return a real status for everything you can: the message the user sees is then about your service rather than about the network.

The listing response

GET {base}/list/{root}/{path} returns a JSON array of objects, one per entry. A body that does not parse, or that is not a JSON array, is an error.

Member	Type	Meaning
`name`	string	The entry name only, with no path. Required -- an entry with no `name`, or whose `name` is not a string, is skipped.
`isFolder`	boolean or number	`true`, or a non-zero number, for a folder. Absent or false means a file.
`size`	number	Size in bytes. Use 0 for folders.
`modified`	string	The last-modified time in ISO-8601, UTC, for example `"2026-08-14T09:12:03Z"`. Omit it when you do not know it.

Members other than these are ignored, so you may return extra information for your own clients.

[
  {"name": "Q1.csv",  "isFolder": false, "size": 20481, "modified": "2026-04-02T17:45:10Z"},
  {"name": "Q2.csv",  "isFolder": false, "size": 21118, "modified": "2026-07-01T08:03:55Z"},
  {"name": "archive", "isFolder": true,  "size": 0}
]

recursive=1 asks for the whole subtree below the folder. When you answer one, name should carry the path relative to the folder that was listed -- "archive/2025/Q4.csv", not just "Q4.csv". That is what lets FileSystemListing report the sub-path the way it does for a local recursive listing. With recursive=0, return only the immediate children.

The engine, not the service, applies any wildcard the model wrote in the last segment of the path, and applies the «files» and «folders» flags of FileSystemListing. Return the folder's whole contents and let it do that filtering.

Write and file-management endpoints

These are optional. A store that implements none of them is a read-only store, and every function that would write to it fails cleanly. Implement them when models are to save files, or to create, copy, move and delete them, in your store. Like the read endpoints, they are relative to the base URL, receive the same Authorization header and use the same path encoding.

Request	Purpose	Answers
`PUT {base}/files/{root}/{path}`	Write a whole file. The body is the complete file, with a `Content-Length`. The header `If-None-Match: *` means "create only -- do not replace an existing file".	`201` created · `200` replaced · `412` exists (create only) · `405` read-only · `413` too large
`DELETE {base}/files/{root}/{path}`	Delete a file	`200` · `404` · `409` it is a folder · `405`
`PUT {base}/folders/{root}/{path}`	Create a folder, and any missing parent folders. The body is empty.	`201` created · `200` already a folder · `409` a file is in the way · `405`
`DELETE {base}/folders/{root}/{path}?recursive=0` or `?recursive=1`	Delete a folder; with `recursive=1`, everything in it as well	`200` · `404` · `409` not empty (with `recursive=0`) · `405`
`POST {base}/copy/{root}/{path}?to={root}/{path}&overwrite=0` or `&overwrite=1`	Copy a file, or a folder and everything in it	`201` new or `200` replaced, with the JSON body `{"copied": «n»}` · `404` · `412` the destination exists (with `overwrite=0`) · `409` a file/folder clash · `405` · `501`
`POST {base}/move/{root}/{path}?to={root}/{path}`	Move or rename a file or a folder. Never overwrites.	`200` · `404` · `412` the destination exists · `405` · `501`

The to parameter is percent-encoded exactly like the path.
A folder is whatever /list reports with isFolder. An object store can represent an empty folder with a placeholder object, provided /list shows the folder and never the placeholder.
405 means read-only, and it may apply to the whole store or to part of it -- one root, say. For a move, the source must be writable as well as the destination; for a copy, only the destination.
501 from /copy or /move means "not possible on the server" -- a copy between two roots that live in different back ends, for example. For a single file, the engine then copies it itself with GET and PUT (and, for a move, DELETE). A move whose source cannot be deleted is undone by deleting the copy it made.
Refuse .. segments, and a bare {root} as the target of any of these requests.

Implementers:

Check the write policy before reading a PUT body, and drain any unread body before sending an early error reply. Some proxies reset the connection otherwise, and the user then sees a network error instead of your status.
Apply your access rules to both the path and the to path of a copy or move.
A service that predates these endpoints leaves the functions that need them failing with the repo:// file store does not support this operation (the store service may need to be updated).

Sizes, timeouts and cancellation

The engine aborts a fetch as soon as it can see that the transfer exceeds FileStoreMaxMB (256 MB by default), whether it learns that from Content-Length or from the bytes received so far. Send a Content-Length where you can, so that an oversized file is refused before it is transferred rather than during.
The connect timeout is 5 seconds and is not configurable. The read timeout is FileStoreTimeoutS, 60 seconds by default. A service that has slow work to do before it can produce a file should still start responding promptly.
Streaming or chunking a large body is fine, and is preferable to buffering the whole thing in the service.
A user can cancel a transfer that is in progress. From the service's point of view that looks like the client closing the connection part-way through a response, which the service should handle gracefully.

HTTP only

The engine's HTTP client has no TLS support, so the base URL must be http://; an https:// URL is refused when the provider is registered. Bind the service to 127.0.0.1, or to a private network or cluster network only. The bearer token and the file bytes are both in the clear over that hop. See Custom file system providers#Security.

An illustrative skeleton

The smallest thing that works, in Python, serving a couple of directory trees. It is here to show the shape of the three read endpoints and of the listing JSON; it implements none of the write endpoints, and it is not something to deploy. It uses Python's single-threaded development server, and its traversal checking is only the obvious one.

import datetime, json, os, urllib.parse
from http.server import BaseHTTPRequestHandler, HTTPServer

ROOTS = {"Sales": "/srv/data/sales", "models": "/srv/data/models"}
TOKEN = "«bearer-token»"

def resolve(rest):                          # "Sales/2026/Q3.csv" -> a local path
    parts = [p for p in urllib.parse.unquote(rest).split("/") if p]
    if not parts or ".." in parts or parts[0] not in ROOTS:
        return None
    return os.path.join(ROOTS[parts[0]], *parts[1:])

def entry(dirpath, name, rel):
    full = os.path.join(dirpath, name)
    isdir = os.path.isdir(full)
    when = datetime.datetime.utcfromtimestamp(os.path.getmtime(full))
    return {"name": rel + name, "isFolder": isdir,
            "size": 0 if isdir else os.path.getsize(full),
            "modified": when.strftime("%Y-%m-%dT%H:%M:%SZ")}

class Store(BaseHTTPRequestHandler):
    def reply(self, code, body=b"", ctype="application/octet-stream"):
        self.send_response(code)
        self.send_header("Content-Type", ctype)
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        if self.command != "HEAD":
            self.wfile.write(body)

    def do_GET(self):                       # HEAD arrives here too
        if self.headers.get("Authorization") != "Bearer " + TOKEN:
            return self.reply(401)
        url = urllib.parse.urlparse(self.path)
        kind, _, rest = url.path.lstrip("/").partition("/")
        target = resolve(rest)
        if target is None:
            return self.reply(404)
        if kind == "files" and os.path.isfile(target):
            with open(target, "rb") as f:
                return self.reply(200, f.read())
        if kind == "list" and os.path.isdir(target):
            recurse = urllib.parse.parse_qs(url.query).get("recursive") == ["1"]
            out = []
            for dirpath, dirs, files in os.walk(target):
                rel = os.path.relpath(dirpath, target).replace(os.sep, "/")
                rel = "" if rel == "." else rel + "/"
                out += [entry(dirpath, n, rel) for n in dirs + files]
                if not recurse:
                    break
            return self.reply(200, json.dumps(out).encode(), "application/json")
        return self.reply(404)

    do_HEAD = do_GET

HTTPServer(("127.0.0.1", 8080), Store).serve_forever()

Testing your provider

Write a configuration file that points at the service and gives it a scheme: a FileStore repo = http://127.0.0.1:8080 line, plus FileStoreToken repo = «bearer-token» if the service wants one.
Make Analytica find that file -- with /stores:«path», with the FileProvidersConfig registry value, or by putting it beside the engine binary. See Where Analytica looks for FileProviders.config.
Start Analytica and look for the registration line, [FileProviders] repo:// -> http://127.0.0.1:8080. It is written to the engine's typescript: the typescript window in Analytica, the log in ADE, ADEW and ACP, and stderr -- the server log -- in AMP. If it is not there, the configuration file was not found, or the line in it was not understood.
List the root with FileSystemListing( "repo://" ). That exercises /list and your JSON shape without needing any file to be readable.
Read something with ReadTextFile( "repo://Sales/Q3.csv" ). That exercises /files, including the HEAD existence check that runs first, and FileExists( "repo://Sales/Q3.csv" ) exercises the HEAD endpoint on its own.
If the service accepts writes, work in a scratch folder: WriteTextFile( "repo://Sales/scratch/a.txt", "hello" ), then FileSystemNewFolder, FileSystemCopy, FileSystemMove and FileSystemDelete on it, checking each result in the service's own storage rather than through Analytica. Try a write to a location that should be read-only, and expect the repo:// file store is read-only and refused the write (HTTP 405).
Then check the failure paths deliberately: a name that does not exist (expect a plain not-found), a bad token (expect a credentials error naming the scheme), and the service stopped (expect could not reach the repo:// file store).