Custom file system providers/API spec

New in Analytica 7.2. See also the parent page, Custom file system providers.

Overview

A store service is an ordinary HTTP service that Analytica calls whenever a model uses a «scheme»:// path. It implements three endpoints, and nothing else is required. No Analytica code is involved, so write the service in whatever language and framework you like, and back it with whatever you like: object storage, a database, a version control system, a document management system, or a REST API of its own.

Analytica is told about the service by one line in FileProviders.config:

FileStore repo = http://mystore.internal:8080

after which every path beginning repo:// becomes a request to that base URL. See Custom file system providers for the rest of the configuration.

The endpoints

All three are relative to the configured base URL, including any path prefix in it. With a base URL of http://mystore.internal:8080/api, the first endpoint is http://mystore.internal:8080/api/files/....

Request	Purpose	Success	Not found
GET {base}/files/{root}/{path}	Fetch a file	200, with the raw file bytes as the body	404
HEAD {base}/files/{root}/{path}	Existence check, without transferring the file	200	404
GET {base}/list/{root}/{path}?recursive=0 or ?recursive=1	List a folder	200, with a JSON array (see below)	404, taken as an empty listing

{root}/{path} is everything that followed «scheme»:// in the model's path. For repo://Sales/2026/Q3.csv the engine requests {base}/files/Sales/2026/Q3.csv. The engine attaches no meaning of its own to {root}: it is simply the first segment. It is there so that one service can host several logical roots -- one per customer, per project or per bucket -- and decide from it which backing store to consult.

The recursive query parameter is sent on every /list request and on no /files request.

A health endpoint is a good idea for your own monitoring, but the engine never calls one and does not require it.

Path encoding

Everything after «scheme»:// is the literal path -- unencoded text, as the model author wrote it. Before sending a request, the engine:

1. replaces every \ with /, and
2. percent-encodes every byte that is not in the unreserved set A-Z a-z 0-9 - _ . ~ /.

Your service must percent-decode the path. Because the encoding is applied byte by byte to the UTF-8 form of the name, spaces and non-ASCII characters round-trip correctly: repo://models/Ventas Q3.csv arrives as /files/models/Ventas%20Q3.csv.

The engine refuses a .. segment before any request is sent, so a well-behaved client never asks for one. Reject traversal in the service as well; defence in depth is worth the three lines it costs.

Authentication

When a token is configured, with FileStoreToken or FileStoreTokenFile, the engine sends

Authorization: Bearer «token»

on every request, to all three endpoints. When no token is configured, no Authorization header is sent at all.

Answer 401 or 403 when the token is missing, expired or wrong. Analytica turns either one into an error that names the scheme, so the user is told the credentials were rejected rather than that the file is missing.

How the engine interprets your status codes

Status	What the engine does
200	Success. The body is the file, or the listing.
404	Not found. Treated as a missing file, with no error text of its own, so the caller reports a missing file in its usual way. On /list, an empty listing.
401 or 403	A credentials error naming the scheme: the repo:// file store rejected the credentials (HTTP 401).
anything else	HTTP «n» from the repo:// file store.

A connection that cannot be made at all, or that times out, is reported as could not reach the repo:// file store (...), with the transport reason in the parentheses. Neither of those is a status code you control, so return a real status for everything you can: the message the user sees is then about your service rather than about the network.

The listing response

GET {base}/list/{root}/{path} returns a JSON array of objects, one per entry. A body that does not parse, or that is not a JSON array, is an error.

Member	Type	Meaning
name	string	The entry name only, with no path. Required -- an entry with no name, or whose name is not a string, is skipped.
isFolder	boolean or number	true, or a non-zero number, for a folder. Absent or false means a file.
size	number	Size in bytes. Use 0 for folders.
modified	string	The last-modified time in ISO-8601, UTC, for example "2026-08-14T09:12:03Z". Omit it when you do not know it.

Members other than these are ignored, so you may return extra information for your own clients.

[
  {"name": "Q1.csv",  "isFolder": false, "size": 20481, "modified": "2026-04-02T17:45:10Z"},
  {"name": "Q2.csv",  "isFolder": false, "size": 21118, "modified": "2026-07-01T08:03:55Z"},
  {"name": "archive", "isFolder": true,  "size": 0}
]

recursive=1 asks for the whole subtree below the folder. When you answer one, name should carry the path relative to the folder that was listed -- "archive/2025/Q4.csv", not just "Q4.csv". That is what lets FileSystemListing report the sub-path the way it does for a local recursive listing. With recursive=0, return only the immediate children.

The engine, not the service, applies any wildcard the model wrote in the last segment of the path, and applies the «files» and «folders» flags of FileSystemListing. Return the folder's whole contents and let it do that filtering.

Sizes, timeouts and cancellation

• The engine aborts a fetch as soon as it can see that the transfer exceeds FileStoreMaxMB (256 MB by default), whether it learns that from Content-Length or from the bytes received so far. Send a Content-Length where you can, so that an oversized file is refused before it is transferred rather than during.
• The connect timeout is 5 seconds and is not configurable. The read timeout is FileStoreTimeoutS, 60 seconds by default. A service that has slow work to do before it can produce a file should still start responding promptly.
• Streaming or chunking a large body is fine, and is preferable to buffering the whole thing in the service.
• A user can cancel a transfer that is in progress. From the service's point of view that looks like the client closing the connection part-way through a response, which the service should handle gracefully.

HTTP only

The engine's HTTP client has no TLS support, so the base URL must be http://; an https:// URL is refused when the provider is registered. Bind the service to 127.0.0.1, or to a private network or cluster network only. The bearer token and the file bytes are both in the clear over that hop. See Custom file system providers#Security.

An illustrative skeleton

The smallest thing that works, in Python, serving a couple of directory trees. It is here to show the shape of the three endpoints and of the listing JSON; it is not something to deploy. It uses Python's single-threaded development server, and its traversal checking is only the obvious one.

import datetime, json, os, urllib.parse
from http.server import BaseHTTPRequestHandler, HTTPServer

ROOTS = {"Sales": "/srv/data/sales", "models": "/srv/data/models"}
TOKEN = "«bearer-token»"

def resolve(rest):                          # "Sales/2026/Q3.csv" -> a local path
    parts = [p for p in urllib.parse.unquote(rest).split("/") if p]
    if not parts or ".." in parts or parts[0] not in ROOTS:
        return None
    return os.path.join(ROOTS[parts[0]], *parts[1:])

def entry(dirpath, name, rel):
    full = os.path.join(dirpath, name)
    isdir = os.path.isdir(full)
    when = datetime.datetime.utcfromtimestamp(os.path.getmtime(full))
    return {"name": rel + name, "isFolder": isdir,
            "size": 0 if isdir else os.path.getsize(full),
            "modified": when.strftime("%Y-%m-%dT%H:%M:%SZ")}

class Store(BaseHTTPRequestHandler):
    def reply(self, code, body=b"", ctype="application/octet-stream"):
        self.send_response(code)
        self.send_header("Content-Type", ctype)
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        if self.command != "HEAD":
            self.wfile.write(body)

    def do_GET(self):                       # HEAD arrives here too
        if self.headers.get("Authorization") != "Bearer " + TOKEN:
            return self.reply(401)
        url = urllib.parse.urlparse(self.path)
        kind, _, rest = url.path.lstrip("/").partition("/")
        target = resolve(rest)
        if target is None:
            return self.reply(404)
        if kind == "files" and os.path.isfile(target):
            with open(target, "rb") as f:
                return self.reply(200, f.read())
        if kind == "list" and os.path.isdir(target):
            recurse = urllib.parse.parse_qs(url.query).get("recursive") == ["1"]
            out = []
            for dirpath, dirs, files in os.walk(target):
                rel = os.path.relpath(dirpath, target).replace(os.sep, "/")
                rel = "" if rel == "." else rel + "/"
                out += [entry(dirpath, n, rel) for n in dirs + files]
                if not recurse:
                    break
            return self.reply(200, json.dumps(out).encode(), "application/json")
        return self.reply(404)

    do_HEAD = do_GET

HTTPServer(("127.0.0.1", 8080), Store).serve_forever()

Testing your provider

1. Write a configuration file that points at the service and gives it a scheme: a FileStore repo = http://127.0.0.1:8080 line, plus FileStoreToken repo = «bearer-token» if the service wants one.
2. Make Analytica find that file -- with /stores:«path», with the FileProvidersConfig registry value, or by putting it beside the engine binary. See Where Analytica looks for FileProviders.config.
3. Start Analytica and look on stderr for the registration line, [FileProviders] repo:// -> http://127.0.0.1:8080. If it is not there, the configuration file was not found, or the line in it was not understood.
4. List the root with FileSystemListing( "repo://" ). That exercises /list and your JSON shape without needing any file to be readable.
5. Read something with ReadTextFile( "repo://Sales/Q3.csv" ). That exercises /files, including the HEAD existence check that runs first.
6. Then check the failure paths deliberately: a name that does not exist (expect a plain not-found), a bad token (expect a credentials error naming the scheme), and the service stopped (expect could not reach the repo:// file store).

See Also

• Custom file system providers -- the parent page: what providers are, and how to configure them
• Analytica Command Line -- the /stores:«path» option
• FileSystemListing
• ReadTextFile
• ReadBinaryFile
• Category:File system functions